So in Part 1 I showed how you can write over the function pointer of a block with another function pointer and the block will still work. It’s probably also possible to construct a block from scratch, but I don’t really see any benefit to that.

So, how to make a C block that will call a Nu block? Fortunately most of the work is already done; if you look into the Nu source code in the objc/bridge.m file you can see how Nu is rigged it up so that Objective-C code can call methods implemented in Nu. The magic happens with the help of libffi (Foreign Function Interface) a library that, among other things, allows one to construct C functions at runtime with arbitrary names and argument signatures. When one of these constructed functions gets called, libffi in turn calls a callback that had been specified earlier. This is that callback in Nu; all message sends from Objective-C to Nu pass through this function:

static void objc_calling_nu_method_handler(ffi_cif* cif, void* returnvalue, void** args, void* userdata)

The first argument is a libffi struct previously generated by the user (in Nu this happens in construct_method_handler()) that contains information about the function that libffi was to generate: return type, argument count and types, etc. The second argument is a pointer to where the return value should be written so that libffi can return it to the original caller. The third argument is an array of the argument values and the fourth is user-defined info. In Nu this fourth argument points to an array of pointers where the first element is the Objective-C type encoding of the return value (a c string), the second is the Nu implementation of the method (as a NuBlock) and the remaining elements are the Objective-C type encodings of the arguments (more c strings).
In short, all of the information that the Nu runtime needs to pass control to the Nu method are there.

So what I did was copy all of this code and tweak it a little bit. Instead of having libffi make a function fit to serve as an Objective-C method (i.e. with an argument list like this: “id sender, SEL __cmd, …” (where the first two arguments are the message recipient and the selector of the message being sent, respectively) I had it make one like this: “id block, …” (recall from part 1 of this post that block functions take as a first, hidden argument a pointer to the associated block object). So in my code all calls to Objective-C blocks implemented in Nu pass through this function:

static void objc_calling_nu_block_handler(ffi_cif* cif, void* returnvalue, void** args, void* userdata)

This function in turns just calls the appropriate NuBlock (stored in element two of userdata).
So all that’s left is to make a dummy Objective-C block and write the pointer to the libffi generated function into it. Here is the code that does that:

static id make_cblock (NuBlock *nuBlock, NSString *signature)
{
	void *funcptr = construct_block_handler(nuBlock, [signature UTF8String]);
	int i = 0xFFFF;
	void(^cBlock)(void)=[^(void){printf("%i",i);} copy];
	#ifdef __x86_64__
	//2*(sizeof(void*)) = 0x10
	*((void **)(id)cBlock + 2) = (void *)funcptr;
	#else
	//3*(sizeof(void*)) = 0xc
	*((void **)(id)cBlock + 3) = (void *)funcptr;
	#endif
	return cBlock;
}

Why that variable i and why the printf()? All that jazz does is make sure the compiler generates a distinct block object every time that code is executed. If it thinks it can get away with it, it will reuse previously created blocks just like it may reuse string literals (try running this code: id a=@”Hi”, b=@”Hi”;printf(“%lx,%lx”,a,b);)
Well the printf doesn’t do anything and will never get called; it’s just an arbitrary expression using the i variable. What’s important is for the block to reference something from the surrounding context, something mutable that will be copied by value. If it does not reference a variable from the surrounding context, or if that variable has the __block qualifier (meaning that it should not be copied), then the compiler will reuse the same block object over and over again.

So that’s how it works. But wait, there is a hitch. The problem is that there appears to be a bug in the Objective-C runtime. Nu uses the runtime functions method_getNumberOfArguments() and method_getArgumentType() in the code that handles message sends from Nu to Objective-C. But these functions (at least as of 10.6.2) return incorrect values for any method expecting a block argument. The encoding type for blocks appears to be “@?” but those runtime functions (which I believe call method_getTypeEncoding and parse the string returned by that) treat that encoding as two separate encodings. Unless this gets fixed in 10.6.3 (my bug report has gone unanswered so far) I’m going to have to write some substitutes for those functions or make a patched version of libobjc.A.dylib. Well, I’m hoping I’ll have time to finish this up soon, and then maybe see if Tim Burks is interested in merging this into the main Nu branch.

For the time being, the version of Nu at my github repository (see sidebar) replaces the above-mentioned runtime functions with calls to similar NSMethodSignature methods. Unfortunately, this seems to raise other more mysterious problems in Nu. But it is useable enough to screw around with.

An Objective-C block can be created by sending the following message to the class NuBridgedBlock:

+(id)cBlockWithNuBlock:(NuBlock*)nb signature:(NSString*)sig

where sig is a string containing the Objective-C type encodings for the return value and arguments. This is a little cumbersome and I plan to implement a convenience macro, maybe something like this:

(cblock int ((id) arg1 (NSRect) arg2) (--nu code here--))

Anyway, I have successfully called various Cocoa methods that expect blocks as well as libdispatch functions from Nu. I just need to work out the kinks and I think it might be pretty useful!

When my brother wrote ATT he was pretty good about not using Cocoa where he didn’t need to meaning that the code base looks to be pretty portable. I’m considering creating a version of it using wxWidgets for the UI bits. Then I could release a version for Windows and possibly FreeBSD/Linux, though I’m not sure how well shareware does on the free *nixes. Most likely I’ll continue using Cocoa for the Mac OS X interface, just to make sure it always looks 100% native.

I’ll do some posting on wxWidgets as I get into. It should be interesting, especially since I haven’t done much in the way of C++ programming since my teens.

Top priority is making ATT 100% Snow Leopard compatible. It works on Snow Leopard, but the text does not highlight to show you where you are in a word. Fixing that, plus a little spit and polish, and it’ll be ready for a new release.

We also have big plans for expanding ATT; things my brother has wanted to do for years but hasn’t had the time. Here’s a peek: options for different keys (right hand/left hand/home keys/etc…), more advanced logging of scores and maybe some others. Of course, ATT is a pretty awesome program and I don’t want to overwork it into something less awesome.

Please send suggestions if you have them!

Stay tuned: there may be a release as early as this weekend.

So anyway, I’m going to a VPS. And since Linux is just way too popular (and I have a mild philosophical disagreement with the GNU license), I decided that I had to have FreeBSD. RootBSD seems to be one of a handful of places offering FreeBSD VPSes. After about 24 hours, I am absolutely thrilled with what I’ve got.

Now, to the subject of this post. Living deep in the boonies, I use satellite for all my internet access. Overall I’m quite pleased with the service, though I wish it cost about a 1/3 of what it does. The major downside is the latency: pings to my VPS are running between 700 and 1110 ms. This makes ssh very difficult to use: press a key, it takes about a second for it register on the terminal. My solution has been to use putty (PuTTY, if you insist) with local echo and local line editing enabled. The local line editing tends to cause problems with anything other than the command line (emacs, ports install screens, etc.), so I’ve been keeping both putty and ssh from Terminal.app open whenever I’m working on the VPS. Together, they make for a useable setup.

So, if you’re in my boat, give putty a try. It kind of feels like a Windows 3.1 app and it never seems to save settings, but I can’t complain.

A quick description of blocks for those who don’t know what they are: also called closures, anonymous functions or lambda functions, blocks are a means of creating a chunk of code inside of other code. The power of blocks is that they can reference context information (i.e. local variables) in the surrounding code.

Blocks are common in high-level languages but until now there had been no mainstream implementation of them in the C family of languages. Here is a simple example of an Objective-C block in action copied from one of my projects:

NSMutableAttributedString *fixedText=[unfixedText mutableCopy];
__block id oldAttrName, newAttrName;
__block int checked,fixed;
id attrBlock = ^(id value, NSRange range, BOOL *stop)
{
	if (value == nil)
		return;
	checked++;

	NSRange goodRange = [plainText rangeOfComposedCharacterSequencesForRange:range];
	if (!NSEqualRanges(range, goodRange))
		fixed++;

	[fixedText removeAttribute:oldAttrName range:range];
	[fixedText addAttribute:newAttrName value:value range:goodRange];
};
//verse
oldAttrName = @"HebrewVerseAttribute",
newAttrName = @"HRVerse";
checked=0;
fixed=0;
[unfixedText enumerateAttribute:oldAttrName
			inRange:NSMakeRange(0, [unfixedText length])
			options:0
		    usingBlock:attrBlock];
printf("\nChecked %i verse attributes\nFixed %i\n",checked,fixed);

//segment/sentence
oldAttrName = @"HebrewSentenceAttribute",
newAttrName = @"HRSegment";
checked=0;
fixed=0;
[unfixedText enumerateAttribute:oldAttrName
			inRange:NSMakeRange(0, [unfixedText length])
			options:0
		    usingBlock:attrBlock];
printf("\nChecked %i sentence attributes\nFixed %i\n",checked,fixed);

I like this code snippet because it uses a Cocoa method that takes blocks and because it does something that would have been a bit more tedious without blocks. I’m not going to step through the whole thing but here are a few quick notes. Note the ‘__block’ keyword. If you don’t use this keyword a block will still have access to that variable, but only via a copy of it. Hence any changes made to that variable after the block is created will not be reflected in the block. Also, the compiler will raise an error if code in the block tries to change the value of the variable.

Be aware that blocks are Objective-C objects; you might notice I use a variable of type ‘id’ to store the block reference. This is convenient and fine for this situation but you should note that you cannot call a block unless it is specifically cast as a block type. So if I wanted to directly call that block above rather than just passing it on to Cocoa to call, I would have declared the variable like so:

void(^attrBlock)(id,NSRange,BOOL*) = …

You can see why I used ‘id’. It also happens to be the case that the compiler doesn’t like you casting a block to an arbitrary pointer type (e.g. void*, int*, etc), but it has no problem with you first casting it to ‘id’ and then to some other pointer type; a useful thing to know if you want to hackishly write over parts of your block…

So what happens when you call a block? Here is the disassembly of a block call in some x86-64 code:

mov    %rax,-0x18(%rbp)		 //the pointer to the block object is in rax
mov    -0x18(%rbp),%rax
mov    0x10(%rax),%rax		//the pointer to the block function is at +0x10 into the block object
mov    -0x18(%rbp),%rdi		//the first argument (this example has no others)
					//is always the pointer to the block object
callq  *%rax

Ah! So the block contains a function pointer and when that function is called, the pointer to the block is passed as the first argument. So let’s run a little test:

void foo(id block, int x)
{
	printf("thrice %i is %i\n",x, 3*x);
}

int main (int argc, const char * argv[])
{
	void(^b)(int) = ^(int x){printf("twice %i is %i\n",x,2*x);};
	b(5);
	*((void**)(id)b+2) = (void*)foo;
	b(5);
};

Guess what this prints out? Cool, huh?

So what’s this *((void**)(id)b+2) monkey business? Well we are casting b to type ‘id’ and then to type ‘void**’. We then offset the pointer by 2 units, remembering that in C when you add n to a pointer it actually adds n times the size of the type the pointer points to. A ‘void**’ points to a ‘void*’ which on x86-64 is 8 bytes long. Twice that is 16 or 0×10 bytes, the correct offset to the function pointer inside of the block object. So I just took another function pointer, making sure it takes the correct arguments, and wrote that into the correct place in the block. And the block still works! Amazing!

So maybe you see where I’m going with this. At the beginning of this post I mentioned that I was trying to bridge Nu blocks and Obj-C blocks. What that means exactly is that I want to be able to use the new Cocoa methods that take Obj-C blocks and call them from Nu code, passing them Nu blocks. So I basically need to generate Obj-C blocks at runtime that will call Nu blocks. But it’s not as simple as it sounds at first. The generated Obj-C blocks need a signature (parameter list) that matches what the Nu blocks expect.

Next post I’ll show you how I did this, mainly by reusing Tim Burks’ code that allows C code to call Nu methods.

Stay tuned for more!

UPDATE:
Part 2!

Pretty slick, I thought to myself. Then I started thinking about how you would probably want it wired to a switch to prevent the power vampirism that afflicts most AC/DC converters. And then maybe you’d want to rig up a little LED in the outlet to help you remember if it’s on or not (this picture makes it look like it might have an LED in upper right corner). But no, according to FastMac’s pre-order page it is NOT a vampire and ensuring that it wasn’t was a high priority of theirs.

Kudos to FastMac for thinking it out and building what looks to be a solid product. By the way, it looks like it costs $19.95, though Macworld was reporting it at half that. Still not a bad price.

I began to suspect that the problem was caused by some custom attributes I was assigning to the text to mark verses and segments. In other words I was using NSMutableAttributedString’s addAttribute:value:range and the like to assign my own keys and values to bits of text.

Apple’s documentation says:
You may assign any name/value pair you wish to a range of characters, in addition to the standard attributes described in the [Apple docs.]

Nevertheless I suspected that my custom attributes were causing these layout problems. So I wrote a little test application that basically goes through all of the text displayed in an NSTextView and applies an arbitrary attribute to each uncomposed character. So vowel markings get their own attributes separate from the letters they are attached to.

NSTextStorage *ts = [textView textStorage];

NSRange r = {0,1};

for (;r.location+r.length<=[ts length]; r.location++)

{

[ts addAttribute:[NSString stringWithFormat:@"Nonsense%u",r.location]

  value:[NSString stringWithFormat:@"Gibberish%u",r.location]

  range:r];

}

This code converts this:

into this:

So I posted a question to the cocoa-dev mailing list and promptly got an answer back from Douglas Davidson, a developer at Apple whose name pops up a whole lot in connection with the Cocoa text system. He suggested I try applying the attributes to composed character ranges only. That is, making sure the ranges include both the characters and all of the markings that modify them.
Apple supplies the following two NSString instance methods to help the developer find what the ranges of composed character sequences are:
- (NSRange)rangeOfComposedCharacterSequenceAtIndex:(NSUInteger)anIndex

- (NSRange)rangeOfComposedCharacterSequencesForRange:(NSRange)range

Sure enough, making sure that attribute ranges only contain whole composed character sequence ranges solves the problem. But is this correct behavior on the part of the text system? Probably not, especially for custom attributes that shouldn’t affect layout anyway (like my verse markings). In fact, Mr. Davidson suggested I file a bug report on this issue. (Which I will do soon…) But he also suggested that a good rule of thumb is to treat the composed character sequences as the actual characters, since that it was the user will perceive to be a single character.

Welcome to the new Rivulus Software blog! Here we will be discussing programming topics (in particular Cocoa related), Mac OS X issues in general, and of course Rivulus news! I hope that users and developers will find some useful info on this blog and post some interesting comments too. Stay tuned!