NSSpellChecker, heap(1), and faking ispell(1) in 89 lines on Mac OS

There's an interesting dilemma when you use Java to write cross-platform software, because there are two conflicting notions of what "cross-platform" means.

It could mean that your program looks and works the same on each system. This might be good for someone who has to use the same program on a variety of systems, but only really has to use that one program.

It could also mean that your program looks and works as much like a native program on each platform as possible. This is probably better for people who also use other software on the various systems.

My editor uses ispell(1) to mark misspelled words, as a word processor might. On Linux, I can use apt-get(1) to install it. On Mac OS, I can't. I did try building from source, but it doesn't build out of the box on Mac OS 10.3, and as I was wondering what to do I realized that I don't really want ispell(1) anyway. I want my editor to use the same spelling checker that all the other programs on the system use.

So I knocked up a little program to imitate ispell(1) using use NSSpellChecker. It's 32 lines of C++ and 57 lines of Objective C. I couldn't use Objective C++ because my attempts to bootstrap GCC 4 killed my Objective C++ compiler. The C++ is the main loop, pretending to be ispell -a, and the Objective C takes care of talking to NSSpellChecker.

I haven't written much Objective C, and I still don't find its approach to memory management particularly natural, so I made two mistakes in this program, and that's what this post is about.

The Leak
The first mistake was one I assumed I'd made. I assumed there would be a leak, because there always is. I knew that objects returned by class methods are autoreleased, and I hadn't used anything else, so I couldn't see where the leak would be, but I was sure there would be one. To find out, I used heap(1). It's the command-line cousin of ObjectAlloc.app, whose praises I've sung in the past.

Given the process id of NSSpell, I could pipe the output of heap(1) to diff(1) or grep(1) and see what was going on.

hydrogen:~$ heap 16951 | grep String
NSCFString = 951 (27584 bytes)
NSAttributedString = 1 (16 bytes)
NSConcreteMutableAttributedString = 1 (16 bytes)
NSConstantString = 1 (16 bytes)
NSMutableString = 1 (16 bytes)
NSDebugString = 1 (16 bytes)
%NSCFString = 1 (16 bytes)
NSPlaceholderString = 1 (16 bytes)
NSPlaceholderMutableString = 1 (16 bytes)

Watching the count of NSCFString instances, I could see I was leaking them. (As an aside, I've no idea what the "%NSCFString" instance is, and Google doesn't seem to know either.)

Thinking about it, I didn't know where or how the NSAutoreleasePool gets cleaned up. Turns out that if you don't use the usual event loop, you're supposed to release NSAutoreleasePools yourself (there doesn't seem to be a public way to reuse them).

I wanted to use boost::scoped_ptr<NSAutoreleasePool>, but even with Objective C++ that isn't an option. Damn you, Apple! Anyway, with a suitable hack in place, my leak was plugged.

The Double Free
Everything seemed fine until I was actually using it, when all of a sudden my editor hung. I killed the NSSpell process so I could finish what I was doing, and didn't have chance to come back to the problem until just now.

I just typed random junk into my editor until it hung, and then I attached gdb(1) to see where we were:

(gdb) bt
#0 0x90010664 in write ()
#1 0x9009e2c0 in malloc_printf ()
#2 0x90001aac in szone_free ()
#3 0x90190d30 in CFRelease ()
#4 0x909f15d4 in NSPopAutoreleasePool ()
#5 0x00003710 in NSSpellChecker_showSuggestions (word=0x130afc "slkjdfhskjfskdjf") at NSSpellChecker.m:29
#6 0x00003300 in main ()
#7 0x00002de0 in _start (argc=1, argv=0xbffffc14, envp=0xbffffc1c) at /SourceCache/Csu/Csu-47/crt.c:267
#8 0x8fe1a558 in __dyld__dyld_start ()

This suggested that the malloc library was detecting a problem during the destruction of the NSAutoreleasePool. It was hanging trying to write its diagnostic. Why? Because it was connected to the editor's pipe, and the editor wasn't seeing what it expected. Killing NSSpell causes an IOException, and the editor gives up talking to the spelling checker and carries on as normal. Minus spelling checking.

Running NSSpell from the terminal, we get to see the complaint:

*** malloc[22264]: error for object 0x130a40: Double free

And, sure enough, I was calling autorelease on an object returned from a class method. I know consciously not to do that, but I don't know it intuitively. I don't understand why a seemingly simple rule doesn't sink in. Perhaps because I do very little Objective C, with such long intervals in between, and during those intervals I'm writing Java and C++, with two different, simpler solutions of their own.

The syntax may be cool, but Objective C's memory management sucks compared to C++ (with boost, for when you have to use pointers) or Java.

Something I found particularly interesting here is that it's actually harder to debug a problem with programs communicating via a pipe than when they're talking through a socket. In the latter case, you have Ethereal, but there's nothing as convenient for pipes.

I guess this is the future.

Apart from the manual memory management crap.

(Calling Objective C's solution semi-automatic is misleading. Java is semi-automatic, if you consider the thought that goes in to avoiding excessive allocation and unwanted heap retention. C++ is part-automatic, if you consider the strong need for discipline, and the extra care needed if you're using pointers. Objective C is only automatic if your objects have no fields. On the bright side, the tools are pretty good.)

Anyway, the working version of the fake ispell(1) is checked in to salma-hayek, available off the Edit page.