2006-05-31

When AppKit methods return nil...

...things fall apart very quietly.

I mentioned my little "NSSpell" hack a couple of years ago in NSSpellChecker, heap(1), and faking ispell(1) in 89 lines on Mac OS. I and others have been using it ever since for as-you-type spelling checking in Java programs.

A year ago, I decided to "fix" NSSpell so that Spin Control.app didn't claim that it was hung all the time. I talked about that in Spin Control and non-GUI NSApplication programs, where I presented a version of the code that, it turns out, contained a race condition.

That post was dated 2005-05-23. It's now just a few days past the problem's first birthday, and I'm back with a fix. That's not to say that this was an especially hard problem; it wasn't. I just didn't previously realize how much more painful this is on a single-processor machine until last week. Such differences being part of the nature of race conditions.

The symptom was this: every now and again I'd start a program that uses our PTextArea text component, and the check-as-you-type spelling checking would report every word as misspelled. Run it again, and it would be fine.

If you've never written native code for Mac OS, you might not know that Apple's documentation (like most documentation) is vague, incomplete, and lacking both as tutorial and as reference; their sample code (like most sample code) is often broken, uninformative, or badly-written; and worst of all (unlike Java, GTK+, the STL, the Linux kernel, Firefox, Ethereal, and all the home-grown stuff I use) you don't get to examine the source.

I had to bite my tongue recently during the storm in a teacup over whether the Darwin/x86 kernel source is available or not. A guy from the "Apple is always right" side of the fence claimed that we shouldn't care because the source wasn't a very useful thing to have anyway. A kernel extension writer corrected him, saying "You must not write kernel extensions for a living [to believe that]", at which point the apologist made me laugh out loud by claiming that "[having] source is just treating a symptom of inadequate docs, not the real issue".

"Obviously not a golfer", as the dude would say.

Java may have lots of breakage (it's a large codebase, so of course it does), but you do get to read the source. Heaven knows how much time having src.zip has saved me. Cocoa, though it's sometimes much better put-together, falls apart when it doesn't "just work". And here's an example.

My original code looked like this:

int main(int, char*[]) {
ScopedAutoReleasePool pool;

// Pretend to be ispell in a new thread, so we can enter the normal event
// loop and prevent Spin Control from mistakenly diagnosing us as a hung
// GUI application.
// It seems to be important that we invoke an instance method rather than
// a class method.
Ispell* ispell = [[Ispell alloc] init];
[NSThread detachNewThreadSelector:@selector(ispellRunLoop:) toTarget:ispell withObject:nil];

[[NSApplication sharedApplication] run];
}

If you look at Apple's documentation for NSApplication, you'll see that they claim Xcode automatically generates a main function like this:

void NSApplicationMain(int argc, char *argv[]) {
[NSApplication sharedApplication];
[NSBundle loadNibNamed:@"myMain" owner:NSApp];
[NSApp run];
}

Their description of sharedApplication says "Your program should invoke this method as one of the first statements in main(); this invoking is done for you if you create your application with Xcode. To retrieve the NSApplication instance after it has been created, use the global variable NSApp or invoke this method". The important point to notice there is that there's a global variable cache of the return value, but it isn't necessarily initialized, and it isn't checked either. And our accidental experiment suggests that there's plenty of Apple code that uses this rather than the safer method.

I say "safer" rather than "safe" because there's no indication that the method's thread-safe, and we can't see the source so we'll have to assume that it isn't.

Much though the global disgusts me, I now use it in NSSpell, both to be closer to the official Xcode idiom (the only thing guaranteed to work) and to hopefully remind myself that Apple have left broken glass on the lawn.

It turns out that if you haven't invoked [NSApplication sharedApplication] before you invoke [NSSpellChecker sharedSpellChecker] the latter will return nil. The documentation, as you've guessed, doesn't mention this, and, as you'll recall, we don't have the source. (Judging by a WebKit commit, this was a problem in Safari too. Which suggests that Safari either has a similar race condition, which seems unlikely, or there are other circumstances in which sharedSpellChecker can return nil. Not only do Apple keep their source secret, they keep their bugs secret, so we can't see what the WebKit committer was trying to fix, even though we know the Apple bug number.)

At this point, if you're not an Objective C programmer, you're probably wondering why this was such a problem. Surely the program would have thrown the equivalent of a NullPointerException or dumped core? Not quite. See the article Nil by "ridiculous fish" (actually by far the best Mac OS programming blog I've come across). For NSSpell, silently returning 0 in r3 meant all words were considered misspelled, with no suggested corrections.

Although the "nil" article says "__objc_nilReceiver is there to let you replace the default message-to-nil behavior with your own! How cool is that!?", without the AppKit source I'm unconvinced that I could get away with a nil receiver that aborts any program trying to message nil. How much of Apple's code relies on this weird behavior? The prospect of introducing a host of crashing pseudo-bugs isn't particularly appealing.

Don't you just hate it when your tools conspire against you? Just as in Ruby, dynamism costs me something in a situation where I've absolutely no use for it. I've little against people adding this kind of dangerous nonsense, but it does annoy me when it's on by default and I can't turn it off.

-*-


On a more positive note, I thought I'd finish by mentioning the race-condition equivalent of "printf-debugging". It's a commonly-used technique, but I don't remember seeing it in any programming text I've read. Perhaps because we're still pretending concurrency isn't a basic part of computer programming. It's a shame, because it's simple, useful, and often a lot more productive than just staring at the code.

There are two possible goals. The simple one is to prove there is a race. The more complicated one is to find out where the race is, assuming it's not obvious. The idea is to insert a sleep (a big one, of seconds, that even a human would notice) in one of the participants. If you're in roughly the right area, you should make the race failure 100% repeatable. You can then move the sleep back until the race stops being 100% repeatable; now you know the crucial piece of code.

2006-05-30

Building Universal Binaries from "configure"-based Open Source Projects

Apple has a technical note TN2137 entitled "Building Universal Binaries from "configure"-based Open Source Projects". It has instructions for building a ppc/i386 "universal" binary given an OSS project that uses autoconf, and it seems like it would be really handy. The trouble is, the instructions have never worked for me.

I always have failures like this:

checking for C compiler default output file name... configure: error: C compiler cannot create executables

Which looks like this in config.log:

configure:1798: checking for C compiler default output file name
configure:1801: gcc -O -g -isysroot /Developer/SDKs/MacOSX10.4u.sdk
-arch i386 -arch ppc -Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk conftest.c >&5
/usr/bin/ld: -syslibroot: multiply specified
collect2: ld returned 1 exit status
/usr/bin/ld: -syslibroot: multiply specified
collect2: ld returned 1 exit status
lipo: can't open input file: /var/tmp//ccBImgUy.out (No such file or directory)
configure:1804: $? = 1

One of the many problems with autoconf is that it doesn't make it easy for a vendor to offer fixed testlets (I don't know what they're really called) to fix existing configure scripts. The versions on the maintainers' machines are hard-coded into the configure script, so even when a test is improved, each project's maintainer has to regenerate their configure script before you feel the benefit.

So, while we wait for that to happen, here's the work-around I've been using:

  1. Run ./configure as normal.

  2. Add -isysroot /Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc to the CFLAGS line in the generated Makefile.

  3. Add -Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk -arch i386 -arch ppc to the LDFLAGS line.

  4. Run make(1).

  5. Use file(1) to check that you got what you wanted.


Obviously, since we lied to the configure script, we might get bitten. But this has worked for me so far, and Apple's cleaner method never has. (Presumably there are some OSS projects out there where it does work, but I haven't needed to build any of them.)

2006-05-22

Making JTable look more at home on Mac OS and GNOME

Back in 2004 I wrote Alternating row colors in JTable which dealt with, as the title suggests, getting alternating row colors in a JTable, in a subclass called ETable. Last week, Daniel Lopez mailed me Creating an iTunes-like JTable where he fixes a clip-rectangle related bug in ETable, adds a tool tips, and gets rid of the border Swing puts on the selected cell.

The latest version of ETable, available from the salma-hayek link on this page, already had a better tool tip implementation. ETable automatically measures the text in a table cell and sets the tool tip to the full text if the cell is too narrow to show it all. So the "..." acts as a visual cue that there's something extra to see, and there's no otherwise-invisible information that only appears in a tool tip (where you can't control how long it stays visible or copy it to the clipboard).

Daniel suggested that it would be even better if it behaved like MS Outlook, which places the tool tip over the cell (rather than below and to the right, which is more typical tool tip behavior) so it looks like the tool tip is simply extending the truncated cell's text. I've always liked the effect (and its guarantee that the tool tip doesn't bob back and forth horizontally as you move your mouse up and down a column), so I now override JComponent.getToolTipLocation.

There are two slightly interesting aspects of the implementation. The first is the use of JComponent.getToolTipText(MouseEvent) rather than the more usual JComponent.getToolTipText(). The latter would just give us the table's global default tool tip, rather than delegating to the appropriate cell's renderer. The other is that even if you've correctly set a null tool tip, unless getToolTipLocation also returns null, Swing will display an empty tool tip if the tool tip is already up and you move over a cell that doesn't need a tool tip. (Hovering over the cell when there isn't already a tool tip up won't however cause a tool tip to appear.) I think there's a bug in ToolTipManager.checkForTipChange in that it should say newText != null && newPreferredLocation != null instead of newText != null || newPreferredLocation != null, but I can't be bothered to report it. (If you can, send me the bug number and I'll add a link.)

Daniel's method of getting rid of the border on the selected cell was interesting, though. Setting a custom border on a cell renderer is a technique I'd used before in SCM to render a list where items are in groups with dotted horizontal lines between the groups, but I hadn't thought of using it to solve the problem with JTable that it (in contrast to most native toolkits) thinks that the selected table cell should be marked with a focus rectangle.

Daniel wanted an effect where the whole row (rather than just the clicked-on cell) gets the border. He used a different border on the first cell in a row, the last cell in the row, and a third border for all the cells in the middle. This let him produce a Windows-like effect where the whole selected row has a focus rectangle.

If you look at iTunes, you'll see that on Mac OS there are no focus rectangles at all. There is a line at the bottom of each selected row, though. And there are those vertical lines that Apple's LAF doesn't draw. It's easy to adapt Daniel's technique to produce this effect on Mac OS, and to simply remove the focus rectangle on GNOME.

There are other Mac OS problems that have been annoying me. The upper right corner of an enclosing JScrollPane doesn't look like a table header, for one thing. And a native table's selection changes color when it loses the focus.

There were also various GNOME problems. The most problematic for me being that the default cell renderer for boolean columns (a JCheckBox) was messing up the alternate row coloring.

Anyway, I spent this afternoon polishing ETable, and now it looks a lot closer on both Mac OS and GNOME. Here's how it looks on Mac OS now:



It's obvious that the JCheckBox is too large, but I haven't had any success working around that yet, and I'm hoping Apple fix it before I get round to wasting more time on it.

And here's the ETable source at the time of writing. As always, the latest version (and the accompanying helper classes) is in the salma-hayek source download.

package e.gui;

import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.border.*;
import javax.swing.table.*;
import e.util.*;

/**
* A better-looking table than JTable. In particular, on Mac OS this looks
* more like a Cocoa table than the default Aqua LAF manages.
*
* @author Elliott Hughes
*/
public class ETable extends JTable {
private static final Color MAC_FOCUSED_SELECTED_CELL_HORIZONTAL_LINE_COLOR = new Color(0x7daaea);
private static final Color MAC_UNFOCUSED_SELECTED_CELL_HORIZONTAL_LINE_COLOR = new Color(0xe0e0e0);

private static final Color MAC_UNFOCUSED_SELECTED_CELL_BACKGROUND_COLOR = new Color(0xc0c0c0);

private static final Color MAC_FOCUSED_UNSELECTED_VERTICAL_LINE_COLOR = new Color(0xd9d9d9);
private static final Color MAC_FOCUSED_SELECTED_VERTICAL_LINE_COLOR = new Color(0x346dbe);
private static final Color MAC_UNFOCUSED_UNSELECTED_VERTICAL_LINE_COLOR = new Color(0xd9d9d9);
private static final Color MAC_UNFOCUSED_SELECTED_VERTICAL_LINE_COLOR = new Color(0xacacac);

public ETable() {
// Although it's the JTable default, most systems' tables don't draw a grid by default.
// Worse, it's not easy (or possible?) for us to take over grid painting ourselves for those LAFs (Metal, for example) that do paint grids.
// The Aqua and GTK LAFs ignore the grid settings anyway, so this causes no change there.
setShowGrid(false);

// Tighten the cells up, and enable the manual painting of the vertical grid lines.
setIntercellSpacing(new Dimension());

// Table column re-ordering is too badly implemented to enable.
getTableHeader().setReorderingAllowed(false);

if (GuiUtilities.isMacOs()) {
// Work-around for Apple 4352937.
JLabel.class.cast(getTableHeader().getDefaultRenderer()).setHorizontalAlignment(SwingConstants.LEADING);

// Use an iTunes-style vertical-only "grid".
setShowHorizontalLines(false);
setShowVerticalLines(true);
}
}

/**
* Paints empty rows too, after letting the UI delegate do
* its painting.
*/
public void paint(Graphics g) {
super.paint(g);
paintEmptyRows(g);
}

/**
* Paints the backgrounds of the implied empty rows when the
* table model is insufficient to fill all the visible area
* available to us. We don't involve cell renderers, because
* we have no data.
*/
protected void paintEmptyRows(Graphics g) {
final int rowCount = getRowCount();
final Rectangle clip = g.getClipBounds();
final int height = clip.y + clip.height;
if (rowCount * rowHeight < height) {
for (int i = rowCount; i <= height/rowHeight; ++i) {
g.setColor(colorForRow(i));
g.fillRect(clip.x, i * rowHeight, clip.width, rowHeight);
}

// Mac OS' Aqua LAF never draws vertical grid lines, so we have to draw them ourselves.
if (GuiUtilities.isMacOs() && getShowVerticalLines()) {
g.setColor(MAC_UNFOCUSED_UNSELECTED_VERTICAL_LINE_COLOR);
TableColumnModel columnModel = getColumnModel();
int x = 0;
for (int i = 0; i < columnModel.getColumnCount(); ++i) {
TableColumn column = columnModel.getColumn(i);
x += column.getWidth();
g.drawLine(x - 1, rowCount * rowHeight, x - 1, height);
}
}
}
}

/**
* Changes the behavior of a table in a JScrollPane to be more like
* the behavior of JList, which expands to fill the available space.
* JTable normally restricts its size to just what's needed by its
* model.
*/
public boolean getScrollableTracksViewportHeight() {
if (getParent() instanceof JViewport) {
JViewport parent = (JViewport) getParent();
return (parent.getHeight() > getPreferredSize().height);
}
return false;
}

/**
* Returns the appropriate background color for the given row.
*/
protected Color colorForRow(int row) {
return (row % 2 == 0) ? alternateRowColor() : getBackground();
}

private Color alternateRowColor() {
return GuiUtilities.isGtk() ? Color.WHITE : GuiUtilities.ALTERNATE_ROW_COLOR;
}

/**
* Shades alternate rows in different colors.
*/
public Component prepareRenderer(TableCellRenderer renderer, int row, int column) {
Component c = super.prepareRenderer(renderer, row, column);
boolean focused = hasFocus();
boolean selected = isCellSelected(row, column);
if (selected) {
if (GuiUtilities.isMacOs() && focused == false) {
// Native Mac OS renders the selection differently if the table doesn't have the focus.
// The Mac OS LAF doesn't imitate this for us.
c. setBackground(MAC_UNFOCUSED_SELECTED_CELL_BACKGROUND_COLOR);
c.setForeground(UIManager.getColor("Table.foreground"));
} else {
c.setBackground(UIManager.getColor("Table.selectionBackground"));
c.setForeground(UIManager.getColor("Table.selectionForeground"));
}
} else {
// Outside of selected rows, we want to alternate the background color.
c.setBackground(colorForRow(row));
c.setForeground(UIManager.getColor("Table.foreground"));
}

if (c instanceof JComponent) {
JComponent jc = (JComponent) c;

// The Java 6 GTK LAF JCheckBox doesn't paint its background by default.
// Sun 5043225 says this is the intended behavior, though presumably not when it's being used as a table cell renderer.
if (GuiUtilities.isGtk() && c instanceof JCheckBox) {
jc.setOpaque(true);
}

if (getCellSelectionEnabled() == false && isEditing() == false) {
if (GuiUtilities.isMacOs()) {
// Native Mac OS doesn't draw a border on the selected cell.
// It does however draw a horizontal line under the whole row, and a vertical line separating each column.
fixMacOsCellRendererBorder(jc, selected, focused);
} else {
// FIXME: doesn't Windows have row-wide selection focus?
// Hide the cell focus.
jc.setBorder(null);
}
}

initToolTip(jc, row, column);
}

return c;
}

private void fixMacOsCellRendererBorder(JComponent renderer, boolean selected, boolean focused) {
Border border;
if (selected) {
border = BorderFactory.createMatteBorder(0, 0, 1, 0, focused ? MAC_FOCUSED_SELECTED_CELL_HORIZONTAL_LINE_COLOR : MAC_UNFOCUSED_SELECTED_CELL_HORIZONTAL_LINE_COLOR);
} else {
border = BorderFactory.createEmptyBorder(0, 0, 1, 0);
}

// Mac OS' Aqua LAF never draws vertical grid lines, so we have to draw them ourselves.
if (getShowVerticalLines()) {
Color verticalLineColor;
if (focused) {
verticalLineColor = selected ? MAC_FOCUSED_SELECTED_VERTICAL_LINE_COLOR : MAC_FOCUSED_UNSELECTED_VERTICAL_LINE_COLOR;
} else {
verticalLineColor = selected ? MAC_UNFOCUSED_SELECTED_VERTICAL_LINE_COLOR : MAC_UNFOCUSED_UNSELECTED_VERTICAL_LINE_COLOR;
}
Border verticalBorder = BorderFactory.createMatteBorder(0, 0, 0, 1, verticalLineColor);
border = BorderFactory.createCompoundBorder(border, verticalBorder);
}

renderer.setBorder(border);
}

/**
* Sets the component's tool tip if the component is being rendered smaller than its preferred size.
* This means that all users automatically get tool tips on truncated text fields that show them the full value.
*/
private void initToolTip(JComponent c, int row, int column) {
String toolTipText = null;
if (c.getPreferredSize().width > getCellRect(row, column, false).width) {
toolTipText = getValueAt(row, column).toString();
}
c.setToolTipText(toolTipText);
}

/**
* Places tool tips over the cell they correspond to. MS Outlook does this, and it works rather well.
* Swing will automatically override our suggested location if it would cause the tool tip to go off the display.
*/
@Override
public Point getToolTipLocation(MouseEvent e) {
// After a tool tip has been displayed for a cell that has a tool tip, cells without tool tips will show an empty tool tip until the tool tip mode times out (or the table has a global default tool tip).
// (ToolTipManager.checkForTipChange considers a non-null result from getToolTipText *or* a non-null result from getToolTipLocation as implying that the tool tip should be displayed. This seems like a bug, but that's the way it is.)
if (getToolTipText(e) == null) {
return null;
}
final int row = rowAtPoint(e.getPoint());
final int column = columnAtPoint(e.getPoint());
if (row == -1 || column == -1) {
return null;
}
return getCellRect(row, column, false).getLocation();
}

/**
* Improve the appearance of of a table in a JScrollPane on Mac OS, where there's otherwise an unsightly hole.
*/
@Override
protected void configureEnclosingScrollPane() {
super.configureEnclosingScrollPane();

if (GuiUtilities.isMacOs() == false) {
return;
}

Container p = getParent();
if (p instanceof JViewport) {
Container gp = p.getParent();
if (gp instanceof JScrollPane) {
JScrollPane scrollPane = (JScrollPane)gp;
// Make certain we are the viewPort's view and not, for
// example, the rowHeaderView of the scrollPane -
// an implementor of fixed columns might do this.
JViewport viewport = scrollPane.getViewport();
if (viewport == null || viewport.getView() != this) {
return;
}

// JTable copy & paste above this point; our code below.

// Put a dummy header in the upper-right corner.
final Component renderer = new JTableHeader().getDefaultRenderer().getTableCellRendererComponent(null, "", false, false, -1, 0);
JPanel panel = new JPanel(new BorderLayout());
panel.add(renderer, BorderLayout.CENTER);
scrollPane.setCorner(JScrollPane.UPPER_RIGHT_CORNER, panel);
}
}
}
}

Hopefully as time passes, more of this will move into JTable and its UI delegates, but for now, we're stuck with work-arounds like this.

2006-05-20

MacBook/Pro

I happened to pass the local Apple Store last weekend, and noticed that the table with the MacBook Pro models was pretty quiet. They've presumably been out long enough now that they're not so exciting any more, but it was the first time I'd had chance to play. The middle model wasn't very impressive. Terminal.app took a spinning-beachball age to start. But then you never know what people have been up to before you get there. I found that it didn't have make(1) installed, but I didn't think of using javac(1) directly. I'd realized that I didn't like the "aluminum" look of the machines, didn't like the keyboard, and I knew I wasn't prepared to spend that much on a laptop anyway.

This week I popped in for a look at the new MacBooks instead. The 2 GHz white model was crashed; no kernel panic, but stuck solid. That left the 2 GHz black model and the 1.83 GHz white model. I played with the black one first. I had a bit of trouble with the keyboard. If you look at pictures you'll see that there's a little moat around each key. I think this looks great, but my fingers kept falling in those gaps. By the time I'd finished with the black model and switched to the bottom-end model, my fingers seemed to have adjusted. (I always have a bit of trouble switching to a new keyboard.)

Anyway, my first test on each was to download and run Terminator. I was impressed. Java on the Intel Macs seemed significantly better than Java on the PowerPC Macs. Terminator felt much closer to how it feels on a Linux box. And this despite the integrated graphics. (Maybe 1280x800 is a low enough resolution that integrated graphics is okay?)

The start-up time of a Java application on an Intel Mac is still poorer than I'd like, but it's better, and when it's actually running, performance is much more like what we've come to expect on other platforms. When I first used Apple's PowerPC JVM I assumed it was early days and that it would soon improve, but it never did.

I downloaded the latest salma-hayek source and tried time find ~/Desktop/salma-hayek/src -name "*.java" | xargs javac -d /tmp and was pleasantly surprised even before that: just the untarring was quicker than I'm used to expecting from an Apple laptop. The build times were strange. The black model consistently took 2.9s (real) and the white model 2.5s (real). Both machines had 1 GiB of RAM. It occurred to me on the way home that the black model probably had its processor deliberately crippled for extended battery life (fat lot of use that battery life is when you're literally chained to table and connected to a wall socket).

The more interesting take-home fact for me was that this is better than the 3.6s (real) that my dual G5 manages. And it's a fair comparison because although my dual G5 has two processors, the MacBook has a dual-core processor, and my benchmark of choice is single-threaded anyway. It would have been interesting to try make -j2 on some of our C++, but Xcode comes as a ".pkg", and you need an administrator password to install that. The Apple Store staff are friendly and helpful but since I wasn't planning on handing over my credit card I couldn't be bothered to hassle them. I think it's fairly clear that a MacBook would be a pretty good machine for Java development.

I didn't much like the shiny screens. It was quite difficult to see in the clinical brightness of the Apple Store, and it seemed that everyone who looked at one had to move the display to be able to see anything but glare. I'd be interested to see one outside in real life.

I did like the new right-click mechanism. Clicking the single trackpad button while you have two fingers on the trackpad itself causes a right-click. It's not as good as a two-button trackpad would be, but it's better than any previous Apple laptop.

The black model looked fine to me, but in person I found I preferred the way the white model looked.

If I had any need for a laptop, I'd probably have bought a MacBook.

2006-05-02

Why you should use jessies.org SCM tools

If you've read about SCM, but weren't convinced by our description of how great the tools are, maybe you should take a look at some alternatives. They'll have you running back here begging for mercy, unless you're one of those masochists. You're not a masochist, are you?

Here's the pain you'd feel if you try doing commits via your editor's find/replace functionality: SubEthaEdit and Subversion part II.

And here's how bad it gets with bash(1) aliases: Useful Subversion shell aliases.

Both of those Heath Robinson/Rube Goldberg solutions reminded me of jwz's comment on using regular expressions to solve a problem: "now you have two problems".

What's most remarkable is that both of these are from Mac OS users.

Anyway, if you've got this far and aren't already using SCM, check it out. It also works if you use BitKeeper or CVS, and Bazaar support went in a couple of days ago. And it's GPL. All you need is Java 5 or later.