2005-08-28

Bug 4995929 "Indeterminate JProgressBar prevents normal application exit"

I was hoping this morning to be able to remove a work-around for Sun's Java bug 4995929, which was fixed in 1.5.0 — all my programs require Java 5 anyway, so why leave in a hack for 1.4.2 users?

The answer, it turns out, is because Apple's aqua LAF is still broken.

Rather than subclass BasicProgressBarUI, which is where Sun added their fix, they copy & pasted BasicProgressBarUI into AquaProgressBarUI (judging by common field names). They may well have been forced into this by the number of private fields, methods and even inner classes in BasicProgressBarUI. And presumably Apple were lagging so far behind with their implementation that by the time they came to write AquaProgressBarUI it was too late.

I've raised Apple bug 4235221 "Sun bug 4995929 (fixed in 1.5.0) not fixed for aqua LAF". I can't give you a link, because Apple are particularly bone-headed in that regard. Judging by the progress that's been made on the other Apple-specific Java bugs I've submitted to Apple, namely "no progress whatsoever", this one won't be fixed before at least 2008.

I wonder whose job it is at Apple to break out diff(1) each time there's a new Sun release and look for changes in Sun's LAF code and work out what fixes need porting to the aqua LAF?

I still haven't raised an Apple bug for the way that java.awt.Robot's createScreenCapture can crash the VM, because I think the correct fix would be for checkValidRect to do a more thorough job. The Linux JVM may not crash in similar circumstances, but its behavior is still questionable. And a potential security hole. I'm still hoping against hope that "internal review ID 488856" will turn into an actual bug number, but it's been over a month now, and that's usually a sign that it's just been ignored.

That's the one thing I dislike about Sun's bug reporting process: there's no way to get an arbitrator in if you and the reviewer disagree. I still can't believe no-one will accept as a bug the fact that JSplitPane uses the wrong mouse cursor, and is forced to do so by the fact that Cursor doesn't offer the right one for either a left-right or top-bottom split pane.

Just my luck to keep getting blind reviewers.

Speaking of bugs, I hadn't seen the problem referred to in "Oh Chess.app, you damnable little bastard" until the day after I read that post, when I found that it's pretty repeatable on my 2001 PowerBook if I just do a slideshow from the Finder of "enough" images. I've yet to see it on my dual G5, but that has significantly meatier graphics hardware.

I see the text-editing problems all the time, but mainly on the dual G5 because that's where I run Mail. (10.4's Mail isn't trustworthy enough for me to run it on two machines concurrently like I used to with 10.3's version.)

Luckily for me, all my important text editing is done in Java applications.

Mighty Mouse, one week later

After a week of light use, I'm still having trouble getting the computer to know whether I mean left click or right click. I still wish they hadn't tried to be clever by having one switch for two buttons, and I still hope they fix this defect in later releases.

I like the usual Apple flat mouse shape. The last few years of orogeny in PC mice has just lead to uncomfortable mice, as far as I can tell.

I like the solid look more than the old "Perspex paperweight" look.

I love the nipple, perhaps because I was bottle-fed.

I keep seeing what I assume is a bug in the new mouse driver. Often – though I've not worked out the exact circumstances – I'll click to give focus to a window, but it won't get focus. It'll come to the front of the window stack, but it won't have the focus. So if I sweep-select some text, it'll sweep out in gray rather than blue. Clicking on stuff doesn't seem to work, either, until you've clicked in another window and then clicked back in the window you'd tried to give focus to.

Google tells me Microsoft still make white-ish scroll-wheel mice without arched backs. Maybe I should try one of them next, even if it means giving up on the nipple.

2005-08-24

Porting JNI code to Win32 with cygwin

Once upon a time, in the dark days when Win32 was the best place to be for a good Java experience (before Sun committed to a Linux JVM, and before Apple had switched to Unix), I was a user. As soon as Sun and Apple gave me choices, though, things went back to normal, and I went back to various flavors of Unix.

When people have asked for a Win32 port of Terminator, I've quoted Dilbert and said "here's a nickel, kid" (misquoted, as it happens: I've always said "proper computer").

My co-conspirator Martin Dorey's a bit more caring, though, and has ported Terminator to Win32. More than once.

Unlike our other projects, Terminator uses a bit of JNI. You just can't do the whole child-spawning dance in Java, so we do it in C++ and write to a private field in FileDescriptor objects allocated by the Java side. We used to have a separate executable, based on W Richard Stevens' "pty" example. Martin ported that the same week that Phil Norman rewrote it all as JNI code.

(We realized we might be making porting to new platforms more difficult because JNI tends to be far more intricately connected to the Java side than a separate process can be, and you lose a lot of room to maneuver when you agree to the tighter coupling. Resulting in problems such as having to make cygwin work in the JVM. We knew also that we would be increasing the chances that a mistake in our native code would kill the JVM. On the other side of the equation, we gained clarity, efficiency, and finer control over what was going on. One of the focuses of our C++ JNI wrappers is to ensure proper error checking. We didn't know about -Xcheck:jni at the time, and we're still not exactly sure what checks it enables. Anyway, our C++ classes constrain us to use the favored idioms.)

The first Terminator port was effected by just building with the cygwin tools. We didn't even have to stub modern POSIX functions, as we did on Mac OS 10.3, which didn't offer them.

The second port wasn't expected to be as difficult as it was. The hope was that the JNI code would just need to be built under cygwin. If you ask Google about cygwin-using JNI code on Win32, though, you'll know it's not.

Pages that mention cygwin almost all use -mno-cygwin or the mingw compiler. The only page Martin could find about building cygwin JNI DLLs (linked with cygwin1.dll) was out of date, and not convincingly correct. And anyway, he only found that while supplying me with material for this post.


  • Google's first match http://www.inonit.com/cygwin/ uses -mno-cygwin as if it's the only option (specifically here).
  • The author of http://sources.redhat.com/ml/cygwin/2005-05/msg00532.html stumbled at the first problem Martin ran in to (see below) but got no reply that was of any use in the general case.
  • BEA have an example http://dev.bea.com/codelibrary/code/jni.jsp that actually uses cygwin natively. It claims to have been updated in 2004, but it uses __cygwin_noncygwin_dll_entry@12 which seems to have rusted away. It talks about cygwin b20 which timestamps at sunsite.bilkent.edu.tr suggest dates from 1998. It mentions EGCS-1.1.2 which is long since obsolete. One makefile mentions jdk1.1.7A (there's a mention of 1.4.0 in the other). Martin got it to build against 1.5.0 with a few hacks, but neither of the example programs worked for him. One caused cygwin to think that something is using an incompatible copy of cygwin1.dll (though nothing is) and the other is linked with javai.dll — the JDK1.1 name for jvm.dll. They do seem to have at least an inkling of the right idea, though — that you need to use the Java Invocation API and compile your invoker with cygwin. But we're getting ahead of ourselves.


Problem one
If there's no other cygwin-using process running, then dynamically loading a DLL which depends (at load-time) on cygwin causes the process to deadlock waiting for a "calibration thread" to complete while the Win32 kernel is preventing other threads from running in this process. This is the problem mailing-list guy had.

Problem two
Dynamically loading cygwin's DLL, or a DLL which uses it, from a non-cygwin executable overwrites the bottom 4 KiB of the stack. This is documented and possible to work around by spawning a thread to load it which takes measures to ensure that the overwritten stack doesn't matter. This can be done from an independent, small, simple, non-cygwin DLL that's loaded before any cygwin-using DLLs that you actually want to use. But this isn't a suitable solution because...

Problem three
...with the work-around from problem two, cygwin's fork(3) still doesn't work, and nor does other more basic stuff either, including cygwin's stdio. Which makes debugging difficult, to say the least.

(The reason why cygwin's fork(3) doesn't work is that it runs the same executable again and subverts its startup code, which it can easily do if the application is a cygwin one — the startup code calls into the cygwin DLL. fork(3)'s already so difficult to implement on Win32 that mere mortals would call it impossible. Making it work for a non-cygwin executable like the JVM would have eaten into their day of rest.)

Solution
The way to solve the cygwin JNI problem robustly is to compile a launcher executable with cygwin. The Java Invocation API works from cygwin and allows a JVM to be launched. This can then System.loadLibrary the cygwin-using JNI DLL, and that can then fork without problem.

If you look in your JDK directory, you'll find the source to the java(1) launcher. You'll only find the source to the version for the operating system you're on, though: Sun's habit of _md files for machine-dependent code means you can't compile it on any supported OS. Moreover, they don't include all the necessary source files (though Google's cache has the missing ones). Another potential problem on the horizon is that the JDK6.0 drops don't seem to include the files. I don't know whether this is an oversight or indicative of an intention to remove them from the distribution.

Worse still, I don't really know what we're allowed to do with the example launcher source. It's nice that most of the intelligence is in the shared object/DLL, but there's still a fair bit of code in the launcher, in particular the code that locates the shared object/DLL.

But if you've got some JNI to port to Win32 and it looks like the only sensible way to do it is to use cygwin, then we think this is your only choice. Feel free to use the source to a Win32 cygwin Java launcher that's now in salma-hayek.

Terminator now works on Win32, and you can ssh(1) to a Unix machine, run vim(1), and everything works right when you resize the window. Quite a testament to cygwin, I think. And to Martin's stubbornness.

2005-08-22

java.io.FileDescriptor on Win32

[Update: in the end, we gave up on the use of FileDescriptor in Terminator. It never caused us trouble on Win32 or Linux, but it did cause us a lot of trouble on Mac OS. In the end, we switched the Unixes over to the same implementation we used on Win32: where we have our own read(2)/write(2) loops in the JNI code and our own InputStream subclass, rather than trying to use FileInputStream and FileOutputStream.]

It's quite clear when reading the JavaDoc for java.io.FileDescriptor that Sun don't want you to use it. Not only is there no way to create an instance that isn't invalid, there's the explicit statement "Applications should not create their own file descriptors".

Not being allowed to use FileDescriptor is a real bummer if you've written some JNI code and have a file descriptor that you'd like to use on Java side. If you ask javap(1) about FileDescriptor it tells you that there's an int fd field. (If you don't like javap(1), you can see the field in the source Sun provides in src.zip.)

So using this knowledge, you write your application and it uses JNI and sets the fd field directly, and you get to reuse all of the higher-level code from the various stream classes. Everything's great. Until someone asks you for a Win32 port...

To cut a long story short, java.io.FileDescriptor on Win32 has two fields. There's our old friend int fd, but there's also a long handle. The "handle" field is a Win32 HANDLE.

There are two Win32 functions to convert between C runtime fds and OS file handles. _get_osfhandle() is analogous to fdopen(3), and _open_osfhandle() is analogous to fileno(3). (Of course, Win32 has fdopen(3) and fileno(3) too, because it supports all three styles: file descriptors, stdio FILE*s, and HANDLEs.) There's a private native method in the Win32 FileDescriptor that presumably wraps ::_get_osfhandle() (it takes an int and returns a long).

The bytecode for FileDescriptor.valid shows that it checks handle first, but will check fd too. You might wonder if it's possible to leave handle invalid and use fd if you just give it the right kind of file descriptor. Setting fd to 1 (STDOUT_FILENO) or -11 (STD_OUTPUT_HANDLE) doesn't work. A watchpoint on the fd field shows that it is being accessed (but perhaps just because of the already-mentioned implementation of valid).

jdb(1) says that even the predefined FileDescriptor instances have invalid fd fields on Win32:

main[1] dump java.io.FileDescriptor.in
java.io.FileDescriptor.in = {
fd: -1
handle: 1808
in: instance of java.io.FileDescriptor(id=318)
out: instance of java.io.FileDescriptor(id=317)
err: instance of java.io.FileDescriptor(id=319)
}

Anyway, for our application it turns out that it's cygwin's read(3) and write(3) that implement the pseudo-terminal behavior. This means that we can't use the standard Java InputStream and OutputStream because we need to make sure we go through the cygwin functions, rather than the ones Sun's native code uses. So the warning about not creating your own FileDescriptor instances came back to haunt us in the end, in a way.

If that's whetted your appetite for more about cygwin JNI, stay tuned...

2005-08-21

A NETGEAR and a Mighty Mouse

A made a trip to Fry's this afternoon. Networking kit, for some reason, is over past resistors and capacitors, in with the low-noise fans and motherboards and other assemble-it-yourself stuff. Weird. I thought home networks were totally mainstream now. Their placement in Fry's would suggest otherwise.

I was quite set on getting a NETGEAR wireless router, after good experiences with the one I got my parents. I was disappointed to see that Fry's stock a whole aisle's worth, many with identical model numbers (but different prices), and more disappointed to find that there's no combined wireless router and gigabit switch. 10/100? How 1990s is that? (Seems 100 Mib/s is still referred to as "Fast Ethernet". Even at the time, that name seemed short-sighted. And superfluous given the perfect "100 Mib/s" name.)

Next time I'm banging on about how NETGEAR's products have the best web interfaces, remind me that most of my experience had been with a Win32 version of MSIE. They don't get on with Camino (my default) all that well, though Safari saved me a trip to Fry's returns desk. My networking worries are now over. My PowerMac and PowerBook are both connected, and the only evidence is a pile of server-room clutter on the opposite side of the room.

Anyway, I'd gravitated as always to the Mac aisle, and was hoping they'd have a "Mighty Mouse". (That name really is one in the eye for anyone who complained about the likes of "AirPort Extreme" — can it get any worse? Thank goodness Apple didn't get to dub GigE "Extreme Ethernet".)

I'd been waiting for a nice-looking mouse with a scroll wheel for my Mac, but I was embarrassed to hear that they'd gone for a two-button mouse in a one-button shell. I'd no idea how having to lift your left finger to right-click would work for me. As it happens, right-clicking is fine. It turns out that I use the mouse with no fingers resting on top, and only bring fingers down to click.

What isn't so fine, though, is left-clicking. What I didn't realize is that I've developed a habit of clicking with two fingers, roughly in the middle of the mouse (just behind the new mouse's nipple). This has a habit of registering as a right click. Hopefully I'll stop, but I'm not convinced: clicking with two fingers is so much more comfortable. I wish they'd made most of the surface a left button, and had a separate little-toe-style right button. If anyone wants to build a mouse that would suit my personal clicking style perfectly, that's what it would be like!

Strangely, I don't think I ever knew before that you could right-click Dock icons. It had always annoyed me that it was so slow to get iTunes' menu, holding down the mouse button over the Dock icon and waiting. The first time I used iTunes after plugging in the new mouse, I right-clicked, and was viscerally pleased to have the menu appear instantly. I wonder if I've always been subconsciously holding down the right side of the one-button mouse to get that menu? Weird.

Speaking of weird, the scroll nipple is pretty damn strange. It feels almost like it's vibrating as you manipulate it. You know those little toy cars where you wind a wheel, and they race off when you let go? It feels like that, only with less resistance.

I like the solid look. I wasn't joking when I said one of the dual G5's noticeable improvements over the dual G4 was a keyboard without the clear plastic; the new machine was otherwise fairly underwhelming. Anyway, now my PowerMac has an almost matching keyboard & mouse. They don't quite match because the mouse is shiny where the keyboard is matt, and the mouse is shaped like an enormous pessary where the keyboard is more like an enormous LEGO block.

Anyway, I'm fairly pleased with today's purchases so far. Time for steak and eggs...

[Note: I wasn't sure whether the correct term is "mouse nipple" or "mouse teat"; PubMed lightly favors the former.]

2005-08-17

comcast

In England, I used Pipex, and they were great. Simple, and transparent. They sent you a username and password for your ADSL router, and they mailed you a couple of weeks in advance whenever they needed two minutes' down time at 03:00 on a Sunday.

Now I'm in the US. Land of temperatures in Fahrenheit, non-metric measurement they have the cheek to call "English", 12 hour clocks, primitive banking arrangements (including laughable "security", various unwarranted fees, and continued use of paper checks for funds transfer), parochial date formats (though at least the Asian community is happy with ISO), and all sorts of other non-futurist non-internationalist nonsense.

So I should have expected a backward approach to internet connectivity. comcast have to send a human round to fiddle with cables. They don't tell me how I'd configure my own cable modem, presumably because they insist I use theirs. They make me run some shitty installer (for classic) from 2003, which does unspecified things which I later find includes writing a setuid/setgid executable ~/.netprefs. A file Google seems to know nothing about, but which contains scary strings such as /bin/rm -rf " and /bin/sh " that you just know are being used with system(3). (Why don't script kiddies stick to scripts?)

Did I mention that the installer runs Internet Explorer?

And deletes your VPN configuration? (/var/log/ppp.log should contain the IP address of the server if you can't remember it.)

bash(1) is pissing me off right now – through no fault of its own – by showing that my hostname is now c-99-999-999-999 (where the nines are my presumably static IP address).

While the cable guy was here, I connected my PowerMac to the cable modem. It seemed like a good idea to set up the machine with the biggest display, rather than start with the router, which doesn't even have a display. But when he'd left and I'd caught up on the day's email and was ready to connect the wireless router to the cable modem, I find that doesn't work. Nor does connecting the PowerBook. Google suggests that I'm stuck with the MAC address that was first connected to the cable modem. I haven't found any indication that I can phone comcast to change this, but presumably I can. (Though I have a fear that they'll insist on me having another day off work so they can send a bloody human round.) Using ifconfig en0 ether aa:bb:cc:dd:ee:ff I've been able to connect the PowerBook to the cable modem by taking the MAC address the PowerMac was using (which doesn't seem to be the MAC address it had before the crappy comcast installer. Are they trying to give everyone the same MAC address?)

The only victory I can really claim right now is that I don't have to suffer the indignity of any telephone or TV services. And that 4 Mb/s downstream (not 4 Mib/s; it's somewhere around 4029 b/s) is faster than I had before.

How does one emigrate to South Korea, exactly? "Silicon Valley", my arse. Museum of antiquities, more like. What am I doing here?