2005-08-22

java.io.FileDescriptor on Win32

[Update: in the end, we gave up on the use of FileDescriptor in Terminator. It never caused us trouble on Win32 or Linux, but it did cause us a lot of trouble on Mac OS. In the end, we switched the Unixes over to the same implementation we used on Win32: where we have our own read(2)/write(2) loops in the JNI code and our own InputStream subclass, rather than trying to use FileInputStream and FileOutputStream.]

It's quite clear when reading the JavaDoc for java.io.FileDescriptor that Sun don't want you to use it. Not only is there no way to create an instance that isn't invalid, there's the explicit statement "Applications should not create their own file descriptors".

Not being allowed to use FileDescriptor is a real bummer if you've written some JNI code and have a file descriptor that you'd like to use on Java side. If you ask javap(1) about FileDescriptor it tells you that there's an int fd field. (If you don't like javap(1), you can see the field in the source Sun provides in src.zip.)

So using this knowledge, you write your application and it uses JNI and sets the fd field directly, and you get to reuse all of the higher-level code from the various stream classes. Everything's great. Until someone asks you for a Win32 port...

To cut a long story short, java.io.FileDescriptor on Win32 has two fields. There's our old friend int fd, but there's also a long handle. The "handle" field is a Win32 HANDLE.

There are two Win32 functions to convert between C runtime fds and OS file handles. _get_osfhandle() is analogous to fdopen(3), and _open_osfhandle() is analogous to fileno(3). (Of course, Win32 has fdopen(3) and fileno(3) too, because it supports all three styles: file descriptors, stdio FILE*s, and HANDLEs.) There's a private native method in the Win32 FileDescriptor that presumably wraps ::_get_osfhandle() (it takes an int and returns a long).

The bytecode for FileDescriptor.valid shows that it checks handle first, but will check fd too. You might wonder if it's possible to leave handle invalid and use fd if you just give it the right kind of file descriptor. Setting fd to 1 (STDOUT_FILENO) or -11 (STD_OUTPUT_HANDLE) doesn't work. A watchpoint on the fd field shows that it is being accessed (but perhaps just because of the already-mentioned implementation of valid).

jdb(1) says that even the predefined FileDescriptor instances have invalid fd fields on Win32:

main[1] dump java.io.FileDescriptor.in
java.io.FileDescriptor.in = {
fd: -1
handle: 1808
in: instance of java.io.FileDescriptor(id=318)
out: instance of java.io.FileDescriptor(id=317)
err: instance of java.io.FileDescriptor(id=319)
}

Anyway, for our application it turns out that it's cygwin's read(3) and write(3) that implement the pseudo-terminal behavior. This means that we can't use the standard Java InputStream and OutputStream because we need to make sure we go through the cygwin functions, rather than the ones Sun's native code uses. So the warning about not creating your own FileDescriptor instances came back to haunt us in the end, in a way.

If that's whetted your appetite for more about cygwin JNI, stay tuned...