jvisualvm

For some reason, I was under the impression that jvisualvm(1) was just a rebranding of jconsole(1). It turns out that it isn't, and that – unlike jconsole(1) – it's not a waste of space.

It's not very clever about dividing up space between its various graphs, but other than that it looks pretty nice. I thought I wanted a quick heap profile to see why one of my applications ate so much memory that my machine always felt swappy afterwards. When jvisualvm(1) showed that there wasn't much heap retention going on, I was almost ready to dismiss it as broken, but thought I'd play about and see what else it could do. Turning to the "threads" tab, I saw my actual problem: I had several hundred idle threads.

It turned out I'd once again forgotten that "an unused ExecutorService should be shut down to allow reclamation of its resources".

I had code like this (only without the call to shutdown):

  ExecutorService service = Executors.newFixedThreadPool(threadCount);
  for (Input input : inputs) {
    service.execute(new Job(input));
  }
  service.shutdown();

Every time this ran, which was pretty frequently, I'd leak another 8 or so threads, that I'd never use again.

I should write myself some kind of "do this batch of jobs on n threads and then kill the threads" wrapper.

Anyway, although I was disappointed not to find any heap retention in any of my applications, I was quite pleased with jvisualvm(1) for finding a performance problem, even though it would have been as easy to spot with control-\ — I think that having all the tools together in one place makes me a bit more likely to bother to poke about now and again.

If you have trouble installing plugins like VisualGC, try quitting and restarting. I found that – contrary to the error message – they had actually installed okay, and appeared in the UI when I restarted. They even seem to work.

xargs -P

Another entry in my occasional series of options Unix never had when I were a lad (last time: tail -F)...

I needed to use xargs(1) the other day to run lots of really cheap jobs, each of which had a high latency.

This was taking a long time until a friendly youngster suggested I use GNU xargs' -P option. Starting again with -P 32, my job finished in less time than I'd already wasted.

Mac OS 10.6

I'm pleased to see Mac OS 10.6 is coming in September 2009, and I'm pleased it's going to be $29 instead of $129, because most Mac users I know stuck with 10.4 which was "good enough" and which they'd already paid for.

I'm slightly surprised by the "Mac computer with an Intel processor" requirement. At least I won't have to feel quite as bad about dropping PowerPC support myself.

I'm disappointed at the lack of detail about Java. "Complete Java JDK, including javac, javadoc, ANT, and Maven tools" doesn't tell me whether Java 6 will finally be the default, which is what I'm hoping for. PowerPC support doesn't cost me much (except the ability to build on 10.4 or 10.5 with the same make rules, which isn't a big problem as long as all the Mac OS-using developers are still on 10.4).

But being tied to Java 5 is increasingly annoying (and limiting). Reflection and other hacks only get you so far, and -source 1.5 doesn't protect you against accidentally using API that wasn't in Java 5 — String.isEmpty having caught me out just yesterday.

Netbeans + OpenJDK = frustration

Although I sometimes claim to try out the various IDEs every few years, what I really mean is that I try out Eclipse every few years. I've only tried Intellij once, recently, and to be honest I think I killed it after a couple of minutes waiting for it to start. I'm pretty sure I never got round to actually doing anything with it.

As for Netbeans, I'm not sure I've even seen it since 1999. Not on my desktop or anyone else's. When I complain of bloated, slow, unresponsive IDEs, I always think of Netbeans. But when I've complained of crashy, bloated, slow, and unresponsive IDEs, well, I'm more likely to have been thinking of Eclipse.

Today I thought I'd fix a few of Sun's simpler Swing bugs in JColorChooser. The kind that have been "fix understood" since 2003 but which shouldn't take more than a few minutes to fix. The aggravating "have the bastards ever tried to actually use this shit?" stuff.

Going to the OpenJDK website, I saw they were advertising Netbeans (currently version 6.5.1) as an easy way to get into OpenJDK development. Fair enough, I thought, I'll give that a go.

The instructions should have warned me off. Don't expect any special OpenJDK-building integration. Netbeans doesn't let you deal with Mercurial forests without manually installing the forest extension; the instructions go straight to downloading a .zip source bundle. Okay. They then go on to explain where to find Netbeans project files to open, but they seem to have moved since the instructions were written. find(1) found them, and I opened the small "swing" project...

My first problem was where Netbeans hung my X11 server. I was selecting text at the time, in the first file I double-clicked on, just to see what the editor was like. Thankfully I was using Linux, so I could switch to a different virtual terminal, log in and kill Netbeans, and then switch back to the virtual terminal with X11.

After restarting Netbeans, I tried to build the "swing" project, but it won't build out of the box (this was OpenJDK 1.7.0-b59). Something to do with Nimbus. (Of course!) So naively I thought I'd open the "world" project and build that. Only Netbeans couldn't really open that project on my puny machine. 4 cores and 4 GiB of RAM just isn't enough, seemingly. I sat waiting for it to finish (because the UI was really unresponsive during its "scanning" process) but it took long enough (using only one core) that the screensaver came on. And, presumably for reasons related to my first problem, I couldn't unlock the screensaver until I switched virtual terminal to kill Netbeans.

Don't get me wrong, I saw hints of interesting features in Netbeans. Its behavior when you return to a file you've edited externally but not in Netbeans, for example, feels better than Evergreen's behavior; Evergreen alerts you to the modification by drawing a watermark behind the content, and requires you to manually "Revert to Saved", showing you a diff when you do so. This is sometimes useful — if you've got a generated file open, for example, it's really nice to be able to keep one golden version while you experiment with the code that generates the file, seeing a diff after each run from the convenience of your editor. But automatically and silently reverting to saved seems a lot more convenient for the more common case where you've simply pull new changes from version control, or you used Vim over SSH on your bus ride home or whatever.

Netbeans was also pretty good-looking for a Swing application, although somewhat uneven; the directory chooser looked like some SwingSet Metal horror that probably hadn't changed since 1999.

And Netbeans' uninstall.sh script (in the directory you installed Netbeans in) seems to work perfectly, too, which was a relief. But overall this was a bad enough experience that I fear it will be 2019 before I give Netbeans another try.

The silly part is that building OpenJDK is pretty quick and easy. (At least compared to fighting Netbeans.) Download and install a source .zip, download and install the remaining binary plug (some SNMP crap), install the things you don't have that the README-builds.html lists, and "make". I also had to install libxtst-dev, which I didn't see mentioned, and I found that the Makefiles don't appear to be "make -j"-safe.

So there's not much difficult about building OpenJDK any more. How about slow?

Even on my too-puny-for-Netbeans 2007-vintage Core 2 Quad with 4 GiB of RAM, a clean build only took about 45 minutes. (With the build seemingly never using more than two cores.)

Despite it being something any monkey could do, if they were of a mind to, it's strangely gratifying to run your own JVM at the end:

$ ./build/linux-amd64/bin/java -version
openjdk version "1.7.0-internal"
OpenJDK Runtime Environment (build 1.7.0-internal-elliotth_2009_05_24_21_45-b00)
OpenJDK 64-Bit Server VM (build 16.0-b03, mixed mode)

If you're wondering why I used a source .zip rather than using Mercurial to check out the forests, it's not that I haven't tried Mercurial a few times. It's just "too hard".

I'll try Mercurial again when I'm running an OS that has a sufficiently up-to-date package. Hopefully one day Sun's choice of Mercurial will look like a wise long-term decision, but it's a bummer that it makes life hard for casuals like me right now. Hopefully, just as building OpenJDK was once "too hard", but is now – once you have the source – perfectly reasonable, we'll be able to say the same about Mercurial.

Confessions of a Windows hater

I've slightly changed my working environment recently, for the first time in years. Other than adding the System Monitor applet to the panel at the top of my GNOME desktop, I've basically been using the default setup since I switched to GNOME a few years ago.

The default setup has a panel at the top of the screen and another at the bottom. The top panel has a few menus, an icon that gets you a new web browser, a few other icons, a clock, and a weird log out/shut down/reboot button duplicating part of one of the menus. The bottom panel has a button to hide all windows, a big space for buttons representing windows, a little representation of your virtual desktops, and a trash can.

Type "ubuntu" into your favorite image search.

Of these things, the only things I used frequently were the icon that gets me a new web browser and the clock. I used the buttons representing windows too, to some extent, but I didn't really like them and often felt like they were costing me more time than they were saving me. Certainly the experience was nothing like as smooth as Mac OS' dock.

What I didn't know was that the top panel and the bottom panel aren't fundamentally different. They're both the same kind of container, different only in the stuff they happen to have in them. I learned this when I saw someone else's desktop and noticed that they'd moved the buttons representing windows into their top panel, and removed their bottom panel entirely. Even on a 30" display, pixels are valuable, and vertical pixels especially so.

I immediately switched to just one panel. Being a Unix/Mac OS kind of person, I kept the top panel. The bottom panel had always bothered me a bit, possibly because it's so Windows-like, but having tried my single panel in both places, I have to say I much prefer it at the bottom of the screen. I can actually feel myself having to look up when it's at the top. You could argue that's good, because it's not particularly useful, and you may as well have it out of the way, but being a miserable sod, I find that just makes it even more annoying when i do have to use it. I also find it easier to hit the web browser icon when it's at the bottom rather than the top; possibly just because it's easier for my eyes to keep up with my mousing.

What about the sides? Good question. On Mac OS, I've only ever used the dock at the side. Left or right. I don't much care. I love this, because it takes the least space, and that space is horizontal space, which I have more of anyway. It works well on Mac OS because everything in the dock is basically square. Any text only shows when you point, and it shows over the desktop, where there's plenty of space anyway. The other reason it works is that the Mac's dock only has one entry per application, rather than one per window.

If you find the exact right pixel to click (right-click the little grip to the left of the buttons; the nice big space on the right gets you a completely different menu) you can set the "Always group windows" property to get a lame-ass imitation of this. It's okay in terms of saving space, but the behavior shows the usual lack of respect for the user and what they might be trying to do. In particular, you can't easily bring all an application's windows to the front. That's not a big problem for something like a web browser or terminal emulator where an application's various windows aren't closely related other than by parentage, but it's a pain for applications that have a small number of genuinely related windows: you can't conveniently bring "the application" to the front. It's even quite awkward to hide (minimize) all of an application's windows. Certainly nothing approaching the "click to show, command-H to hide" I so miss from Mac OS. What else is wrong? The visual indication of minimized windows could be more distinct (as it is, the title is simply surrounded by square brackets). The title shown in the panel is the application title if you have more than one window, but the window title if you only have one window. I'm sure this was well-intentioned, and it does sound reasonable until you use it, but it just doesn't work if you keep opening and closing windows. The text you're looking for keeps changing, and its position changes too, though I'll admit that I honestly can't work out what the sorting criteria are. Random shuffle? Finally, most difficult to fix, you only get an entry per window. So if you use tabs, you'll only see the title of the currently-visible tab in each window. This really sucks, but even the non-free OSes don't get this right yet.

Even with those reservations, I'm still happier like this. I have enough windows open at any given time that the other style just isn't useful. Fifty little buttons all with nothing more than an icon and "..." may as well not be there at all for all the good it does. (Which strikes me as odd; surely users who don't open many windows don't really need the buttons anyway?)

Just as I thought I was done messing with configuration, I accidentally removed my "Applications", "Places", and "System" menus. Which was 99% fine by me, but I do actually need them once a week or so. So I choose "Add to Panel..." and add "Main Menu" back. Only it turns out that "Main Menu" is just an Ubuntu logo. Clicking on it gets you a single menu that's basically the "Applications" menu with the semi-useful bits of the other menus tacked on the bottom. So there's a few hundred more pixels you can save: remove the menus and add "Main Menu" back for no loss of functionality but plenty more space. (The new menu is nice and easy to hit, too, being right in the corner.)

The one change I made after this was to swap the relative positions of the system monitor and my web browser and terminal emulator icons. I originally had the icons right after the wordy menus, and the system monitor in the wasteland to the right. Even when I switched to one panel at the bottom, I saw no reason to change this. But making the menus really small moved those icons further from the center of the screen where, on average, my mouse pointer is. Swapping the two moves the buttons closer to hand. (Or, if you prefer, brings them within range of a single mouse sweep rather than requiring two, which is far more aggravating than you'd credit.)

Anyway, here are my suggestions again:

1. One panel is enough for the stuff that actually does anything.

2. The bottom of the screen seems to be a better place for this panel than the top.

3. "Always group windows" isn't as stupid as the default behavior.

4. Removing the menus and adding "Main Menu" back looks and feels better.

The only other things I have in my panel are the "lock screen" icon and the date/time/weather thingy. I used to have the "lock screen" icon with the other two, but I found myself sometimes confusing the terminal emulator icon and the lock screen icon, or at least pausing to make sure which one I wanted to press (they're both basically small dark squares representing monitors), which is stupid when one of them I want a hundred times a day and the other maybe twice. (No, I don't lock my machine when I go for a piss. I piss in the corner of the office.) Moving the lock screen icon out next to the clock thingy cured that.

I will admit that, all in all, I've made my desktop look a lot more like Windows than it did. But I'm man enough to admit it's better for it. And I'm grumpy enough to wish I didn't have to waste my declining years pissing about with stuff that shouldn't be getting in my way in the first place.

Maybe next year I'll learn how to use gconftool(1) to stop the window manager from opening every window in the top left corner right on top of the last one, as soon as the screen is too "full" to place the window in completely empty space. (You know, ten seconds after you log in.) Presumably GNOME developers have 15" displays and only open one maximized window at a time, like the Windows users of Windows-haters' myth?

Playing mp3s from Java

JNI

It's pretty obvious that mp3d needs to be able to actually play mp3s. This was another reason why I initially chose C++, my assumption being that there would be plenty of C++ mp3-playing libraries to choose from.

In a way, there were. But, perhaps because audio code isn't very portable and perhaps because Linux's audio API is abandoned for something new every six months or so, the libraries I came across were mainly focused on decoding mp3 data in some raw form. Presumably if I'd looked hard enough I'd have found the other pieces of the puzzle and been able to put them together, but after a brief search I gave up and started looking at command-line tools instead.

Spawning subprocesses

This is how the first version worked. If all you want to do is play an mp3 from start to finish and have no real control over – or feedback about – what's going on, this is a really easy option. But I was surprised by the strange programmatic interfaces offered. mplayer(1) seemed the best overall, and does indeed have a mode where it expects to communicate via a pipe, but the command set available seemed to assume a greater autonomy for mplayer than I was looking for. I wanted to keep the "playlist" in my server, for example, and basically use mplayer as my low-level API. It wasn't clear that I could really do that, and the various bits of calling code I could find on the web didn't suggest otherwise.

When I used mplayer from Java, I actually used Unix signals to control it, but I soon decided I wanted more control, and didn't particularly want to be tied to running on Linux (even though personally I'm unlikely to ever run mp3d anywhere else).

javax.sound

You might think that Java would have built-in support for the most popular audio format, but it doesn't. Presumably licensing problems. I hope this will be fixed by OpenJDK, at least for Linux, where it seems possible that whatever Free library they use for audio will have out of the box support for mp3 (and ogg and all kinds of other thus-far neglected formats).

Sun's bug database considers JMF (remember that?) to be the answer.

JavaZOOM JLayer

Despite the awful name (though I of all people should be careful about throwing stones where naming is concerned) JavaZOOM's JLayer is "the" Java library for decoding/playing mp3s. There may be others, but this is the one that everyone seems to use, so it was the one I went for too.

The API is a bit odd. Some of the names suggest no native speakers of English were involved. You stop a Player by calling "close", for example.

The API is also rather limited. If seems like more of a demo for the underlying mp3 decoder than an real API anyone's actually expected to use. If all you want to do is play an mp3 from start to finish, without pause or fast-forward (though you can stop prematurely), you're fine. Anything else and you'll be using the low-level API to shuffle frames from the decoder to the audio device.

Even the trivial case isn't well documented, so here's the minimal sample:

  String filename = "muzak.mp3";
  AudioDevice audioDevice = FactoryRegistry.systemRegistry().createAudioDevice();
  Player player = new Player(new FileInputStream(filename), audioDevice);
  player.play();

The call to "play" will block until the mp3's finished playing, but you can call "close" in another thread if you want.

(I'd have sent in patches, but the project doesn't have any visible source repository and appears pretty dormant.)

This is actually pretty much the code I'm using in mp3 at the moment. I keep expecting to want more, but I'm not yet convinced. I will admit to having used SIGSTOP and SIGCONT from a shell on a couple of occasions, but "pause" doesn't make as much sense as you might think on the web. Maybe if I bound my "media keys" on all my machines to POST appropriate form data I'd be more convinced. That would actually be pretty cool.

Without moving to something distinctly more AJAXy, though, I'm not sure "pause" fits the UI. And you probably would have to present it for the benefit of other users.

Using com.sun.net.httpserver

One of the things I needed for mp3d was an HTTP server. Sun added one to JDK6 – com.sun.net.httpserver – but the documentation's a bit sparse, there's no example code (and the snippet in the package description doesn't compile), and judging by the results of my web searches, it's not widely used.

Despite these obstacles, it was pretty easy to do everything I needed. Here are a few notes from my experiences.

Getting query parameters

If you're using HTML forms with the GET method, you'll need access to your query parameters. The obvious way to get them would be to call HttpExchange.getRequestURI and then call URI.getQuery on the result. Unfortunately, that's broken: URI's built-in decoder only copes with %xx escapes, and doesn't know that '+' needs to be translated to ' '. Bizarre.

What you need to do is use URLDecoder on the result of URI.getRawQuery instead.

(Either way you need to break the string into key-value pairs yourself.)

Getting form data

If, on the other hand, your HTML forms use the POST method, you'll need access to your form data. Here the obvious scheme works fine: just read the input stream. Again, I'm surprised there's no convenience method to help with this common task.

Setting headers in the HTTP response

If you want to set an HTTP header in your response, use the HttpExchange.getResponseHeaders method. That gives you a Headers object which is just a map. You can use Headers.put as if it were a plain old HashMap or whatever, but I preferred to be explicit and use Headers.add or Headers.set to make my exact intention clear.

(Getting the response headers object and calling a setter on that is pretty obvious when you know, but it took me longer to find than it should have done, in part because I wasn't sure the functionality even existed, but mainly because I was looking to start my journey with a setter, not a getter.)

Exception handling

Exceptions not caught by your HttpHandler implementations are reported, but in very spartan fashion, without stack traces. There's also no way to set an uncaught exception handler (as there is with AWT, say). You'll probably want to write your own code to deal with this. An obvious possibility is to write your own abstract class implementing HttpHandler with a "handle" method that catches and reports anything thrown by your class' new abstract method that does the actual handling of the request.

HTTP response codes

If you want constants for HTTP response codes so your code's not full of magic numbers such as 404 and 307, look at HttpURLConnection (from the standard java.net package). Then you can use, for example, the more obvious HttpURLConnection.HTTP_NOT_FOUND instead of 404.

GZIP content encoding

There's no built-in support for the GZIP content encoding, but it's pretty easy to do it yourself. Given an HttpExchange 't':

t.getResponseHeaders().set("Content-Encoding", "gzip");
t.sendResponseHeaders(HttpURLConnection.HTTP_OK, 0);
final GZIPOutputStream os = new GZIPOutputStream(t.getResponseBody());
os.write(...);
os.finish();
t.close();

You might want to check the request headers to ensure that the client can cope with this encoding, but I couldn't be bothered. As the Zen master said, a bug's not a bug until it inconveniences somebody.

How does HttpServer choose the appropriate HttpContext?

When you set up your HttpServer, you add "contexts", which are basically associations between a path and the HttpHandler that corresponds to that path. You can just have one that parses the path itself, if you like. The current implementation is an unsorted linked list, but that's fine for the kinds of things you'd sensibly use this code for.

Is it kosher?

You'll have noticed that the HTTP server packages are com.sun.* packages. Although they're mentioned in the Networking Features part of the JDK6 release notes, they're not indexed in the Java Platform API Specification.

The rules for com.sun.* packages aren't mentioned in the JDK6 documentation's Note About sun.* Packages.

As I understand it, com.sun.* differs from sun.* in that Sun supports the former but not the latter. In neither case are you guaranteed that it'll be available in other implementations. (For the record, Apple's Mac OS 10.5 JDK6 does include the com.sun.net.httpserver packages.)

Past experience suggests that a com.sun.* API in one JDK release won't necessarily be in the next, so you might not want to base your business on this API. But for a weekend hack that would be equally fine with an 80-line HTTP server using little more than a ServerSocket, it's a handy jump start.

Sample code?

Even I have better things to do than write you a nice little tutorial, but the mp3d source currently contains working examples of all the above.

I'm pretty sure I spent longer writing this post than I did working any of this stuff out or writing the code.

Why do I keep writing C++?

When you think of Unix daemons, you probably think of C or C++. When you think of cobbling together a program from a handful of libraries on Linux, you probably think of all the C and C++ libraries just a dependency away.

This, at least, was how I ended up writing the very first version of mp3d in C++. And it was a huge mistake. Throwing away two days of work and starting again in Java was the best move I made.

Aren't all daemons written in C/C++?

Tradition is a state, not a reason. Just because the daemons we grew up with (or wrote) years ago were in C doesn't imply that C was particularly suited to writing daemons, or that we should use it to write any more.

I think part of the problem here was that I could pretty much see the boilerplate, ready to flow out of my fingers. And that boilerplate was in C++. The opposite would have been the case if I'd been thinking of a GUI application. It wouldn't have even crossed my mind to consider C++ for one minute. The relevant boilerplate would have been ready, and this time it would have been in Java.

In the server case, such "thinking" is short-sighted. If the boilerplate is the hard part of what you're about to do, what you're about to do probably isn't worth doing at all.

But all those great libraries...

I think what really convinced me, though, was the idea that I'd be able to grab one of any number of HTTP libraries and id3v2 tag libraries if I chose C++.

Never mistake quantity for quality.

The HTTP library wasn't much of a problem, though the selection was much more limited than I'd imagined. The least unpromising choices were very much C libraries, with all the ugliness that that entails. Manual memory management everywhere, and no notion of stream or string. Three fairly fundamental things that even C++ programmers take for granted. I won't even waste breath on my chosen library's lack of const-correctness.

The id3v2 tag library situation was far worse. The situation there appears to be that there are several alternatives, but none of them is actively maintained. As far as I can tell those applications that manage to work with real-world id3v2 tags without crashing are using their own forked/patched versions of these libraries. And even though you can get a C++ (rather than C) library for handling id3v2 tags, it's quite the runniest kind of shite. It's the kind of C++ you see written by people who don't show any sign of understanding why C++ is better than C. ("Is it the // comments, perhaps?")

Don't expect to see any use of the standard library, not even std::string.

And even if they had used C++ to its fullest, you soon run up against the edges of that "fullness".

You want to cope half-way sensibly with non-ASCII text? You'd better learn and use ICU4C then. (Sure, C++ gets along okay-ish on its own in an environment where its input is UTF-8 and it doesn't actually have to process text so much as just pass it around. But as soon as you need to either deal with encodings other than UTF-8 or need to actually process the text, you need ICU4C.)

Or maybe you'd like threads? Or maybe you need concurrent collections? Perl regular expressions?

I can't remember the last program I wrote that didn't need all of these things.

I'm not saying that all Free Java libraries are great. Of course they're not. 90% of everything is crud. But you don't need nearly as many third-party libraries because what you get "built in" is mostly pretty good, and certainly covers a lot of ground. And it's much easier to write a decent Java library than a decent C/C++ library. I've yet to see a Java library invent its own string class, for example. And there's much less effort expended on questions of ownership (though there are certainly traps for the unwary, should they hand out mutable objects they intend to continue using).

Why do you keep fooling yourself?

I fear C and C++ fall into the same class as awkward editors. Why do people use emacs or vi rather than a proper editor? Why do people make scale models of intricate buildings using only matchsticks? Sometimes the very fact that a thing is hard is what makes it fun. (Actress, bishop.)

I evidently don't know how to stop myself falling into this trap, but maybe it will help I come out in public more often and confess. Along the lines of "hi, I'm Elliott, and I wasted a couple of days trying to start my project in C++, fighting an awkward language and a stunted standard library and several really badly-built libraries, and in the end I realized I had nothing to show for these two days, came to my senses, started again in Java, and after just a couple of hours I had something I could actually use".

It's not like using sensible tools takes all your fun away; it's just a way of trading a puerile unproductive kind of fun for the fun of doing a good job of something useful, in a shorter time than the job would otherwise have taken. Trading short-term gratification for long-term satisfaction.

I guess I'll be fighting that part of human nature for a long time.

mp3d

The few lines of code in last post, Registering a service with DNS-SD from Java were lifted directly from one of the things I'm working on at the moment. As you could tell from the service name, this program is called mp3d. If you know what an mp3 is, and you know that Unix tends to name daemons something ending in 'd', you've probably got the picture already.

All I wanted was a little web server that would let me play my music from any computer, with the sound coming from the one computer that has decent speakers.

At this point, you may well be muttering "reinventing the wheel again, dude?". And you'd be forgiven for thinking so. I certainly assumed that the internet must be positively dripping with examples of the program I was looking for. Indeed, I spent as long searching the web for such a thing as it actually took me to cobble together the first version when I finally gave up looking.

(If you ignore my unfortunate detour into the world of C++ that I may talk about some other time. For now though, let's pretend I was a sensible boy and chose Java from the outset.)

Distraction: RhythmWeb

The first candidate I found on the web was a plugin for RhythmBox, which sounded ideal. I was at that time using RhythmBox as my mp3 player, and although RhythmBox isn't great, I could have lived with a web interface to it.

Unfortunately, RhythmBox and RhythmWeb didn't get on very well with each other. I'd hoped for some kind of unified state, but they seemed to be fighting amongst themselves and keeping track of their combined state (and thus what my actions would do) was pretty confusing.

I could have stopped using RhythmBox locally and used the web browser even on the box with the speakers, but there was a worse problem: RhythmWeb was slow. So slow it was almost impressive. I don't have a large mp3 collection – less than 5000 tracks – but RhythmWeb couldn't cope with that collection. Not in any reasonable time frame, anyway. Not even when only asked to display a few matches. Python: all the elegance of a silly walk, and all the speed of a dead parrot.

RhythmWeb did look quite nice, though, in a minimalist kind of way. I've tried to keep something of that style in mp3d.

Distraction: mpd

The only other candidate I came across that seemed close to what I was looking for was mpd. But that turned out to not be quite what I expected. My idea was basically "a web server that lets me play my music". What I found in mpd was more "a server that would play my music if I ran an appropriate client to make the necessary RPC calls".

There followed an interlude where I looked at a bunch of clients before eventually giving up and deciding to write my server.

Frankenstein's Mp3 Player: mp3d

I started off with directory-indexing code copied & pasted from one of my other projects. I started with the id3v2 tag parser recommended by Jesse Wilson in his roundup (though I eventually wrote my own, which I may talk about another time). I used Sun's com.sun.net.httpserver.HttpServer which was a blessed relief after suffering libmicrohttpd in the C++ version. I remembered just about enough HTML to knock together a TABLE. I copied RhythmWeb's CSS. I spawned mplayer(1) to actually play the mp3s. That was the first version, done in a couple of hours.

The code's all in code.google.com/p/enh/, home to projects that are still in early states (and dumping ground for dead projects). mp3d's been through a few changes since the first version, and it's got a way to go still, but it's already the only mp3 player used on any of the computers at home. You're welcome to try it if you're interested, but I'd no intention of packaging it in the near future, so you'll have to build from source.

I bring it up mainly as an introduction to the next few posts, which will cover various things that cropped up while working on mp3d.

Registering a service with DNS-SD from Java

DNS-based service discovery (DNS-SD, but better known as Rendezvous, Bonjour, or ZeroConf) is a great way for services to announce themselves, and for potential clients to see what services are available. Mac OS has mDNSResponder, Linux has Avahi, and Windows – as far as I know – has its own competing alternatives.

I've played with this stuff a few times in the past, but never really needed it. Recently though, I wrote a small web service that will mainly be used from Mac web browsers on the same network. What's the easiest way of finding the server? Probably the fact that registered _http._tcp services automatically appear in both Camino and Safari's bookmarks menus. Unfortunately, Firefox does not support anything like this, even on Linux. Nor, as far as I'm aware, does Chrome. (Epiphany does, though, if you're looking for a Linux browser with DNS-SD support.)

So. What's the easiest way to get a Java HTTP service registered?

avahi-publish(1)

Since my server is running on Linux, and I have the avahi-utils package installed, avahi-publish(1) is a convenient choice. Given my ProcessUtilities class, it's just one extra line to spawn a child that publishes news of my service. And, conveniently, that child dies when I die, so I don't need to do anything special about deregistering. Normal process semantics take care of things well enough.

Here's the single line it took to get my service (called "mp3d") registered:

ProcessUtilities.spawn(null,
    "/usr/bin/avahi-publish", "-f",
        "-s", "mp3d", "_http._tcp", Integer.toString(portNumber));

Pros: trivial to use, from any language.

Cons: Linux-only.

Avahi bindings

I did look at Avahi's bindings, but they don't include Java. And since that wouldn't actually solve the "con" of avahi-publish(1), it's not really obvious that I'd gain anything by going down such a route even if it were open to me.

Pros: none.

Cons: neither useful nor written yet.

mDNSResponder

Apple published mDNSResponder as open source under the APL 2. It supports Mac OS (even the prehistoric pre-Unix variant that it's believed the cavemen used), Windows, and "POSIX platforms". This does come with Java bindings, but they're just that: bindings to the native code using JNI.

This would provide portability, but by this time I'd realized that ideally, I wanted a pure Java solution.

Pros: somewhat portable.

Cons: awkward JNI dependency, recompilation requirements.

waiter

The waiter project is very proud about being written using Test Driven Development and what kind of mocks it uses, but not terribly interested in telling you how to use it. Or what state of development it's at. The code looked a bit odd, too. More like a Java-hater's parody of Java than anything.

Pros: none.

Cons: no documentation or sample code, weird attitude, not obviously alive.

mahalo-mdns

The mahalo-mnds mahalo-mdns project was a fork of JmDNS. A fork on which apparently nothing of substance happened.

Pros: none.

Cons: dead.

JiveDNS

The JiveDNS project was a fork of JmDNS. A fork that apparently didn't get as far as actually committing any code.

Pros: none.

Cons: dead.

JmDNS

So what about the project everyone else seems to fork but then not do anything with? This appears to be the best of the bunch, though it's not without problems.

I tried version 2.1, and the API looked reasonable. More Java-like than the others, certainly. The documentation seems to think the API uses constructors, but the code itself has changed. Luckily, the example programs it ships with have been kept up to date.

I noticed that the project's site has bug reports about deadlock and dodgy use of JVM shutdown hooks going back years. Given that the second of their example programs did indeed deadlock in a shutdown hook, leaving me having to kill -9 the JVM after suspending it, I'm ready to believe those reports.

Anyway, here's the single line I needed after dropping the jmDNS JAR file into my project's lib/jars/ directory:

JmDNS.create().registerService(ServiceInfo.create("_http._tcp.local.", "mp3d",
                                                  portNumber, ""));

And sure enough, if you add something like that to your program, you'll find your program too will deadlock if interrupted. Suboptimal.

Under other circumstances, I'd suggest jmDNS was ripe for a forking, but that doesn't seem to have worked in the past!

Pros: portable.

Cons: deadlock-prone, under-maintained.

Conclusion

So what am I going to use? avahi-publish(1). I was only expecting to run on Linux anyway, so I'll worry about portability when I come to need it.

But... it turns out that I can't use DNS-SD for what I originally wanted it for. My plan was to switch from a hard-coded port number for my HTTP server (8888, just like we used to use in the good old days) to just accepting whatever ephemeral port I got. I thought that if I announce my service using DNS-SD it wouldn't matter what port I was on.

To a certain extent that's true. The missus doesn't need to know the port number if she uses the Bonjour bookmark to get there. But if she wants to get there via the history and the server's restarted, she's screwed. Likewise if the server restarts while she has a browser window open, when she comes back to that window, it's "broken". So it turns out I can't really get away with accepting an ephemeral port after all. This seems like a real pity. Maybe one day we'll be able to use DNS-SD service names as virtual port names, as in http://lithium.local.:mp3d./ say. But not today.

So it's back to the four fat ladies of my hard-coded port number, but I'll leave the DNS-SD registration in anyway. I'm used to it now, and it'll be a comfort in my old age, when my memory's completely gone.