2008-11-24

Review: "The Old New Thing"

If you waste your time reading crap on the internet (as you clearly do), you've probably come across Raymond Chen's blog "The Old New Thing". You may also be aware that he wrote a book with the same title, and, for all I know, it's something all Windows programmers have read.

I, though, am "one of those condescending Unix computer users".

I've used Windows on and off since 1993, but I've only used Windows out of some kind of necessity. It always seemed to me to be badly designed and obscure and cluttered with legacy crap, and I might try to use that as an excuse for not liking Windows, if it weren't for the fact that you could say the same things about Unix. It may be that the real difference is just that I'm much better versed in the bad design and obscurity and clutter of Unix.

I don't actually believe that, but doesn't matter anyway: the fact is, Windows exists, and Windows is huge. One reason I've never been convinced by the argument that "Linux will win the desktop because it costs nothing, and people love not paying for stuff" is that you don't have to strike up too many conversations with strangers to find that the man on the street considers Windows (and MS Office and the like) to cost nothing either. Software either comes with the computer (and they get a new version when they get a new computer), or, if they're really keen, it comes from work or a friend.

People proudly tell me they copy Windows, which I've always found strange on many levels. But that in itself tells you something about Windows' ubiquity and how it's perceived.

Ignoring Windows is like ignoring religion. You may personally think it's a load of shite, but you can't deny its influence, and that influence alone makes it worthy of study.

Although I sometimes read posts on Chen's blog when they're linked to from places I read, and I was aware of the book's existence, I hadn't been curious enough to go out and buy "The Old New Thing", but I saw a copy on a shelf at work, borrowed it, and read it over this past weekend.

I loved it. It's full of random stuff from a perspective quite different from my own, but with an obvious shared heritage of double-frees and memory leaks and performance problems and so forth. There were whole chapters I basically skipped over (chapter 15, "How Window Messages Are Delivered and Retrieved", for example), but the range of the book is sufficiently diverse, and Windows' influence so far-reaching, that there's bound to be something of interest.

Topics range from memory allocation (and problems all the way from 16- through 32- to 64-bit), concurrency primitives, text rendering, debugging, reverse engineering, to why various Windows features/APIs are the way they are.

I liked the debugging tips slipped in here and there; the kinds of things you learn on the job rather than in class. "The poor man's way of identifying memory leaks", for example, talks about just letting a program leak away, and seeing what the heap's full of. And the fact that the caller whose allocation attempt causes your system's out of memory failure is probably responsible (despite your temptation to assume innocence unless proven guilty). And that the former can help you confirm or deny the latter. These are the kinds of things that aren't usually considered academic enough to be written down, but are still bloody useful.

Sure, you and I know that one, and probably most of the other debugging tips in the book (though there were also common Windows API mistakes, and thus fairly meaningless to me), but we also know people who still don't know this stuff, and we can probably still remember the time when we didn't know it either.

If, on the other hand, you're thinking "dude, get better tools", just wait: one of these days you'll be looking at a customer's production system, or looking at a postmortem, or a system too large or new or small to be able to afford better tools.

The Windows-specific stuff is sometimes a bit annoying, but it's only to be expected in a book like this, and it's not too hard to work out what's going on. For example, it seems like Windows uses the terms "brush" and "pen" for something like (but not exactly the same as) Java2D's Paint and Stroke respectively. It's easy enough to work out, and the (surprisingly few) sections that are obviously of no relevance outside Windows itself are easily skimmed over.

The book opens with a chapter of anecdotal examples of the kinds of lessons the Linux desktop still needs to learn. It's interesting at many points in the book, in fact, to contrast the three camps. Chen usually only talks about the Windows camp, for obvious reasons, and their position can seemingly be summarized as "we started this a long time ago, on really crappy hardware, and we know better now, and try to do new stuff in a way that shows we've learned some of these lessons, but our business is selling OS upgrades, and every incompatible change we make costs us a sale, so we go to ridiculous lengths to avoid ever breaking anything, even stuff that only ever worked by coincidence, or that deliberately went out of its way to fiddle with undocumented internals".

Mac OS is pretty much at the opposite end of the scale, with "we know that you can't make an omelet without breaking eggs, so eggs will be broken if we consider it to be in our greater good, and you get about one year of deprecation and the year after that, your code breaks; we're a hardware company, so we know people will buy the new release anyway, even if only because it comes 'free' with your next machine, but actually, they'll probably buy it anyway, because our main customers are individuals, not corporations, and they like shiny new stuff and don't have any DOS applications written in assembler in 1982 that they've long since lost the source to".

Linux? "Nothing really works very well yet, but it's free and Free, and you can fix it yourself, and it doesn't matter anyway because next week we're going to rip out all this half-working stuff and replace it with stuff that fixes a few things that currently don't work while at the same time breaking things that did work; it doesn't matter because we don't have customers, and our non-customers don't have any software they don't have the source to anyway".

Of course, you never hear these philosophies stated explicitly like that. Apple's too secretive, no-one can seriously claim to speak for "Linux", and I've no idea whether Chen is really representative of Microsoft, assuming it's any more possible to be representative (in the non-legal sense) of a company that large than it is to represent "the Linux community". But Chen has a clear voice, and it's an interesting perspective to me, especially presented through the course of a book, rather than in bits and pieces as I read occasional linked-to blog posts.

If you want just one contrast, try this, from "Why does DS_SHELLFONT = DS_FIXEDSYS | DS_SETFONT" (someone spank the editor who changed == to = there!):

... This allowed people to write a single program that got the Windows 2000 look when running on Windows 2000 and got the classic look when running on older systems. (If you make people have to write two versions of their program, one that runs on all systems and one that runs only on the newer system and looks slightly cooler, they will usually not bother writing the second one.)

This, to me, was a revelation. By "people", he means, of course, "Windows programmers". But how interesting that, from my position outside the Windows camp, I actually did a double-take when reading his parenthetical comment. Surely, I thought, he means they won't bother writing the backwards-compatible version? Let the dead bury the dead, and all that? Isn't "looks slightly cooler" an important selling feature, even for Windows (the OS)?

Try to find a Mac application (that's still maintained) that still runs on 10.2, say, and you'll not have much luck. Try to run the latest version of a Linux app on Ubuntu 8.04 (the "long-term" release that some people will be stuck with until 2010-04) without resorting to building it yourself from source... and even then, you'll probably need to build the latest versions of umpteen different libraries it depends on.

I was well aware that Microsoft bend over backwards to keep stuff working (and I was disappointed that Chen doesn't actually go into much detail on the techniques they use, which was something I would have been particularly interested to read about), but I didn't realize the backwards-compatibility-above-all mindset extended to third-party developers too, or is at least perceived to do so.

Going back to some of the other topics, I was interested to hear both that Windows gets the old "pointer change without mouse movement" problem right, and how, and that it causes complaints (presumably from people who haven't suffered systems that get it wrong), and that no-one appreciates the effort much because the related "pointer change when a component scrolls under the mouse" problem needs to be solved per-application, and often isn't. I've always hated programming for systems that won't even let me get those details right, and it's interesting to hear a Windows programmer ("of all people!") saying how much he hates it when apps don't get those details right.

The weird part is that in the final chapter of the book he mentions several "funny stories" that are actually just annoying bugs in Microsoft software, but which he seems to see as cases of user error. It's almost like the last chapter is written by a different person, that depressing guy you always imagine works at Microsoft, cranking out shit code with a shit UI and with no interest in rising to the challenge of writing something that works well under all the awful constraints imposed on it, but who's just interested in getting all the relevant forms filled and boxes checked so he can move on to the next task he also doesn't care about.

Maybe Chen is that guy, at times, and his editor asked for an extra 20 pages Chen didn't want to write? Maybe we're all that guy at times, though most programmers I've known tend to be good at losing tasks in their to-do lists, because there's always more stuff to do than there is time to do it.

What else did I like? I liked the repeated mentions of the need for a holistic view of performance: that making one thing better at the expense of some other thing isn't necessarily a good idea. There are repeated reminders that it's bad to assume that your application is the most important application in the user's day, every day. (You'd hope this wouldn't even need mentioning, but everyone who's ever used software knows it does.)

Speaking of things that shouldn't need mentioning, I was amused to see an analog of something I first came across in a Studs Terkel interview with a Republican ex-governor of a state like Montana. This old Republican was criticizing modern politicians on both sides, complaining that they'd lost the insight of looking at any new legislation as if it had been brought up by the opposition for the opposition's own use. His point being that at some point, anything can and will be used against you. And if you don't like the look of it from the unfriendly end of the barrel, maybe you ought not be handing it out in the first place. Chen's version is more like "what could a virus writer do with this API?" or "if I was a user, how could a developer make a really annoying application with this API?".

One of Chen's code snippets had me mystified. I'll ignore the use of SIZE_T instead of size_t and assume I don't want to know, but there was a bit of C++ that contained this:

// Since this is just a quick test, we're going to be sloppy
using namespace std; // sloppy

The code contains three lines that would need an explicit "std::", none of which, with the "std::" added, would be as long as the "sloppy" comment. Bizarre. And a missed opportunity to show a handy trick I learned from Martin Dorey. Don't want to keep repeating std::cout (which were two of the three uses of "std::" in Chen's code)? Do this:

std::ostream& os(std::cout);
os << blah;

This is nice even in code that's "just a quick test", but it's better still when that "quick test" turns into production code, and you need to refactor so you can pass in an arbitrary stream. Your code is already referring to just "os" (the usual name for an ostream), so all you need to do is turn the local "os" into an extra argument. Job done.

Returning one last time to things I liked, there was an explanation of why NTFS keeps a Unicode case mapping table on disk, which is something I'd wondered about, but obviously never enough to ask. And the weird code I just mentioned was actually in a section titled after the only thing I knowingly quote from a Microsoft developer, Rico Mariani: "a cache with a bad policy is another name for a memory leak".

Would I buy this book? No. It's not a classic, or a reference work I can imagine I'd keep coming back to. But it was an enjoyable read, and an interesting insight into a parallel universe. Borrow.