signal(2) versus sigaction(2)

Normally, I'm pretty gung-ho about abandoning old API. I don't have head space for every crappy API I've ever come across, so any time there's a chance to clear out useless old junk, I'll take it.

signal(2) and sigaction(2) have been an interesting exception. I've been using the former since the 1980s, and I've been hearing that it's not portable and that I should be using the latter since the 1990s, but it was just the other day, in 2010, that I first had an actual problem. (I also knew that sigaction(2) was more powerful than signal(2), but had never needed the extra power before.) If you've also been in the "there's nothing wrong with signal(2)" camp, here's my story...

The problem

I have a bunch of pthreads, some of which are blocked on network I/O. I want to wake those threads forcibly so I can give them something else to do. I want to do this by signalling them. Their system calls will fail with EINTR, my threads will notice this, check whether this was from "natural causes" or because I'm trying to wake them, and do the right thing. So that the signal I send doesn't kill them, I call signal(2) to set a dummy signal handler. (This is distinct from SIG_IGN: I want my userspace code to ignore the signal, not for the kernel to never send it. I might not have any work to do in the signal handler, but I do want the side-effect of being signalled.)

So imagine my surprise when I don't see EINTR. I check, and the signals are definitely getting sent, but my system calls aren't getting interrupted. I read the Linux signal(2) man page and notice the harsh but vague:

The only portable use of signal() is to set a signal's disposition to
SIG_DFL or SIG_IGN. The semantics when using signal() to establish a
signal handler vary across systems (and POSIX.1 explicitly permits this
variation); do not use it for this purpose.

POSIX.1 solved the portability mess by specifying sigaction(2), which
provides explicit control of the semantics when a signal handler is
invoked; use that interface instead of signal().

It turns out that, on my system, using signal(2) to set a signal handler is equivalent to using the SA_RESTART with sigaction(2). The Open Group documentation for sigaction(2) actually gives an example that's basically the code you'd need to implement signal(2) in terms of sigaction(2).) The SA_RESTART flag basically means you won't see EINTR "unless otherwise specified". (For a probably untrue and outdated list of exceptions on Linux, see "man 7 signal". The rule of thumb would appear to be "anything with a timeout fails with EINTR regardless of SA_RESTART", presumably because any moral equivalent of TEMP_FAILURE_RETRY is likely to lead to trouble in conjunction with any syscall that has a timeout parameter.)

Anyway, switching to sigaction(2) and not using the SA_RESTART flag fixed my problem, and I'll endeavor to use sigaction(2) in future.

Assuming I can't stay the hell away from signals, that is.

Alternative solutions

At this point, you might be thinking I'm some kind of pervert, throwing signals about like that. But here's what's nice about my solution: I use a doubly-linked list of pthreads blocked on network I/O, and the linked list goes through stack-allocated objects, so I've got no allocation/deallocation, and O(1) insert and remove overhead on each blocking I/O call. A close is O(n) in the number of threads currently blocked, but in my system n is currently very small anyway. Often zero. (There's also a global lock to be acquired and released for each of these three operations, of course.) So apart from future scalability worries, that's not a bad solution.

One alternative would be to dump Linux for that alt chick BSD. The internets tell me that at least some BSDs bend over backwards to do the right thing: close a socket in one thread and blocked I/O on that socket fails, courtesy of a helpful kernel. (Allegedly. I haven't seen BSD since I got a job.) Given Linux's passive-aggressive attitude to userspace, it shouldn't come as a surprise that Linux doesn't consider this to be its problem, but changing kernel is probably not an option for most people.

Another alternative would be to use shutdown(2) before close(2), but that has slightly different semantics regarding SO_LINGER, and can be difficult to distinguish from a remote close.

Another alternative would be to use select(2) to avoid actually blocking. You may, like me, have been laboring under the misapprehension that the third fd set, the one for "exceptional conditions", is for reporting exactly this kind of thing. It isn't. (It's actually for OOB data or reporting the failure of a non-blocking connect.) So you either need to use a timeout so that you're actually polling, checking whether you should give up between each select(2), or you need to have another fd to select on, which your close operation can write to. This costs you up to an fd per thread (I'm assuming you try to reuse them, rather than opening and closing them for each operation), plus at least all the bookkeeping from the signal-based solution, plus it doubles the number of system calls you make (not including the pipe management, or writing to the pipe/pipes when closing an fd). I've seen others go this route, but I'd try incest and morris dancing first.

I actually wrote the first draft of this post last August, and the SA_RESTART solution's been shipping for a while. Still, if you have a better solution, I'd love to hear it.

Java on a thousand cores

Cliff Click wrote a recent blog post that started "warning: no technical content", which wasn't true. Buried 3/4 of the way through was a link to the PDF slides for the "Java on 1000 cores" talk he's been giving recently. I was in the audience of the specific instance of the talk he mentioned, but if you weren't, you can now watch the video on YouTube.

I tend to avoid talks because so many of them are a complete waste of time. This one, though, was so good it had me thinking about why I've mostly given up on talks.

For one thing, most speakers speak far too slowly. I know one of the rules everyone's taught about public speaking is to speak more slowly. That because you're nervous you'll speak faster than you realize, and that sounds like you're rushing, and that's bad. Which might have made sense in the 1700s when public speaking likely meant boring your congregation to sleep every Sunday morning, where the point was less the content than the submission to authority. Going slow might make sense if you're trying to educate beginners (though there I think repetition and usage is what matters, not slow presentation). I can even admit that going slow is downright necessary if you're walking someone through a physical procedure, but it's an awful way to present summaries or reviews (in the sense of "review paper"). And most talks are, one way or another, summaries or reviews.

As if looking for a less literal way in which they can speak too slowly, few speakers have a good feel for how many words any particular point deserves. You know these people. You've been in their talks. These people have their four obvious bullet points on their slide, four bullet points that you grasp in the second or two after the slide is up, and somehow they manage to spend five minutes laboriously wading their way through these points while adding nothing to your understanding. And all the time, you stare at the screen willing the next slide to appear. Wishing you'd brought your Apple remote. (They're not paired by default, you know.)

Don't be afraid to have a lot of stuff on your slides, either. The people who tell you not to have more than a couple of points per slide have nothing to say. At best, they're salesmen. (See Steve Jobs' slides and presentations. Things of beauty both, but designed for selling, not transfer of technical information and ideas.) The naysayers also assume you're making all the usual mistakes too, when they should instead tell you to speak fast and only speak useful words, in which case it won't matter how much stuff is on each slide, because no-one will be bored enough that they're sat there just reading the slides. If they're not listening, taking away the slides is not a fix for your problem.

Another bad habit is not asking anything of your audience. I don't mean questions. I mean prerequisites. Seriously, those five people at your talk about building STL-like containers who don't know what the STL is? Tell them to fuck off and stop wasting everybody's time. Or, if you think that's a bit much, just start with "this talk assumes a basic familiarity with X, Y, and Z; if you don't have this, I'm afraid you probably won't get much out of this talk". You're not helping anyone by spending ten minutes rambling on about stuff that anyone who's going to get anything out of your talk already knows anyway. Unless you're someone like Steve Jones, you probably don't need to explain science to the layman. It's much more likely you're talking to an audience of your peers, and you should respect their time as you'd expect them to respect yours.

Also, please don't waste my time with talks where the only thing I learn is how much you like the sound of your own voice, or talks that only exist because you get some kind of merit badge for having given a talk. In the former case, get a blog. Then people can decide for themselves whether they like the sound of your voice, and subscribe if they do and ignore you if they don't. In the merit badge case, corporate life is full of bogus achievements; I'm sure you can find one that minimizes waste of other people's time. Hell, if it helps, I'll make you a little "I didn't waste anyone's time giving a gobshite talk" certificate.

Et cetera.

Anyway, getting back to Cliff Click's talk... his was a great example of what talks should be like. His content was interesting and concentrated. He assumed his audience knew the basics of the areas he was talking about. He spoke fast. Fast enough that I was often still digesting his previous point while he was on to the next. (And for all you MBAs: this is a good thing. Better too much content than too little. I can always re-read the slides/re-watch the video afterwards. And sometimes thinking about the last thing I was interested in helps me coast through a bit I'm less interested in.)

One thing I particularly like about Cliff Click's stuff in general is his practitioner's point of view. He asks the important practical questions: "Have you successfully built one?", "Did it do what you expected?", "What is it good for?", "What isn't it good for?", "Is it worth it?".

As for this particular talk, there were several new ideas to me (I hadn't heard of the optimistic escape detection [as opposed to escape analysis] he mentioned in passing, for example), several things I found surprising (that write buffers turned out to be unimportant with Azul's CLZ [unrelated to ARM's CLZ], for example), and a lot of familiar stories about mixed hardware-software designs, interesting mainly because it hadn't occurred to me that they might be common to all hardware-software companies.

Plus it was a model of how to give a good talk.