2006-01-08

How does Terminal.app know what processes might die?

With Mac OS' Terminal, if you close a window that has processes still running in it, you'll be warned and asked if you really want to close the window. The warning will include all the processes' names. So if you're running top(1) from vim(1) from bash(1), you'll be told that "Closing this window will terminate the following processes inside it: bash, vim, top".

There are two obvious things these processes share. Firstly, their process ids (pids) and parent process ids (ppids) would let you work out their relationship, and Terminal presumably remembers the pid of the process it started. Alternatively, the processes will all have access to the same tty, so if you could find all those processes, you'd also have your list.

The advantage of finding the processes using the tty is that you automatically include processes whose ppid chain back to the direct descendent of Terminal is broken, and you automatically exclude processes who've given up their connection to the terminal.

So, if we want similar functionality in our Terminator terminal emulator, how are we going to get at the list? One option would be to look at the output of lsof(1). It even has a fairly convenient mode for being called by other programs. That's fine for Linux, but it's way too slow on Mac OS. It's 10 times slower, in fact. Here's Mac OS:

hydrogen:~$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.4.3
BuildVersion: 8F46
hydrogen:~$ tty
/dev/ttyp4
hydrogen:~$ time lsof -w -Fc /dev/ttyp4
p14244
cbash
p14526
clsof

real 0m1.453s
user 0m0.076s
sys 0m1.367s
hydrogen:~$ time lsof -w -Fc /dev/ttyp4
p14244
cbash
p14528
clsof

real 0m1.431s
user 0m0.076s
sys 0m1.351s
hydrogen:~$

And here's Linux:

Linux helium 2.6.12-10-amd64-generic #1 Fri Nov 18 11:51:07 UTC 2005 x86_64 GNU/Linux

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
helium:~$ tty
/dev/pts/2
helium:~$ time lsof -w -Fc /dev/pts/2
p920
cbash
p928
clsof

real 0m0.182s
user 0m0.062s
sys 0m0.119s
helium:~$ time lsof -w -Fc /dev/pts/2
p920
cbash
p930
clsof

real 0m0.182s
user 0m0.063s
sys 0m0.118s
helium:~$

Even if the performance were acceptable, though, lsof(1) on Mac OS can't recognize when top(1) is running. The reason for this is that top(1) is setuid root so it can get the information it needs. Ugh. lsof(1), on the other hand, is setgid kmem, so that it can open /dev/mem. So although lsof(1) can find out about open file descriptors, it can only find out about ones for processes running as the same user, which won't include setuid programs like Mac OS' top(1).

hydrogen:~$ ls -l `which top`
-rwsr-xr-x 1 root wheel 83088 Mar 20 2005 /usr/bin/top
hydrogen:~$ ls -l `which lsof`
-rwxr-sr-x 1 root kmem 111356 Mar 26 2005 /usr/sbin/lsof
hydrogen:~$

So lsof(1) doesn't look like it's going to be so convenient after all.

If you try a few plausible Google searches or play about with apropos(1), you'll likely come across kvm(3). In particular, kvm_getprocs(3), which lets you pass KERN_PROC_TTY to get kinfo_proc structures for all matching processes.

It's badly documented, but it's not too hard to work out. The main trick is working out that the KERN_PROC_TTY parameter is the st_rdev field.

#include <iostream>

#include <fcntl.h>
#include <kvm.h>
#include <sys/sysctl.h>
#include <sys/stat.h>

int main(int argc, char* argv[]) {
const char* tty_name = argv[1];

struct stat sb;
if (stat(tty_name, &sb) != 0) {
perror("stat");
return 1;
}

kvm_t* kvm = kvm_open(0, 0, 0, O_RDONLY, "kvm_open");
if (kvm == 0) {
return 1;
}

int count = 0;
kinfo_proc* procs = kvm_getprocs(kvm, KERN_PROC_TTY, sb.st_rdev, &count);
for (int i = 0; i < count; ++i) {
std::cout << procs->kp_proc.p_pid << " "
<< procs->kp_proc.p_comm << std::endl;
++procs;
}

int result = kvm_close(kvm);
return result;
}

This solution is nice and fast, and it gives us the right answer in the "bash(1) running vim(1) running top(1)" case, but because it needs access to /dev/mem it needs to have privileges at least equivalent to setgid kmem or you'll get "kvm_open: /dev/mem: Permission denied". Here it is running as root:

hydrogen:~$ time sudo ./kvm /dev/ttypd
17762 top
17724 vim
6603 bash

real 0m0.032s
user 0m0.004s
sys 0m0.015s
hydrogen:~$

So this gives us the right answer, in a sensible amount of time, but its requirements are unacceptable.

So, what would Brian Boitano do? Let's ask the kernel using ktrace(1) and kdump(1), Mac OS' much less convenient equivalent of strace(1). Here's the relevant excerpt, with a bit of context:

198 Terminal RET read 24/0x18
198 Terminal CALL read(0x21,0xf2243bd0,0x200)
198 Terminal CALL __sysctl(0xbfffe5d0,0x4,0,0xbfffe5f0,0,0)
198 Terminal RET __sysctl 0
198 Terminal CALL __sysctl(0xbfffe5d0,0x4,0x1996800,0xbfffe5f0,0,0)
198 Terminal RET __sysctl 0
198 Terminal CALL __sysctl(0xbfffe910,0x4,0,0xbfffe930,0,0)
198 Terminal RET __sysctl 0
198 Terminal CALL __sysctl(0xbfffe910,0x4,0x1996800,0xbfffe930,0,0)
198 Terminal RET __sysctl 0
198 Terminal CALL __sysctl(0xbfffe910,0x4,0,0xbfffe930,0,0)
198 Terminal RET __sysctl 0
198 Terminal CALL __sysctl(0xbfffe910,0x4,0x1997400,0xbfffe930,0,0)
198 Terminal RET __sysctl 0
198 Terminal CALL __sysctl(0xbfffe420,0x4,0,0xbfffe440,0,0)
198 Terminal RET __sysctl 0
198 Terminal CALL __sysctl(0xbfffe420,0x4,0x1996800,0xbfffe440,0,0)
198 Terminal RET __sysctl 0
198 Terminal CALL __sysctl(0xbfffd6c0,0x4,0,0xbfffd6e0,0,0)
198 Terminal RET __sysctl 0
198 Terminal CALL __sysctl(0xbfffd6c0,0x4,0x1996800,0xbfffd6e0,0,0)
198 Terminal RET __sysctl 0
198 Terminal CALL ppc_gettimeofday(0xf0e9c8c0,0)
198 Terminal RET ppc_gettimeofday 1136687072/0x43c077e0

So it looks like it's calling sysctl(3). Looking at the man page, there's an example that fetches process information for processes with pids less than 100 that suggests we're on the right track, and further down there's this:

KERN_PROC
Return the entire process table, or a subset of it. An array of
pairs of struct proc followed by corresponding struct eproc
structures is returned, whose size depends on the current number
of such objects in the system. The third and fourth level names
are as follows:

Third level name Fourth level is:
KERN_PROC_ALL None
KERN_PROC_PID A process ID
KERN_PROC_PGRP A process group
KERN_PROC_TTY A tty device
KERN_PROC_UID A user ID
KERN_PROC_RUID A real user ID

The system header files suggest that (as the example earlier in the man page implies) we can ignore that stuff about pairs of structs, and just talk in terms of kinfo_proc. As with the kvm functionality, the tty device is the st_rdev field from stat(2).

There's a slight complication in that with sysctl(3) we have to allocate memory for the table ourselves, and we need to do the usual Unix dry-run trick so we know how much space to allocate. Here's the code:


#include <sys/types.h>
#include <sys/sysctl.h>
#include <sys/stat.h>

#include <iostream>
#include <vector>

int main(int argc, char* argv[]) {
const char* tty_name = argv[1];

struct stat sb;
if (stat(tty_name, &sb) != 0) {
perror("stat");
return 1;
}

// Fill out our MIB.
int mib[4];
mib[0] = CTL_KERN;
mib[1] = KERN_PROC;
mib[2] = KERN_PROC_TTY;
mib[3] = sb.st_rdev;

// How much space will we need?
size_t len = 0;
if (sysctl(mib, sizeof(mib)/sizeof(int), NULL, &len, NULL, 0) == -1) {
perror("sysctl test");
return 1;
}

// Actually get the information.
std::vector<char> buffer;
buffer.resize(len);
if (sysctl(mib, sizeof(mib)/sizeof(int), &buffer[0], &len, NULL, 0) == -1) {
perror("sysctl real");
return 1;
}

// Dump it.
int count = len / sizeof(kinfo_proc);
kinfo_proc* kp = (kinfo_proc*) &buffer[0];
for (int i = 0; i < count; ++i) {
std::cout << kp->kp_proc.p_pid << " "
<< kp->kp_proc.p_comm << std::endl;
++kp;
}
return 0;
}

And see how it runs:

hydrogen:~$ time ./sysctl /dev/ttypd
17762 top
17724 vim
6603 bash

real 0m0.007s
user 0m0.001s
sys 0m0.006s
hydrogen:~$

We get the right answer, really quickly, and we don't need any special privileges. Awesome!

The only bummer is that Linux's sysctl.h doesn't include KERN_PROC_TTY, so I guess we'll have to grub around in /proc or call lsof(1) there.