A kernel of truth: the plot thickens

John Siracusa
As usual, I got some great email and comments onmy last postabout the rumblings surrounding the Mac OS X kernel. Two comments inparticular have prompted this quick follow-up. The first was from theever-inscrutableRosyna who pointed out a blindingly obvious reason for the still-unreleased x86 kernel source code.
It seems you have missed the most logical reason for Applenot releasing the source to the x86 tree of the kernel. Namely, that itmay contain third party source code (for Rosetta support) that theycannot release.
Duh, I had entirely forgotten about Rosetta (a testament to itstransparency, maybe?) and the licensed technology it‘s based on. Theneed to trim out all the (presumably) closed-source bits licensed fromTransitive would certainly put a damper on timely x86 kernel source publication.
Assuming it‘s possible to release a kernel source tree with all theRosetta code carefully removed, two questions arise. First, is that a reasonable thing to do? Part of the point of releasing source is so that otherscan help find and fix bugs, both in the kernel itself and in their owncode. But if a Transitive-free kernel is significantly different thanwhat‘s actually shipping, how much of a help is it to have this source?
Second, assuming the removal of Rosetta doesn‘t move the source toofar from the shipping kernel to be useful, then why hasn‘t it beenremoved by now? It‘s been months since the x86 kernel source wasexpected. Surely if it‘s just a matter of making sure all proprietarycode is excised, it‘d be done by now.
This, in turn, leads into my theory about the Tiger x86 kernelsource being an evolutionary dead-end. If it‘s a pain to remove all theRosetta code and the shipping x86 kernel bears little or no relation to the Leopard kernel, then the "stall untilAugust" strategy makes perfect sense.
The second comment, from Michel Galle, touched on a topic that I meant to address.
Apple clearly said while the tiger release than there will no change in the kernel api for years to come, to stabilize kernel extensions and drivers. So I cannot fathom a change of kernel, it will totally break "kernel api freeze."
This topic was alluded to by the link to thekernel section of my Tiger reviewin the last paragraph of my earlier post. The introduction of a stable,supported kernel API was perhaps the most important core OS change inTiger. But I don‘t agree with the conclusion drawn by Michel. In fact,I think the stable kernel API and abstraction layer added in Tigerlends support to the theory that major kernel changes are coming in Leopard. Abstraction is exactly what‘s needed when you plan to change the underlying implementation.
This, finally, leads to the big question lurking behind the GreatKernel Swap theory. If Mach is being replaced, what will it be replaced with? I haven‘t bought into the Glamour Choices: FreeBSD and Linux. Althoughmoving to the Linux kernel would probably be a PR coup, it‘s apolitically and legally incompatible—not to mentionAPI-incompatible—choice.
The FreeBSD kernel is the obvious contender. Apple uses a bunch ofthis code already. But Mac OS X also takes advantage of many of thefeatures that are unique to Mach. Entirely replacing themicrokernel-welded-to-a-BSD-kernel monster with a plain old BSD kernelwould require some significant re-plumbing and sacrifices.
For the sake of completeness, I will also explicitly pooh-pooh the Crazy Choices. Jonathan Schwartz‘spipe-dreamsnotwithstanding, the OpenSolaris kernel, with its hardcoreserver/enterprise focus, is just too big of a philosophical mismatch.Never mind that partnering with a potential hardware and softwarecompetitor on the core of your flagship software product is not such ahot idea regardless of the technological issues. Speaking of which,forget about "the NT kernel" or "the Vista kernel" or any other theoryinvolving Microsoft in any way. (Save it for the entirely separate"Apple switching to Windows" conspiracy theory where it belongs.)NuKernel, the BeOS kernel, QNX, Newton: no.
So what‘s left? To find a spiritual and technological successor toMach, it helps to understand the history. Mach started as aproject at Carnegie Mellon University, and had a long life in academia. It spawned many other microkernel projects, includingMach 4at the University of Utah and several offshoots inspired by Mach thatdon‘t share the "Mach" name at all. Many of these projects attempt toaddress the shortcomings of Old Man Mach.
Historically, the operating system that started as NeXTSTEP and hasbecome Mac OS X today has lagged behind the cutting edge of academicresearch when it comes to kernel selection. For example, NeXTSTEP stuckwith a variant of Mach 2.x long after Mach 3.0 was the latest andgreatest. Today, the microkernel portion of the Mac OS X kernel remainsa mishmash of many Mach versions, including Mach 3. But even Mach 4 ispretty old news. Whither the new hotness?
A little Googling around the topic of operating system research will bring up some prettycrazystuff. That‘s academia for you. What Apple needs is something that‘sclose enough to a traditional microkernel to be feasible replacementfor Mach, modern enough to have addressed most of Mach‘s shortcomings,but old enough to have been proven to some degree in the real world.
As you might have guessed, I have a contender. It‘s a microkernelthat should be familiar to most people who have studied operatingsystems in the past decade or so. It‘s calledL4. LikeLLVM, another semi-obscure infrastructure-type project that Apple hasrecently taken a shine to,L4 is just the right size and character to be adopted by Apple. It‘snot too big, like Linux or Solaris or anything else with tons ofpreexisting developers and legacy baggage, nor is it completely obscureand untested. L4 seems just right.
Of course, it‘s still not quite a drop-in replacement for Mach,which means that there‘s still the issue of getting the existing Mac OSX driver model and welded-on BSD kernel to talk to L4 in the same waythat they talk to Mach. But hey,would you look at this...
The L4/Darwin project is an experimental port of Darwin tothe L4 microkernel to study the characteristics of a large-scalemicrokernel-based system. It includes a port of IOKit to L4, a modifiedlibc to communicate with the Darbat Server, and of course XNU with manyof the machine-dependent parts heavily modified (pmap, thread/taskcreation, etc) but much left unchanged (most of BSD, and large parts ofOSFMK work without modification).
That seems mighty convenient, doesn‘t it? It‘s not a slam dunk, butif Mach really is being replaced in the Leopard time frame, L4 lookslike the prime contender to me. WWDC can‘t come soon enough...
|Next »
Posted June 19, 2006 @ 11:56AM byRayiner Hashem
The L4 move is perhaps more unlikely than even the FreeBSD one. L4isn‘t just "not a drop in replacement for Mach", it‘s based on acompletely different paradigm. At the most fundemental level, IPC, L4is a synchronous RPC model, while Mach is an asychronous ports model.Moreover, L4 is minimal in the extreme. Much of the functionality thatthe BSD layer depends on Mach to provide is not provided in L4.Replacing Mach with L4 would be an enormous task, likely no lessonerous than just porting OS X to the FreeBSD kernel. Doing itproperly, in a way as to benefit from the performance of L4, wouldrequire even more work. A high-performance OS based on L4 looks nothinglike one based on Mach, because L4 is a fundemental rethinking of whatthe kernel/server interface looks like. It certainly doesn‘t lookanything like the existing Darwin BSD layer.
Youpoint to Darbat, but I think you‘re overlooking something. Darbatdoesn‘t replace Mach with L4 --- it runs XNU (Mach and all) on top ofL4. That is, XNU runs in more or less its complete form, as a systemserver on top of L4, as Linux does in L4/Linux. Mach is still therehandling paging, threading, etc, and all the weaknesses of Mach thatyou might want addressed in a replacement remain.
Posted June 19, 2006 @ 11:59AM byemagius
L4 seems to be a solid microkernel and one that could be what Apple‘sgoing for, but there are a plethora of choices. I don‘t think that wehave enough information to call it -- or anything else -- a primecontender. (That said, I also find it unlikely that Apple will replaceMach for 10.5 (perhaps 10.6 or whatever follows).) Darbat is a redherring, anyhow; even if Apple has a hand it it, it won‘t be ready andproperly tested in time for 10.5.