PE U: The Mac OS X Leopard Windows API Myth
December 5th, 2007
Daniel Eran Dilger
Some ideas just won’t die. Proponents of the Mac OS X Leopard Windows API Myth are so convinced that Apple desperately needs to wedge Microsoft Windows into Mac OS X that they’ll run with any hint that might suggest a plausible way for this to happen. The latest take on the subject is that Mac OS X Leopard loads PE files and requests Windows DLL files, which more than a few pundits have determined must be a new development because Tiger didn’t do this. Therefore, they’ve decided that the only sensical conclusion to jump to is that Apple is secretly implementing the Windows API so that Macs will be able to run Windows programs natively. They’re wrong, here’s why.
This All Happened Before.
Last year, I highlighted the Red Box Myth among the top ten Apple-related myths of the year. While involving new details, the latest incarnation of this myth fails its sanity check for many of the same reasons.
The idea of a “Red Box” that natively runs Windows applications on the Mac desktop is easy to conceptualize, just like time travel. The problem is that actually delivering the technology is a different matter, just like time travel. Further, actually accomplishing it might mess things up in ways that can only be speculated about, just like time travel. But of course, just like time travel, harsh realities haven’t stopped imaginative minds from excitedly writing about the concept.
Mark Stephens–writing as Robert X Cringely–brought up the Red Box Myth last year and speculated into existence the idea that Apple could deliver this without too much work because of its 1997 patent sharing agreement with Microsoft. He suggested that because the two companies agreed not to sue each other over their respective patent portfolios, that Apple has somehow gained access to both the source code of Windows and a perpetual, universal license to deploy and amend it in any way Apple chose. For the record, that is absolutely absurd.
Really, Really Hard Problems.
In reality, even having full and unfettered access to Microsoft’s code, development tools, and strategic plans for the future would not make it easy or simple for Apple to deliver the “Windows API” on the Mac. The general public rarely considers just how difficult it actually is to deliver complex software. Consider:
1. Microsoft itself struggled to deliver Vista over a seven year period, starting over several times. Despite “being” the same Windows API with some additions and changes, Vista still has hardware compatibility issues, significant performance problems, and lots of security and stability problems for Microsoft to address over the next several years.
2. Apple similarly worked for years to deliver the classic Mac API as the Carbon environment in Mac OS X. This took five years to fully deliver, and has been in constant refinement over the last five years since Mac OS X 10.2 Jaguar appeared in 2002. Integrating Carbon into NeXTSTEP to deliver Mac OS X required extensive changes to both.
Despite the fact that Apple fully owned both environments and had scores of engineers intimately familiar with them, there were still battles about how to best deliver the features of both sides together into a single package. Integrating in a Windows environment would be far more complex, because Apple couldn’t change Windows to make it fit. At the same time, Microsoft’s definition of Windows would be evolving in an independent direction.
3. The Open Source community has several independent projects that aspire to deliver workalike alternatives to the Windows API, including the ReactOS project to clone Windows NT, and the Wine project to map Windows API functions to the native API of Unix-like operating systems. Wine has been in progress for 14 years and is still considered an experimental beta.
All three projects were phenomenal undertakings involving lots of very talented engineers. If any of them had the capacity to make their respective products better or deliver them faster, they would have. All three represent the state of the art in what those groups can deliver.
Solvable Problems.The idea of running Windows applications on the Mac isn’t an unsolvable problem. Even PowerPC Macs could run Windows apps in emulation, by hosting a full Windows install on top of a software layer that presented itself as a virtual PC environment. On the new Intel Macs, Windows can run natively with just a kick start from Boot Camp, or directly on the Mac desktop using an environment such as Parallels Desktop or Fusion.
All of these solutions actually require a copy of Windows however. Microsoft will happily sell Mac users a copy at full retail prices that are ten times higher than what it charges PC makers to license OEM copies on their new machines. The idea of Wine or the Red Box is to replace Windows itself, so that applications designed for Windows can run without needing to install Microsoft’s software first.
That problem isn’t unsolvable either. A version of Wine is sold by TransGaming Technologies to enable specific Windows programs to run without Windows. This product enables Windows games (or Apple’s iTunes, or Microsoft’s Office) to run on Linux, and was used by EA Games to launch its new titles for Intel Macs this year. Parallels also uses portions of Wine to handle its support for DirectX.
However, creating a general purpose system to run any Windows application without Windows is much more difficult. Smoothly integrating it into the Mac desktop is even more complex. At any point in the future, Microsoft could easily make changes to the Windows API that broke compatibility, or release its own Windows apps–such as Office–using code that refused to work on Windows workalikes.
IBM discovered this in the early 90s, when Microsoft did both of those things to kill off any competitive pressure from OS/2, which was specifically designed to run Windows applications in partnership with Microsoft. If Microsoft was ready to kill its partner’s implementation of the Windows API, imagine what it would do to rivals! More recently, Microsoft invested money into Corel to stop it from contributing to the Wine project, and it specifically checks for Wine when running Windows Update.
Further, Apple has limited resources. Even if it had the technical ability to deliver what Wine hasn’t been able to do in a decade and a half of trying, and even if Apple ignored the reality that Microsoft is in the middle of killing off Win32 to introduce a new Windows API related to .Net, the company simply has better and more profitable things to do than to deliver an alternative solution to existing Mac products.
Imagine the Outcry.
Can you picture the rage that would accompany Apple’s immediate obsolescing of Parallels, Fusion, and other commercial products that have already worked hard to deliver Windows compatibility solutions for Mac OS X?
When Apple introduced Sherlock 3.0 and later Dashboard, pundits ripped their garments apart, scraped themselves with broken pottery, and shook their fists at the merciless god that had abandoned its third party developers. Their crocodile tears created a deluge of Biblical proportions.
However, in the case of both Karelia Watson and Konfabulator, Apple was only expanding its own software into obvious directions. In order to destroy the market for Parallels and Fusion, Apple would have to go out of its way to launch a massive developmental undertaking to solve a problem that is already solved. The only thing more ridiculous than suggesting Apple could deliver Red Box is to suggest it should.
Be Careful What You Wish For.
There’s also a problem with solving a problem too well. By integrating Carbon into Mac OS X, Apple ensured that the classic Mac OS API would linger along for a very long time. Similarly grafting on a Windows workalike environment would make Apple the curator of another legacy API, one that Apple has no good reason to preserve. Even Microsoft would happily bury Win32 if it could only herd its developers to .Net (and find the time rewrite its own applications using its own modern API).
Apple would also inherit the work of trying to patch up the security holes in a piece of technology cheesecloth, while also figuring out how to emulate all the bugs in Windows and all the hacks performed to support bad third party software. Opera devotes its entire existence to doing that work just for Internet Explorer; imagine inheriting that job for the entire Windows operating system.
This year, Apple delayed the release of Leopard from April to October in order to ship the iPhone. The iPhone is making Apple millions of dollars in hardware revenues, millions more in service sharing revenues, and early next spring it will cast the Mother of All Halos over the development world, bringing more attention to Cocoa and Objective-C than ever before in history.
What would Apple get from delaying new efforts like that in order to devote far more work into replicating the Windows API? No direct revenue in hardware or software, and certainly no new attention for its own development tools. If anything, Apple would only be cutting off its third party developers to spite its own platform.
But What About PE?
The layers of absurdity related to the idea of Apple releasing its own implementation of the Windows API were ignored all over again recently when a few people began talking about support for Windows’ PE files within Mac OS X Leopard.
This excitement blinded a variety of speculators long enough to allow them to unearth the rotten corpse of the Red Box Myth and parade it around town as if it had bright prospects as a potential elected official who could lead the world into a bright new future. Credulity knows no bounds.
PE stands for Portable Executable, and relates to the file format for binary executables and DLLs used by Windows applications, including Windows 3.1, 95/98/Me, NT/2000/XP/Vista, and even WinCE. PE is based on the old Unix COFF specification, but most modern versions of Unix and Linux now use ELF; Mac OS X uses the Mach-O format due to its Mach kernel heritage.
Because PE was a stepping stone from DOS (Windows applications all launch a DOS app to start), standard PE files typically include the line “This program cannot be run in DOS mode.”
To PE or Not to PE.
“Windows PE” (this time, for “preinstallation environment”) is also the name of a separate Microsoft product designed to deploy and repair Windows. Microsoft doesn’t want to cede control over what goes into Windows or how it is configured to the general public, so WinPE is only provided and licensed to OEM vendors and other institutional users in a service contract with Microsoft. With full control, users would be able to turn Vista Basic into Vista Ultimate without paying an extra couple hundred bucks, and be able to install it on several PCs they own; that would eat into Microsoft’s profits.
Windows needs some sort of basic operating system to launch in order to manage a new Windows system installation image. Because of the firmware limitations of standard BIOS PCs, the hardware also needs to launch something else in order to bring up a usable environment for repairing a Windows installation. Prior to Windows XP, the role of WinPE on consumer Windows systems was played by DOS.
Using WinPE, a Windows vendor or admin can create a customized version of Windows to be deployed on a variety of machines. WinPE can also be used to perform system recovery on a PC with a damaged Windows install. WinPE is really just a minimal version of Windows, using the same NT loader and kernel, and supplying a few of the same utilities.
Mac OS X is much easier to customize and deploy in volume, and Apple supplies disk imaging tools for free in Disk Utility. Additionally, the Mac OS X installer DVD boots up a basic version of Mac OS X with various utilities, making it a more sophisticated, graphical, non-expensive, and unregulated equivalent to Microsoft’s WinPE.
Why Apple Supports PE in Leopard.
While it has its own version of WinPE, Apple has to support the unrelated PE format for another reason. Prior to the Intel Macs, Apple used a sophisticated firmware developed by Sun called OpenBoot; on the Mac, Apple referred to it by its IEEE name: Open Firmware. This system, based on a basic Forth environment, allowed PowerMacs to boot up into an environment that offered a number of features unavailable on PCs:
- Option Booting scans the system to find bootable devices and allows the user to select one.
- Target Mode boots the system up as a Firewire disk, allowing other systems to mount its drives.
- Net Booting allows the system to find a network share and boot from that remote disk image.
- Other keys signal environment variables that Open Firmware passes to the operating system after launching it.
In moving to Intel Macs, Apple chose to migrate to a new firmware environment Intel had created rather than porting Open Firmware. A major reason for doing this was to make sure Macs would be compatible with future developments, as generic PCs would also someday migrate to the new standard. It was called the Extensible Firmware Interface, or EFI.
EFI and Itanium.
Intel created EFI for use with Itanium, the 64-bit processor that was supposed to replace the old x86 architecture and also replace all of the competition and heterogeneous processor architectures of the world with one true ring to bind them all. Itanium intended to be an entirely new, clean rethinking of all of the existing legacy problems on the PC and other systems, but ended up being delivered very late and much more complex and expensive than planned.
Itanium ended up as a major failure that never moved beyond the server market, which historical revisionists now insist was its only intended target. In reality, Itanium was presented as a competitor to PowerPC back in 1994; by 1997 IDC was estimating Itanium would balloon into a $38 billion annual market by 2001. As it turned out, Itanium wasn’t even released until 2001, and only sold a few thousand units.
Itanium floundered as Intel’s x86 line also lumbered toward failure with the disappointing Pentium 4. However, even as late as March 2003, John Dvorak was insisting in the pages of PC Magazine that “Apple will announce its use of the Itanium chip, which can be used in such a multiprocessor design and will become the first desktop use of the chip.”
Dvorak predicted Apple would shift to Itanium before the end of 2004; instead, Apple introduced new Power Macs based on the G5 in 2003 and surprised everyone. Years later, Apple rapidly phased PowerPC out, rather than using a dual-processor architecture as Dvorak projected, and started with profitable portables rather than desktop systems running Photoshop, which would see the least benefit from a processor transition and involved the users most sensitive to change. Of course, Apple also never used Itanium chips, either.
Microsoft’s EFI Involvement.
Intel had developed Itanium and EFI together with Microsoft, which shipped an “IA-64” version of Windows for it. Microsoft didn’t deliver anything like Apple’s Universal Binaries however, so any software for the IA-64 version of Windows will only run on Itanium systems, and mainstream Windows applications for the x86 PC will not run on Itanium systems.
Microsoft’s other contribution was to get Intel to base EFI on PE rather than open software. One of the goals of EFI was to facilitate a Microsoft Palladium-like system where hardware vendors could limit what users could do with their computer. Apple simply used EFI as a stand in for Open Firmware (which EFI was modeled after), allowing today’s EFI based Intel Macs to operate similarly to earlier Macs.
Besides Macs, there are no other PCs that use EFI, and the standard version of Windows XP and Vista for PCs do not support EFI. This creates a catch-22 that prevents Windows PCs from matching the smarter firmware-related features Mac users have enjoyed for well over a decade.
EFI on Intel Macs.
Support for EFI in Mac OS X 10.4 Tiger required hacking at the foundations of the system to support a new firmware layer based on PE rather than Forth. Those changes were among many Apple had to make to introduce Intel-based Macs in 2006 that appeared to run the exact same software as the earlier PowerPC Macs. In reality, Tiger for Intel was an entirely different operating system than Tiger for PowerPC.
Those changes resulted in Apple holding up the kernel source for the latest Intel Macs for several months after the first Intel Macs shipped. This incensed several wags, including Tom Yager, who dramatically described it as a slippery slope toward Apple’s abandonment of open source.
In 2007, Apple released Mac OS X 10.5 Leopard as its first Universal release; rather than having separate install disks for PowerPC and Intel, the entire Leopard operating system is built from Universal Binaries and installs and runs on either platform.
The recent discussion about PE support in Leopard relates to its native support for EFI. Tiger originally didn’t need to support EFI, because Intel Macs using it were still a year away when it was released. The EFI support in the updated Tiger for Intel doesn’t work identically to Leopard, and of course was never needed at all in the PowerPC version of Tiger.
Because EFI is built using PE, Leopard has to support loading PE files and the DLL architecture of Windows. Because this isn’t a publicly supported interface, Apple doesn’t advertise it. Because of the legacy of PE, there are files in Mac OS X that include the line “This program cannot be run in DOS mode.” However, extrapolating from that an idea that this somehow foreshadows a move by Apple to replicate the vast Windows API–or incorporate Wine into Mac OS X in a bid to cut off the legs of its own partners–is simply an idea that hasn’t been given enough thought by those advancing it.
Thinking Outside the Red Box.
Rather than developing new colored boxes for Mac OS X, Apple is working to deliver a single operating system and a composite modern API that runs on multiple platforms, and scales from the iPod Touch to the iPhone to the Apple TV, laptops, desktops, and servers. Apple’s growth is going to come from the iPhone and new iPod models, along with Mac Books and likely new ultra portable laptops released next year.
It’s not going to come from efforts to pair a clone of the popular operating system of the 90s with the 90s desktop PC form factor that is now in the throes of death. PC hardware makers are facing a mature market that has plateaued. Along with the shift to smarter, more mobile devices will come a demand for an operating system that does more than run yesterday’s software.
There’s a lot to learn from the past, but attempts to repeat past events rarely work out as expected. The history that will be repeating isn’t going to be a simple repetition of Microsoft’s lucky power grab of the tech industry using a slightly revised new API, but rather a modern version of IBM’s blind fall from its position as a monopolist giant of the computing world, this time played out by Microsoft Corp.
Like reading RoughlyDrafted? Share articles with your friends, link from your blog, and subscribe to my podcast! Submit to Reddit or Slashdot, or consider making a small donation supporting this site. Thanks!