Road to Mac OS X 10.6 Snow Leopard: 64-Bits
August 26th, 2008
Prince McLean, AppleInsider
Next year’s 10.6 reference release of Mac OS X promises to deliver technology updates throughout the system without focusing on the customer-facing marketing features that typically sell a new operating system. Here’s a look at what those behind-the-scenes enhancements will mean to you, starting with new 64-bit support.
The march toward 64-bit
Through the 1980s, personal computers rapidly moved from 8-bit to 16-bit to 32-bit architectures, with each advance enabling the operating system and its applications to address more memory and more efficiently handle the memory available to them. The 8-bit computers of the early 80s could only directly address 64K, the upper limit of their 16-bit memory addressing; early Apple II systems switched between two banks providing 128K. DOS 8086 PCs with 20-bit addressing could handle a whopping 1MB of RAM, but overhead effectively limited them to using 640K of it. These early machines also highlight the fact that a CPU’s architecture, memory address bus, and its data registers (used to load and store instructions) may all have different bit widths.
Similarly, the 1984 Macintosh jumped to using a 32-bit 68000 processor with 24-bit addressing, allowing the theoretical use of “only” 16MB, although at the time that was far more RAM than anyone could afford. That seemingly high limit eventually became a problem for memory hungry applications, particularly with the increased demands required by graphical computing and multitasking.
By the end of the 80s, Apple had delivered full 32-bit hardware with the Mac II’s 68020 processor and the “32-bit clean” Mac System 7 software, which together enabled applications and the system to theoretically use as much as 4GB of directly addressable memory. By 1995, Microsoft was shipping its own 32-bit Windows API with WinNT and Win95 to take advantage of Intel’s 32-bit 80386 and 486 CPUs.
A decade later, the 4GB limit of 32-bit memory addressing would begin to pinch even home computers. To accommodate that inevitability, Apple began its migration to PowerPC in 1994 to make progress toward 64-bit computing and break from the limitations of the Motorola 680×0 processors it had been using. PowerPC offered a scaled down version of IBM’s modern 64-bit POWER architecture, with 32 individual 32-bit general purpose registers; Intel’s 32-bit x86 was a scaled up version of a 16-bit processor, and only offered 8, 32-bit GPRs. The lack of registers on x86 served as a significant constraint on potential performance and complicated development.
In order to attack the RAM limitation problem in advance of moving to 64-bit CPUs, Intel added support for “Physical Address Extension” or PAE to its 32-bit x86 chips, which provided a form of 36-bit memory addressing, raising the RAM limit from 4GB to 64GB. Using PAE, each application can still only address 4GB, but an operating system can map each app’s limited allocation to the physical RAM installed in the computer.
Being able to use more than 4GB of RAM on a 32-bit PC requires support for PAE in the OS kernel. Microsoft has only supported this extra RAM in its Enterprise, Datacenter, and 64-bit versions of Windows; the standard 32-bit versions of Windows XP, Vista, and Windows Server are all still constrained to using 4GB of physical RAM, and they can’t provide full access to more than about 3.5GB of it, making the limit an increasingly serious problem for desktop Windows PC users.
In the late 90s, Windows NT was ported to 64-bit architectures such as Digital’s Alpha, MIPS, PowerPC, and Intel’s ill-fated Itanium, but this also only benefitted high-end workstation users. Apple’s own mid-90s PowerPC transition prepared the Mac platform for an easier transition to 64-bit computing, but it wasn’t until 2003 that the PowerMac G5 introduced real 64-bit hardware. The G5 processor delivered 32 individual 64-bit GPRs and a 42-bit MMU (memory management unit) for directly addressing 4TB of RAM, although the PowerMac G5 hardware was limited to 8GB.
The mainstream PC remained stuck at 32-bit conventions until AMD released its 2003 Opteron CPU using an “AMD64” architecture that turned out to be a more practical alternative to upgrading into the world of 64-bits than Intel’s entirely new Itanium IA-64 design. The new 64-bit PC, also called x86-64 and x64, largely caught up to PowerPC by suppling 16, 64-bit GPRs, and potentially a 64-bit memory bus to address 16EB (16 million TB) of RAM. AMD’s x64 processors can theoretically address 48-bits, or 256TB, in hardware. In practice, no PC operating system currently supports more than 44-bits, or 16TB of virtual memory, and of course considerably less physical RAM.
There’s currently no immediate need for such vast amounts of RAM among home users, but consumers are running into the 4GB barrier of 32-bit PCs, while facing additional problems that prevent mass migration to x64. The main problem is that the potential of the hardware has to be exposed by operating system software. There are two problems: the first is simply addressing more than 4GB of total RAM for the entire system, and the second is allowing RAM-hungry applications to individually access large amounts of RAM.
Even with the 64-bit Power Mac G5 hardware, there were still software limitations in 2003’s Mac OS X Panther; the 32-bit OS allowed the system to support more than 4GB of memory but still corralled each application into its own 32-bit, 4GB space. With 2005’s Mac OS X Tiger, Apple enabled desktop apps to spin off processes and servers that could handle enormous memory addressing of their own: up to a theoretical 16 EB of 64-bit virtual memory and a conceptual 42-bits or 4TB of physical RAM, although shipping Macs still could only support 8GB of RAM.
To enable this, Tiger supplied a 64-bit version of libsystem, the system library handling most of its Unix APIs. This followed the LP64 model to allow broad compatibility with 64-bit versions of Linux and commercial Unix. It also delivered a 64-bit PowerPC ABI (application binary interface) for accommodating native 64-bit apps on the G5. Tiger still used a 32-bit kernel (although it was not limited to 32-bit memory addressing, so it could actually make use of the 8GB of RAM installed in G5s), and it was also still missing a 64-bit version of the Cocoa or Carbon APIs, which meant apps with a user interface had to be 32-bit.
However, a 32-bit graphical app on Tiger could spin off a faceless 64-bit background process to perform number crunching on a vast data set requiring a 64-bit memory space, which could then communicate the results back to the 32-bit foreground app running in parallel. Apple also delivered a mechanism for deploying applications using a bundle of both 64-bit and 32-bit code, allowing the system to automatically run the appropriate version for the Mac hardware in use. Tiger itself also supplied both 32- and 64-bit underpinnings, allowing one OS to run on any Mac. This has made it easier for Apple to rapidly migrate Mac users toward 64-bit hardware.
In contrast, a separate 64-bit version of Windows is required to run 64-bit Windows apps on 64-bit x86 PCs, and any 32-bit apps have to run in a special compatibility environment (below). There is no slick mechanism for deploying bundles of mixed code that “just work” on both architectures, and 64-bit Windows itself lacks the ability to run on either type of PC. This has had a chilling effect on the popularity of and the momentum behind 64-bit Windows that parallels the problems with Vista.
This is particularly unfortunate because the advances delivered in the x64 PC are more desperately needed by PC users to gain the same benefits that Mac users and developers gained from the move to PowerPC over a decade earlier. The 32-bit PC is particularly hampered by a lack of GPRs and the 4GB RAM limit imposed by the desktop versions of 32-bit Windows. In addition, 32-bit Windows itself eats into that 4GB to only leave 3.5GB of RAM or less for apps and the system to use, and typically limits individual apps to a tiny 2GB address space.
Software compatibility, a lack of drivers, and other problems have also complicated the move to 64-bit Windows, leaving mainstream Windows users stuck at 32-bits. Windows 7 was initially supposed to move users to 64-bits in perhaps 2010, but reports indicate that it too will be delivered in separate 32- and 64-bit versions.
When Apple began migrating to Intel in 2006 it actually had to take a step backward, as it only initially supported 32-bit Intel systems with the Core Solo and Core Duo CPUs. Apple had to cope with the same 32-bit PC limitations Microsoft had been dealing with. in the Intel transition, Mac developers lost the features supplied by PowerPC, including its liberal supply of registers. However, Intel’s new 32-bit Core Duo was fast enough in other areas to skirt around the problem, particularly in laptops where the aging G4 was holding Macs back.
By the end of the year Apple had widened support to include the 64-bit x64 PC architecture in the new Mac Pro and Xserve, and subsequent desktop Macs using the Core 2 Duo also delivered 64-bit hardware support. With updates to Tiger, Apple delivered the same level of 64-bit support for x64 Intel processors as it had for the PowerPC G5.
Within the course of one year, Apple had not only adroitly moved its entire Mac product line to Intel but also paved the way forward to rapidly push its users to 64-bits, narrowly escaping the disaster of being left the last member of the desktop PowerPC party. In its spare time, the company also threw the iPhone together while also working to develop its next jump in 64-bit operating system software.
The 64-bit GUI in Leopard
In Leopard, Apple expanded 64-bit support further, adding 64-bit support in the higher levels of Carbon and Cocoa. Apple delivered its own Xcode app in Leopard with support for both PowerPC and Intel in both 32-bit and 64-bit versions, all within the same application bundle. The entire OS is now a Universal Binary as well; it automatically runs on whatever hardware it is installed on. Incidentally, one of the biggest issues in getting Mac OS X to run on generic PC hardware is the need to turn off PAE in the kernel for older CPUs that don’t support it.
While all of Cocoa is now 64-bit, Apple chose not to deliver full 64-bit support in Carbon’s user interface APIs (including legacy parts of QuickTime), forcing developers to migrate their apps to use the modern equivalents in Cocoa in order to deliver full 64-bit applications with a user interface. Carbon can still be used to build faceless 64-bit background apps that interact with a 64-bit Cocoa front end, similar to how Tiger supported 64-bit background apps. Earlier, Apple had added transitional support for mixing Cocoa into Carbon apps to make this move easier.
Apple’s decision to withhold the development of 64-bit Carbon caused Adobe to announce this spring that its upcoming Creative Suite 4 would only be delivered as a 64-bit app on Windows. Because CS4’s legacy code is based on Carbon, Adobe said it wouldn’t be able to deliver a 64-bit version of its Mac apps until at least CS5, because it will require porting the interface code of Photoshop and its companion apps to Cocoa in the model of Photoshop Lightroom. Most desktop apps don’t necessarily demand 64-bit support, but Photoshop’s use of extremely large image files makes it a good candidate for porting.
Currently, Mac OS X Leopard hosts both 32-bit and 64-bit apps on top of a 32-bit kernel (below). Using PAE, the 32-bit kernel can address 32GB of RAM in the Mac Pro and Xserve; Apple’s consumer machines only support 4GB RAM, but unlike 32-bit operating systems they can use the entire 4GB (with appropriate hardware support). Leopard’s 32-bit kernel enabled Apple to ship 64-bit development tools to give coders the ability to build applications that can work with huge data sets in a 64-bit virtual memory space (and port over existing 64-bit code), without also requiring an immediate upgrade to all of Mac OS X’s drivers and other kernel-level extensions. That transition will happen with Snow Leopard.
How big of a deal is the move to 64-bit apps? As Apple’s developer documentation points out, “To put the difference between 32-bit and 64-bit computing into perspective, imagine that you are working with a dataset in which the road area of the Golden Gate bridge can be represented in a 32-bit address space. With 64 bits of address space, you have the ability to model the entire surface of the Earth at the same resolution.”
Apple is expanding its 64-bit support in Snow Leopard down into the kernel. This will enable Mac systems to accommodate more than the 32GB of RAM currently available via 32-bit PAE. With kernel support for full 64-bit memory addressing, Apple can add as much RAM as users can afford. Of course, if you’re buying RAM from Apple, upgrading a Mac Pro to 32GB of RAM currently costs $9,100, so it might be some time before home users decide they need more than that much RAM.
While Leopard’s 32-bit kernel can run both 32- and 64-bit apps, a 64-bit app can not load 32-bit plugins or shared libraries, and vice versa. The 64-bit kernel similarly requires 64-bit kernel extensions and drivers, as it can’t mix 32- and 64-bit code either. The move to a 64-bit kernel will therefore require an across-the-board upgrade for all kernel drivers in Snow Leopard.
Snow Leopard will also require developers who write any plugins for Mac OS X apps to recompile their code to 64-bit. This includes everything from System Preferences panes to web plugins. The reason for the massive upgrade will be that Apple will also deliver the entire system compiled as both 32- and 64-bit, from the Finder to iTunes to Safari. On 32-bit Macs, Snow Leopard will run normally, but on x64 Macs, everything will get a significant boost as every app on the system will benefit from the advantages of x64, particularly the extra registers supplied by x64 and missing from the 32-bit PC.
That advantage will outweigh the additional overhead caused by moving to 64-bits and the resulting use of larger data items. In contrast, there would be no real advantage in recompiling Snow Leopard and its apps for 64-bit PowerPC G5s, as the G5 is not currently constrained by the register problem of 32-bit x86; the 64-bit G5 has the same number of registers as the G4, because the G4 already had plenty. The G5 actually runs 64-bit apps slightly slower because of the increased overhead imposed by 64-bit addressing. For that reason, Snow Leopard will apparently be Intel-only.
More information on Snow Leopard appears on AppleInsider’s Mac OS X 10.6 page.