When designing Mac OS X, Apple determined that X Window couldn't provide the level of interaction necessary to support a Mac-like interface, and that licensing issues with Adobe required an alternative to Display PostScript. Further, Apple needed to bring Carbon applications to Mac OS X, and that required supporting the classic Mac QuickDraw graphics engine.
Apple removed all drawing, including the PostScript language, from the core window engine, and introduced a "lightweight window server," called the Quartz Compositor. It routes mouse-click type events to the appropriate program, and composites layers of windows to send to the display hardware.
The Quartz Compositor is particularly amazing because it layers windows with active transparency, providing subtle effects like drop shadows and translucent windows. Even better, much of the heavy lifting done by the Quartz Compositor can be implemented in OpenGL to provide graphics hardware acceleration.
Drawing in Mac OS X is done outside of the Quartz Compositor, either in legacy QuickDraw, in the new Quartz 2D (which follows the graphics model of PDF), in OpenGL (for 3D models), or in QuickTime (for 4D video). The way the Quartz Compositor is designed, any drawing engine could be added to hand it graphics to display.
Apple also left hooks in place to enable applications to perform remote display. While this is certainly an interesting option, Apple development in the last five years has centered on improving Quartz Compositor and Quartz 2D for local performance, accelerating both using the otherwise idle graphic CPU.
If Apple has any plans to exploit the latent possibility of remote display functionality in Mac OS X's Core Graphics, they are keeping it a tight secret.
My Mac OS X 10.5 Leopard Wishlist article on remote display and management only asks Apple to deliver a modest remote display feature.