X Window System (thing)

The X Window System is the foundation of the graphical environments on Unix and Unix-like operating systems, such as Linux. Since Unix predates the graphical user interface and widespread availability of computer graphics, it has no built-in facility for graphics at the lowest levels of the system. However, as the de facto standard for graphical applications under Unix-like systems, X has been near-ubiquitous on such systems for the last 10-15 years. Unlike most graphics layers, the protocol between applications and the system is network-transparent, allowing programs from multiple machines to appear on a single display without requiring external support.

X History

The X Window System was originally developed at MIT as part of their pioneering computing access program, Project Athena. X originated as an adaptation of Stanford University's W network window system to a much more efficient network protocol, completed by Bob Scheifler in May 1984. Unlike other graphics systems of the time, such as the Macintosh's QuickDraw, X was designed to be both hardware-independent and vendor-independent, since Project Athena intended to connect all systems at MIT regardless of their origin.

The initial implementation of X was quite limited, and over the next year it was extended in a number of backwards-incompatible ways. The result was X Version 9 or X9, which was the first version released under the permissive 'MIT License' in September 1985. This license would have an important effect on the future of the X Window System, allowing anyone to use and modify the code for any purpose whatsoever, provided that the original author's copyright notice is preserved and the recipient understands that the original author provides no warranty of any kind. These permissions have been broadly abused in the intervening time, but also ensured the widespread availability of X throughout the Unix world.

It soon became apparent that, despite the best efforts of X's designers, the X9 and X10 protocols were still quite hardware-dependent. A comprehensive redesign was begun under the joint auspices of MIT and Digital Equipment Corporation, which resulted in the release of X Version 11 in September 1987. Since then, there have been no backwards-incompatible changes in the core X protocol, an impressive record of compatibility.

X11 continued to be developed by a consortium centred at MIT for several years, with many important improvements made to the core X codebase. These changes did not modify the core X11 protocol itself but improved its efficiency, programming interface, and included tools. Ports to a variety of systems, including DEC's non-Unix OS VMS, were made, some of which were contributed back to the MIT X distribution, while some of them were retained as proprietary software by their authors, as the MIT license permits. During the late 1980s and early 1990s X was one theatre of the so-called 'Unix wars' with various vendors adding proprietary extensions to their own versions in an attempt to achieve differentiation from their competitors.

In 1991, X11 Release 5 was released, containing many improvements including a port to the x86-based 'PC' architecture called X386. With the emergence of free, open source Unix variants and lookalikes for the 32-bit x86 platform, this port grew in importance and soon had its own community of maintainers. These maintainers eventually broke with the original authors after they began to make their new versions proprietary, forming the XFree86 project. Over the course of the 1990s, this project would move to the forefront of X development.

The MIT X Consortium shut down at the end of 1996, with its series of successors becoming increasingly ineffectual due to the influence of differentiation-craving proprietary Unix vendors. The leverage of XFree86 as the most active developer of open source X prevented the reference implementation from switching to a new restrictive license for X11R6.4, but the reference implementation continued to rot under neglect.

XFree86 succeeded in modernizing the architecture of X's core with XFree86 4.0, but afterwards XFree86 development began to stagnate. Development was controlled quite tightly by a Core Team with commit access to the repository, and several members of this team were reluctant to give up control. While the burgeoning Linux desktop was making ever heavier demands of X, the XFree86 developers were slow to change in response.

This situation came to a head in early 2004, when the lead developer of XFree86, David Dawes, unilaterally changed the XFree86 license. The new license was widely vilified as being incompatible with the most common free software license, the GNU GPL. As a result a group of developers, led by longtime X developers Keith Packard and Jim Gettys, created a fork from the last MIT-licensed XFree86 release under the auspices of freedesktop.org. A new X consortium, the X.Org Foundation, was created by the old, moribund X.org organization and freedesktop.org, and the XFree86 fork, generally called Xorg, was imported as the new reference X implementation. A resurgence in X development followed, with the resulting changes making a noticeable difference in the modern Linux desktop.

Under the stewardship of the new X.org, the reference X distribution quickly replaced XFree86 in most Linux distributions and BSD flavours. Though their first release, Xorg 6.7.0, was very little different to the poorly-licensed XFree86 4.4, the writing was on the wall and most distributions were running Xorg by early 2005. Rarely has a software fork replaced its progenitor so quickly, but virtually all X developers doing new work were founders of the fork or quickly joined. This included Keith Packard, architect of many of XFree86's innovative features, whose vocal public clashes with the XFree86 core development team were key in generating the will to fork.

The first Xorg release with a full release cycle, 6.8.0 followed in September 2004, which included a number of important additions in different degrees of stability. Most prominent but most experimental was the XComposite extension, which allowed much greater control over the display of windows. While this initially was used to implement true transparent windows, it permits a variety of uses including screen magnification for accessibility and the presence of ordinary X windows within a fully-3D interface such as OpenCroquet.

The much-delayed release of Xorg 7.0 brought with it many important improvements, but none as prominent as the division of the core X distribution into discrete modules with their own release cycles, which succeeded in accelerating the introduction of X improvements into distributions and also in lowering the barrier to entry for new X developers. Since 7.0 there have not been as many sweeping changes but a slow, ceaseless introduction of new features, bugfixes, and cleanups, similar to the Linux kernel or GNOME. X has finally joined the rest of the modern free software ecosystem.

X Architecture

The X distribution primarily consists of a display management program called the X server, a protocol that other programs, called X clients can use to display themselves on the X server's screen, and a library for writing X clients without detailed network programming, Xlib. A complete X install also includes a variety of basic clients and utilities for managing the general X environment. These parts were traditionally bundled together into a single software package, but, as of Xorg version 7, the X distribution is divided into a number of individual packages with their own release schedule.

The original X was amazingly primitive by modern standards. Lacking a facility for managing windows or launching programs, early X sessions consisted of a preselected list of programs started and positioned in the .xinitrc configuration file. Any additional programs run from a terminal emulator also required manual specification of position and size on the command line. The X developers' response to this was typical of the X development process; rather than adding window management and automatic window borders to the X server, they added a special X application, the window manager, whose sole purpose is to handle the placement, movement, and decoration of windows.

The introduction of window management highlighted an issue that has come back to haunt X many times over the years; many operations that are fundamental 'server-side' parts of other display layers require cooperation by X clients to implement. The Inter-Client Communication Conventions Manual (ICCCM) was promulgated in an attempt to standardize this cooperation, but it is a complex standard that is difficult to implement and contains a number of now-archaic restrictions. This confusion was responsible for at least a decade of poor X usability before high-level free software toolkits that abstracted away the costs of the ICCCM became ubiquitous in the early 2000s.

Another facility that X does not provide is a set of standard graphical interface elements. The X protocol is extremely low-level, dealing mainly with maintaining square drawing canvasses and routing input events to applications. An additional library above Xlib called a widget toolkit is required to produce buttons, scrollbars, menus, and text boxes, in another instance of the general X convention 'mechanism, not policy'. X distributions do contain an extremely primitive widget library called the Athena widgets or Xaw, but unfortunately Xaw widgets are more primitive than even the original Macintosh widgets, and have a number of conventions quite unlike any other widget set, especially in its idiosyncratic scroll bars.

As a result, a variety of widget toolkits have appeared over X's long lifetime. Early on, the OpenLook and Motif toolkits were available, though both were reasonably proprietary. Motif gained prominence as the basis for the first integrated desktop environment for X, the Common Desktop Environment or CDE. However, as the 1990s wore on, the heavy Windows 3.1-like Motif controls fell out of fashion, and began to dwindle when a group of students at Berkeley, frustrated with the Motif licensing terms, began developing their more modern, free replacement, GTK+. With the GIMP image editor driving GTK+ development and the Norwegian startup Trolltech releasing the X11 version of their cross-platform Qt GUI toolkit to the community, Motif's downfall became inevitable. It is a testament to the power of X's architecture that the system was not stranded with an antiquated 1980s look-and-feel and programming interface but could move ahead maintaining both backwards and forwards compatibility.

An important part of the core X protocol is a mechanism for adding new capabilities to the X protocol, generally referred to as extensions. The availability of this feature has allowed the interaction between the X server and its clients to be drastically altered without breaking compatibility. A number of important features of the modern toolkits depend on extensions, rather than the core protocol, such as the SHAPE extension for non-rectangular windows and widgets, and the MIT-SHM extension for the transfer of images through shared memory rather than the command stream.

The modularity of the X protocol also allows for many of the client-side libraries to be compatibly changed without modifying the server. Most X applications communicate with the server through Xlib, a library that attempts to smooth some of the complexities of the full X protocol by handling certain things itself. The assumptions that were made during Xlib's development in the mid-1980s have not aged well, and many of the details that it 'takes care of' have become a hindrance to modern toolkits as many of these details have become more relevant over the years. In particular, the single command queue used by Xlib limits the use of multi-threading in applications. Furthermore, the ancient Xlib code is known to have errors that are almost impossible to find in its ageing codebase.

When Xlib was written, many X applications were built directly upon it, necessitating an API that was somewhat 'user-friendly'. In the last decade, very few programs have been written on bare Xlib, using a higher-level toolkit such as GTK+ or Qt instead. As such, the authors of the new XCB (X C Bindings) library have reasoned that a small, simple library for interfacing directly to the X protocol would be useful, as its added difficulties would only be a hindrance to relatively few developers. XCB has now reached full stability, with the Xlib in Xorg 7.2 and later having been re-written on top of XCB to maintain compatibility. The toolkits have not yet been rewritten to use XCB instead of Xlib but most believe that it is only a matter of time.

X Today

Under the stewardship of the X.Org Foundation and freedesktop.org, the standard X distribution has broken out of its development rut and has become one of the more active areas for improvement of the free software desktop. With the KDE and GNOME desktop environments building modern user interfaces on X and their underlying Qt and GTK+ toolkits allowing applications to be built with similar ease as under other major platforms, X has emerged as an important competitor for the graphical interfaces of other major operating systems.

An important part of modern X is its efficiency in the most common case of a server and client running on the same machine. Lacking a network separating the two programs, they can communicate through an efficient inter-process communication method such as Unix domain sockets, and use shared-memory image transfer to speed up the most message-intensive operations. When these methods are used, X is similar in weight to competing display systems, which have themselves adopted a window server/client architecture similar to X.

One of the first tasks undertaken by X.Org was a full modularization of the core X distribution. Following this separation, the X server, client libraries, graphics drivers, and utilities could be updated independently, with semi-yearly comprehensive 'katamari' releases maintaining a common baseline. The separation of drivers from the main X server code base has enabled rapid development of many drivers, especially those for Intel and ATI graphics hardware.

The most important modern addition to X's display model is the Xrender extension. Xrender adds Porter-Duff composition as a basic display operation, and although this sounds somewhat obscure its effects are widespread and impressive. The most common use of Xrender is to display smooth, anti-aliased graphics and text, which the Xrender architecture is designed to accelerate. The advent of anti-aliased text coincided with the wholesale replacement of X's antiquated, user-hostile font system, finally ending most Linux users' font headaches.

The composition capabilities of Xrender can be used to more spectacular effect in combination with the Xcomposite extension. Xcomposite allows a particular client, called the compositing manager, to intercept window contents before they are drawn to the screen, and then to combine the windows in whatever way it sees fit to produce the final screen contents. This opens the door for advanced desktop effects similar to those used by Mac OS X and Windows Vista, with or without GPU acceleration. The Compiz window manager was built from the ground up to include a compositing manager, and is used in a number of modern distributions including Ubuntu and openSUSE to provide translucent windows, drop shadows, and Expose-like window switching, among other things.

Composition managers are not only for fancy 'bling' effects, though. Accessibility applications benefit greatly from the ability to modify the appearance of a window before it is shown, as magnification or increased contrast of windows can help visually impaired users (or even users on poor displays) use the computer's GUI. Composition can also help with the use of virtual desktops, as the desktop can be made to 'slide' across, providing a visual cue to the user. While this is most useful to the novice user, power users may also find benefits from the composition manager. Compiz can display virtual desktops on the faces of a 3D polygonal prism, called the 'desktop cube' for the common case of four desktops, which can be freely rotated to visualize and choose a desktop.

The new Plasma desktop in the KDE 4 desktop uses composition as the basis of its attempt to redefine the basic GUI elements for the better. While it is still beta software and currently implements a relatively conventional, though very pretty, desktop, Plasma aims to be a test-bed for new styles of human-computer interaction. Surely, the promise of pervasive desktop composition has yet to be fully realized by the more creative programmers in the free software world.

Summary

The X Window System has come a long way from its cradle at MIT to become the keystone of the widespread free software desktop. Through all of this it has maintained an astounding amount of compatibility; a client from the late 1980s will still be able to display on today's newest servers, even without a recompile (though they would be unlikely to work on the same machine). While the primitive nature of its drawing model limited it to the most powerful computers when it was first released, X11's overhead is now comparable to or even less than competing display layers from Apple and Microsoft. It remains under active development and will continue to be optimized and extended for many years to come.