(See bottom for most recent update.)
(Please note this wu is under revision.)
Security Enhanced Linux (or 'SE Linux') was released by the NSA in January 2001 under the GPL. It consists of a patched kernel plus utilities, and was written with the goal of incorporating a flexible mandatory access control architecture in to the Linux 2.4.x kernel.
Background
Most modern operating systems use a discretionary access control (DAC) security model. This means that they restrict access to objects based on their classification. This type of control is discretionary in the sense that a subject with a certain set of access permissions is capable of passing those permissions on to another subject. For example, any program you run while logged on as a certain user has the same access rights as you do. Rights are set by another user (for example, root or administrator).
Any particular permission (read, write, execute, etc.) can be thought of as a two-dimensional matrix with users on one axis and objects on another. In essence, DAC systems check the validity of credentials presented to them against stored information.
Another security model is mandatory access control, or MAC. This controls access in an different manner. Whereas DAC security models are authentication-based, MAC systems rely on authorization, not only of the user but also of each object loaded by the system.
A MAC system controls objects individually, and make decisions on the rights and/or permissions of objects based on a security policy, which can define what rights the object should be accorded based on different variables.
An example of how discretionary versus mandatory access control styles could affect the operation of a computer would be a cgi script. If the script allows an external entity to insert and execute malicious code on a computer system under a DAC system, the malicious code now has the same access rights as the code that executed it - the cgi script. This is one of the reasons why buffer overflows work.
A MAC system can restrict the rights of a certain process to only the resources needed for normal operation. A cgi script may create a process (or it may be forbidden), but that process might not have the same set of permissions as the process that created it.
Security Policies
The MAC system which is used by SE Linux is called Flask. Flask is based on the idea of nested sandboxes originally grew out of a collaboration between the Information Assurance Research Office of the NSA and the Secure Computing Corporation (SCC) to develop a strong, flexible MAC system based on Type Enforcement (explained below). Two prototypes, DTMach and DTOS, were developed, using the Mach microkernel as a starting point.
The system was then ported over to a research operating system called Fluke, which is/was maintained by the Flux research group of the University of Utah. During the porting, several advanced features were added, and the final result is the Flask MAC architecture. This is what has been ported to Linux.
One of the main features of Flask is the separation of policy definition and policy enforcement. All security policy related logic is done by the 'security server' - this was external to the Fluke kernel, but is implemented in Linux as a kernel sub-system.
The security policy configuration is defined in a text file which is compiled by a separate program, normally at boot time, and loaded in to memory. Only the security server can make policy decisions on the permissions of an object.
This changes the manner in which system administration is done. Instead of having the distinction between root and non-root, there is instead the user responsible for setting the security policy.
The security policy enforcement is done by components called object managers, which receives requests from client objects, submits queries to the security server, and enforces the resulting decisions. Flask specifies the interface between the security server and other subsystems.
The SE Linux implementation of the security server uses a combination of type enforcement (TE) and role-based access control (RBAC). It can also use Multi-Level Security (MLS) which is mainly used in military environments for dealing with different data classifications (i.e. Secret, Top Secret, Compartmentalized, etc.) and less so elsewhere. It is not discussed here.
Type enforcement
Type enforcement makes security decisions based on what kind of object (in the case of Flask, the object managers' client) is requesting the permissions. For example, object types could include a regular file, a directory, a process, a socket, or further sub-divisions of these rather broad categorizations. Type enforcement is an object labelling system, that, combined with access mapping (from the domain of the object requesting permission, and to the type of the object requested), returns a decision which defines the permissible actions of the object.
An example of how type enforcement works would be if you gained root access through a buffer overflow in a network server daemon in it's own domain, where files accessible to that domain have one of two possible types: normal files with which you have only read and execute permissions, and incoming files on which you have only write. As you have root access, all discretionary controls are bypassed, but the fact that you are running in a domain with no write access to normal files prevents you from writing anything to, for example, /etc/passwd.
This is a simplified example, as in a MAC system which used type enforcement you would have many more domains and types. The type enforcement used in Flask is a generalised version of domain and type enforcement (DTE), which has been implemented in the Linux Kernel.
Role Based Access Control
Role Based Access Control (RBAC) assigns permissions to objects in a computer system based on the role they play within that system. In practice this means that a process would have it's permissions based on it's parent process, the user logged on at the time, and any number of other variables.
A real-world example of RBAC would be a hospital system, where the role of nurse would include preparations for surgery, and applying medication. But surgery itself is exclusive to the role of the surgeon. The role of a researcher collecting tissue samples would have a more limited set of rights.
RBAC in Flask can depend on any number of variables, as the role an object plays in a given system can be subjected to any amount of analysis.
Policy Enforcement
Flask can combine type enforcement and RBAC when making decisions. It can also make use of MLS. To start with, each object is assigned a security identifier (SID) by the object manager charged with enforcing the decisions of the security server. SIDs in SE Linux are 32-bit numbers, that correlate to a 'security context', which is a data structure private to the security server. A security context represents the context in which the object was created, and is used by the security server to make decisions about that object's permissions.
A normal context consists of user ID, role (as discussed above), type, and maybe an MLS range. Roles are normally relevant only to processes, while sockets and files have a generic role unless otherwise defined. The security server only assigns SIDs to legal combinations of user, role, type etc. according to the security policy. As in the example with the cgi script above, you could configure the security policy to deny a SID to a process created by a given object, in this case the piece of cgi may have been compromised.
There are three main types of objects which can be assigned SIDs from the security server in SE Linux - processes, file systems, and sockets.
Processes
Under SE Linux each process has a SID, which governs the access permissions of the process. The SID of a process can be changed, which allows processes the ability to transition across security domains, although the capability must be explicitly allowed.
A new process is usually assigned the type of it's parent. The role of a process is normally a combination of the process that spawned it, the current user ID, and it's intended type. Also controlled is the ability of processes to configure network interfaces and routing tables.
File systems
The file system object manager controls ten object types, of which the four main ones are mountable file systems, directories, files, and file description objects. At present SE Linux only supports the ext2 file system, although more are planned.
SIDs are different in a file system context in that file objects are assigned persistent SIDs (PSID)s - unlike normal SIDs, they don't change over boots. Each object's PSID is stored in an unused field of the inode.
For file creation, the PSID is created from the process, parent directory, and the kind of file. The security server may also use other external information to make decisions.
Sockets
Flask controls the use of sockets at the socket, transport and network layer. This means that it is possible to forbid a user (or other object) access to a raw IP socket (which allows the possibility of forging packets), while allowing access to UDP or TCP sockets. Setting port-specific permissions is also possible.
The process-socket interaction and socket communication privileges can also be controlled. In addition, it is possible to send and receive messages through network interfaces to other nodes in the network
On most tasks the overhead SE Linux imposes due to security checks and policy decisions is on the order of 1 to 2 percent thanks to various caching mechanisms. However, in networking, the overhead is often as high as 10 percent. This has an obvious impact on the potential uses of SE Linux in network-oriented deployments.
How processes, file system objects and sockets communicate with each other is defined by the security policy. This is written in the form of a large human-readable text file which has it's own (fairly complex) syntax. In particular, the security policy governs how different types and roles may interact, along with any specific rules.
SE Linux has a kernel compile-time configuration option that switches the resulting kernel in to 'audit' mode: instead of enforcing security decisions, it simply logs all decisions taken, and any violations of the security policy that occur. This is used when setting up the system initially, to gauge which objects need which permissions, and to help in setting up a comprehensive security policy. In active mode, the decisions of the security server are enforced.
At present SE Linux provides binary compatibility with existing applications, and source compatibility with kernel modules. The patch also adds some security-related system calls. SE Linux currently supports most filesystems as it goes through the VFS layer. Only ext2, ext3 and ReiserFS have been tested. Others may be mounted, but permissions cannot be set at a resolution lower than the file-system classification itself. The current implementation of SE Linux is x86-specific. (I think)
Why it exists
SE Linux was released by the 'Information Assurance Research' office of the NSA. This department belongs to the non-scary half of the NSA that is responsible for securing computer-related national infrastructure, protecting US companies (Such as Boeing, Lockheed Martin, other defence contractors, hi-tech companies, etc. ) from industrial espionage, and the like.
Public research in this area stems partly from an increased interest in low-cost and secure systems using MAC architectures throughout the Department of Defence. Additional factors spurring research include the ever-increasing expense of the air gap solution to compartmentalisation of classified data, and some unease about increased reliance on operating systems such as Trusted Solaris, AIX, or IRIX.
Sources:
http://www.nsa.gov/selinux/index.html
http://www.cs.utah.edu/flux/
http://www.radium.ncsc.mil/tpep/library/rainbow/NCSC-TG-003.html
http://csrc.nist.gov/publications/nistbul/csl95-12.txt
http://csrc.nist.gov/rbac/
http://research-cistw.saic.com/cace/index.html
http://www.linuxplanet.com/linuxplanet/tutorials/1527/2/
http://www.newsforge.com/article.pl?sid=01/03/23/1534247
http://www-106.ibm.com/developerworks/library/s-selinux/?n-s-381
http://www.usenix.org/publications/library/proceedings/als2000/full_papers/hallyn/hallyn_html/
September 16 2001: The newest version (released a couple of weeks ago) is built against the 2.4.9 Kernel, and interfaces with the rest of the Kernel through the Linux Security Module (LSM) which is (briefly) a collection of hooks for people who want to write security mod's for Linux.