An Operating System is interposed between applications and the physical hardware. Therefore, its structure has a dramatic impact on the performance and the scope of applications that can be built on it. Since its inception, the field of operating systems has been attempting to identify an appropriate structure: previous attempts include the familiar monolithic and Micro Kernel operating systems as well as more exotic language-based and Virtual Machine operating systems. Exo-kernels dramatically depart from this previous work. An exo-kernel eliminates the notion that an operating system should provide abstractions on which applications are built. Instead, it concentrates solely on securely multiplexing the raw hardware: from basic hardware primitives, application-level libraries and servers can directly implement traditional operating system abstractions, specialized for appropriateness and speed.
See: http://pdos.lcs.mit.edu/exo/exo-slides/sld001.htm to navigate through all 45 slides on the subject where the term originated.
http://pdos.lcs.mit.edu/exo/
(Source: WikiWikiWeb.)