Encapsulating Security
Payload (ESP) is a
header format of
IPsec. In particular, it is designed to provide
confidentiality and limited
traffic-flow confidentiality, as well as similar services to the
authentication header. This is accomplished primarily through
encryption.
ESP provides encrypted encapsulation. The TCP packet (or any other protocol running on IP) is encrypted and included as part of ESP. Alternatively, the entire IP packet can encapsulated in another IP packet. Encrypting only the subprotocol of IP is called transport mode, while encrypting the entire packet is called tunnel mode.
Tunnel mode is much more common in my experience, as generally you want to set up a virtual private network (VPN) that allows you to communicate with multiple hosts. In transport mode, you would have to negotiate keys with each host. In tunnel mode, you only have to nogotiate keys with a single server, making deployment and management much simpler.
ESP is designed for use with symmetric encryption algorithms. Moreover, because packet order is not guaranteed, each packet must be self-contained. Consistent with the rest of IPsec, ESP does not specify the encryption or authentication to use, leaving that to the security association part of ISAKMP. However, the specification does list required algorithms to support in a conforming implementation.
Note that ESP contains within it support for authentication data, providing similar functionality that AH in addition to encryption. However, ESP does not protect any header fields unless they are part of the encapsulation. Thus, the outside IP header is not authenticated by ESP.
ESP is defined in RFC 2406.