Tunnel mode

Tunnel mode provides the protection of an entire IP packet by treating it as an AH or ESP payload. With tunnel mode, an entire IP packet is encapsulated with an AH or ESP header and an additional IP header. The IP addresses of the outer IP header are the tunnel endpoints, and the IP addresses of the encapsulated IP header are the ultimate source and destination addresses.

AH tunnel mode

As shown in the following illustration, AH tunnel mode encapsulates an IP packet with an AH and IP header and signs the entire packet for integrity and authentication.

ESP tunnel mode

As shown in the following illustration, ESP tunnel mode encapsulates an IP packet with both an ESP and IP header and an ESP authentication trailer.

The signed portion of the packet indicates where the packet has been signed for integrity and authentication. The encrypted portion of the packet indicates what information is protected with confidentiality.

Because a new header for tunneling is added to the packet, everything that comes after the ESP header is signed (except for the ESP authentication trailer) because it is now encapsulated in the tunneled packet. The original header is placed after the ESP header. The entire packet is appended with an ESP trailer before encryption occurs. Everything that follows the ESP header, except for the ESP authentication trailer, is encrypted. This includes the original header which is now considered to be part of the data portion of the packet.

The entire ESP payload is then encapsulated within the new tunnel header, which is not encrypted. The information in the new tunnel header is used only to route the packet from origin to destination.

If the packet is being sent across a public network, it is routed to the IP address of the tunnel server for the receiving intranet. In most cases, the packet is destined for an intranet computer. The tunnel server decrypts the packet, discards the ESP header, and uses the original IP header to route the packet to the intranet computer.

ESP and AH can be combined when tunneling, providing both confidentiality for the tunneled IP packet and integrity and authentication for the entire packet.

Using IPSec tunnels

IPSec tunnels provide security for IP traffic only. The tunnel is configured to protect traffic between either two IP addresses or two IP subnets. If the tunnel is used between two computers instead of two routers (also known as gateways), the IP address outside the AH or ESP payload is the same as the IP address inside the AH or ESP payload. In Windows XP, IPSec does not support protocol-specific or port-specific tunnels. Configuration is performed using the IP Security Policies and Group Policy snap-ins by configuring and enabling two rules:

1.

A rule for the outbound traffic for the tunnel.

The rule for the outbound traffic is configured with a filter list that describes the traffic to be sent through the tunnel and a tunnel endpoint of an IP address configured on the IPSec tunnel peer (the computer or router on the other side of the tunnel).

2.

A rule for the inbound traffic for the tunnel.

The rule for the inbound traffic is configured with a filter list that describes the traffic to be received through the tunnel and a tunnel endpoint of a local IP address (the computer or router on the local side of the tunnel).

Additionally, filter actions, authentication methods, and other settings need to be specified for each rule.

For conceptual information about IPSec policy tunnel settings, see Tunnel endpoint For information about configuring an IPSec tunnel, see To specify an IPSec tunnel For information about how tunneling is used for virtual private networking, see Virtual private networking with IPSec