NAT compatible IPSec with UDP

UDP encapsulation allows IPSec traffic to pass through a conventional NAT device. Review this topic for more information about what it is and why you should use it for your VPN connections.

The problem: Conventional NAT breaks VPN

Network address translation (NAT) allows you to hide your unregistered private IP addresses behind a set of registered IP addresses. This helps to protect your internal network from outside networks. NAT also helps to alleviate the IP address depletion problem, since many private addresses can be represented by a small set of registered addresses.

Unfortunately, conventional NAT does not work on IPSec packets because when the packet goes through a NAT device, the source address in the packet changes, thereby invalidating the packet. When this happens, the receiving end of the VPN connection discards the packet and the VPN connection negotiations fail.

The solution: UDP encapsulation

In a nutshell, UDP encapsulation wraps an IPSec packet inside a new, but duplicate, IP/UDP header. The address in the new IP header gets translated when it goes through the NAT device. Then, when the packet reaches its destination, the receiving end strips off the additional header, leaving the original IPSec packet, which will now pass all other validations.

You can only apply UDP encapsulation to VPNs that will use IPSec ESP in either tunnel mode or transport mode. In addition, at V5R2, the system can only act as a client for UDP encapsulation. That is, it can only initiate UDP encapsulated traffic.

The graphics below illustrate the format of a UDP encapsulated ESP packet in tunnel mode:

Original IPv4 datagram:
After applying IPSec ESP in tunnel mode:
After applying UDP Encapsulation:

The graphics below illustrate the format of a UDP encapsulated ESP packet in transport mode:

Original IPv4 datagram:
After applying IPSec ESP in transport mode:
After applying UDP Encapsulation:

After the packet is encapsulated, the system sends the packet to its VPN partner over UDP port 4500. Typically, VPN partners perform IKE negotiations over UDP port 500. However, when IKE detects NAT during key negotiation, subsequent IKE packets are sent over source port 4500, destination port 4500. This also means that port 4500 must be unrestricted in any applicable filter rules. The receiving end of the connection can determine whether the packet is an IKE packet or a UDP encapsulated packet because the first 4 bytes of the UDP payload are set to zero on an IKE packet. For it to work properly, both ends of the connection must support UDP encapsulation.