IPsec VPN: Understanding Benefits, Standards, Modes, and Architecture

This page explains the fundamentals of IPsec VPN, including its benefits, standards, operational modes (transport and tunnel), and overall architecture.

IPsec, short for IP Security protocol, is a suite of protocols used to provide security at Layer 3 of the OSI model, the network layer.

VPN connection

VPN stands for Virtual Private Network. It extends a private connection between a client and a server over a public network like the internet, while ensuring a secure connection. This secure connection is established using tunneling protocols.

VPNs create secure tunnels over public networks between clients and firewalls, routers, and even between firewalls themselves. Different tunneling protocols exist at Layer 2 and Layer 3. IPsec operates at Layer 3. For more details, refer to the working principles of VPNs and Layer 2 VPN protocols like L2TP and PPTP.

Normal IP packets inherently lack security features. There’s no way to verify:

1. The claimed sender is the actual sender.
2. The data has not been modified during transit.
3. The data has not been viewed by a third party.

IPSec Encryption

Benefits of IPsec

IPsec addresses the aforementioned problems with the following benefits:

• Authentication: Ensures the sender’s identity using digital signatures and certificates.
• Integrity: Verifies that data remains unaltered during transit. Routers at each end of the tunnel calculate checksums or hash values of the transmitted data to detect modifications.
• Confidentiality: Protects data from unauthorized access through encryption, utilizing key management and other protocols within the IPsec architecture.

IPsec Standards

The following table outlines key IPsec standards, their RFC numbers, and brief descriptions:

IPsec Modes: Transport Mode vs. Tunnel Mode

IPSec Transport mode vs IPSec Tunnel mode

IPsec operates in two primary modes:

• Tunnel Mode: The entire IP packet is first encrypted. This encrypted packet becomes the data component of a new, larger IP packet. This mode is commonly used in IPsec VPNs for site-to-site topologies.
• Transport Mode: An IPsec header is inserted into the original IP packet without creating a new packet. This mode is suitable for networks where packet size increases are a concern and is often used in remote access VPN topologies.

IPsec Architecture

IPSec Architecture

IPsec provides security services at the IP layer, protecting both the IP layer and upper layers from potential hacking. Other protocols like SSL and TLS provide security for the transport layer, while HTTPS provides security at the application layer.

The IPsec framework allows system administrators to select various cryptographic algorithms and protocols to meet specific user requirements.

IPsec handles the following security services:

• Access control
• Connectionless integrity
• Confidentiality
• Data origin authentication

IPsec

With IPsec, traffic can be securely carried between two gateways using a single encrypted tunnel. It’s also possible to create a separate tunnel for each TCP connection for hosts communicating through gateways.

IPSec can be divided into four categories:

• AH and ESP security protocols
• IKE/IKEV2 key management protocols
• Algorithms such as DES, HMAC with MD5
• Security associations and databases (e.g., SA, SAD, SPD)

ESP (Encapsulating Security Payload) and AH (Authentication Header) are security protocols within IPsec used to provide authentication and confidentiality services.

Key management algorithms require keys to be communicated to involved parties. Manually managing these keys is difficult. Therefore, the Internet Key Exchange Protocol (IKE) automates and communicates them. IKEv2 dynamically performs the same function for involved parties.

SA refers to the set of rules for two IPsec systems to communicate after agreeing on algorithms, keys, protocols, and other parameters. IKE establishes SAs. Each association has entries in the database. SAs are developed for each direction, requiring two SAs to enable security in both directions.

The SA database contains the following parameters:

• Sequence number counter
• Sequence number overflow flag
• Antireplay window
• Lifetime of SA
• IPsec protocol mode (tunnel or transport)