Blog Post

Our goal as a Network Security Engineer is to be able to provide secure channels of communication over insecure networks. E.g. Internet. IPSEC is a framework or we can say a protocol suite that provides a secure tunnel between 2 nodes along with Confidentiality, Integrity, and Authentication. For E.g. There are 2 offices between which we want users to communicate with each other over the Internet. Also, the requirement is that the traffic should be encrypted while going over the Internet. That’s where IPSEC helps us to establish a Site-to-Site VPN providing Confidentiality, Integrity, and Authentication with the help of Cryptographic algorithms.  

IPSEC Protocols and Packets

We need to understand that there are 2 types of traffic that we need to be aware of while configuring IPSEC.

1. Control Plane – Traffic generated and destined by/for VPN Device to set up a secure tunnel.
2. Data Plane- Traffic generated by User Endpoint in plain text, can be destined for another remote user/server.

Before setting up the control plane, you need to decide the IKE version to be used. There are 2 versions of IKE available. IKE Version 1 and IKE Version 2.

IKEv2 is better than IKEv1. Below mentioned are the basic differences between IKEv1 and IKEv2.

We will understand the packets exchanged by IKEv1 in this blog. IKEv2 is just an enhanced version of IKEv1, if IKEv1 is clear then IKEv2 is just a piece of cake.

There are 2 phases and modes in IKEv1. Phase-1 is known as Main Mode and Phase-2 is known as Quick Mode. There is a total of 9 messages that get exchanged in IKEv1 out of which 6 messages get exchanged in Phase-1 (Main Mode) and the remaining 3 messages get exchanged in Phase-2 (Quick Mode).

Phase-1 (Main Mode)

1. Main Mode Message-1: The very 1^st packet is sent by Initiator Tunnel Endpoint to Responder Tunnel Endpoint. This message contains the proposal of the Security Association which contains the Encryption, Hashing, Authentication, DH group, and Lifetime. Also, this message contains the Initiator cookie. This message is unencrypted.

• Main Mode Message-2: The 2^nd packet is sent by Responder Tunnel Endpoint to Initiator Tunnel Endpoint in response to the 1^st message. This message contains the Encryption, Hashing, Authentication, DH group, and Lifetime proposed by Responder. Also, this message contains the Initiator cookie as well as the Responder cookie. This message is unencrypted.

• Main Mode Message-3: If the algorithms shared in the Message-2 match with the Initiator, then only Initiator will proceed further with the 3^rd packet. In the 3^rd packet, Initiator sends a Diffie Hellman Public key along with the Nonce Key of Initiator. This message is unencrypted.

• Main Mode Message-4: Responder replies with its Diffie Hellman Public key and the Nonce key of the responder. This message is unencrypted.

• Main Mode Message-5: Initiator sends an ICV/Digest/Hash Value of its own Preshared key in encrypted format for authentication.  This message is encrypted.

• Main Mode Message- 6 Responder sends an ICV/Digest/Hash Value of its own Preshared key in encrypted format for authentication.  This message is encrypted.

Once the Hash value (digest of Preshared Key) matches on the IPSEC peers, they move further to the 2^nd Phase of IKEv1 i.e. Quick Mode.

Before moving on to the Quick Mode, You must be thinking about what the Nonce Key is, Initiator/Responder Cookie, Diffie Hellman Public key. So, Nonce Key and Cookie are the random values generated by both Initiator and responder. These keys help in generating the Session Key IDs which will be ultimately used for encrypting the further IKE messages, encrypting the data plane (ESP) traffic, integrity check of data plane packets using HMAC. DH we know as an Asymmetric Algorithm used for key exchange and generating the common shared secret.

Below mentioned are the Session Key IDs that get generated on both the tunnel endpoints after the 4 messages are exchanged. Session Key ID is a number created by a mathematical algorithm called “Pseudo-Random function (prf)”.  Below mentioned are the formulas of SKEYIDs.

1. SKEYID= prf (Pre Shared Key + Nonce Value of Initiator + Nonce Value of Responder).

This Session Key ID will be the same on both the IPSEC peers because both have the same preshared key and Nonce Values exchanged during the 3^rd and 4^th packet.

• SKEYID_D= prf (SKEYID + Initiator Cookie sent in 1^st packet + Responder Cookie sent in 2^nd packet + Shared Secret generated by the Diffie Hellman algorithm after exchanging Public Keys in 3^rd and 4^th Message). This SKEYID_D will be used for encrypting and decrypting the data plane traffic encapsulated with ESP.

• SKEYID_A= prf (SKEYID_D + SKEYID + Initiator Cookie + Responder Cookie + Shared Secret)

This SKEYID_A will be used for integrity check of data plane ESP packets as HMAC.

• SKEYID_E= prf (SKEYID_A + SKEYID + Initiator Cookie + Responder Cookie + Shared Secret)

We know that there is a total of 9 packets get exchanged in IKEv1 to set up the tunnel for the data plane. As of now, 4 packets are exchanged which were unencrypted but from the 5^th packet, all remaining packets will be encrypted. For encryption we need a key, This SKEYID_E will be used for encrypting ISAKMP Packet 5 to ISAKMP Packet 9.

Phase-2 (Quick Mode)

In Quick Mode, 3 messages are exchanged which basically contains the parameters/ protocols/ algorithms to be used for the data plane. Once this negotiation is completed, User traffic (data plane) can be started moving on the IPSEC tunnel.

• Quick Mode Message-1: Initiator sends this packet which contains majorly 3 important things. One is the protocol of encapsulation to be used for data plane whether ESP or AH, Second is the Encryption algorithm to be used for encrypting data plane traffic and third is the Hashing Algorithm to be used for integrity check of data plane traffic. This message is sent in an encrypted packet.
• Quick Mode Message-2: Responder sends this packet to the Initiator containing its own parameters/protocols/algorithms for encapsulation, encryption, hashing. This message is also encrypted.

• Quick Mode Message-3: Once the parameters match on both ends for data plane traffic exchanged in Quick Mode Messages 1 and 2, the initiator sends a message to start the data plane traffic. This message is also encrypted. After these 3 messages of Quick Mode, User traffic can be sent over an IPSEC tunnel using the parameters NEGOTIATED IN Quick Mode.

Are you planning to deploy IPSec VPN Solution between your 2 or more sites?

If you are planning for a Green-field Deployment, Tech Refresh, Migration from your legacy MPLS or plane GRE VPN to IPSec Solution or for implementing advanced features in your already deployed IPSec Solution or for deploying any VPN Solution. You can always reach out to us, the team at Zindagi Technologies consists of experts in the field of Network Security, Data Centre technologies, Enterprise & Service Provider Networks, Virtualization, Private Cloud, Public Cloud, Data Center Networks (LAN and SAN), Collaboration, Wireless, Surveillance, Openstack, ACI, storage and security technologies with over a 20 years of combined industry experience in planning, designing, implementing and optimizing complex Network Security and VPN deployments. To get in touch with us, you can give us a call at +91 9773973971.

Author
Harpreet Singh Batra
Consulting Engineer- Data Center and Network Security