Learning VPP: IPsec IKEv2

logo_fdio-300x184

Overview

Internet Key Exchange or IKE is the protocol used to set up IPsec connection using certificates.

Setup

Two Ubuntu 18.04 VMs with VPP 20.05.

VPP IKEv2

Prerequisites

First we need generate private keys and certificates and place them accordingly. To do that we need to install the strongswan and strongswan-pki packages. After that we run the following commands.

ipsec pki --gen  > server-key.der
ipsec pki --self --in server-key.der --dn "CN=vpp.home" > server-cert.der
openssl x509 -inform DER -in server-cert.der -out server-cert.pem
openssl rsa -inform DER -in server-key.der -out server-key.pem

ipsec pki --gen  > client-key.der
ipsec pki --self --in client-key.der --dn "CN=roadwarrior.vpn.example.com" > client-cert.der
openssl x509 -inform DER -in client-cert.der -out client-cert.pem
openssl rsa -inform DER -in client-key.der -out client-key.pem

VPP configuration

We need to configure responder first.

Responder

ikev2 profile add pr1
ikev2 profile set pr1 auth rsa-sig cert-file client-cert.pem
set ikev2 local key server-key.pem
ikev2 profile set pr1 id local fqdn vpp.home
ikev2 profile set pr1 id remote fqdn roadwarrior.vpn.example.com
ikev2 profile set pr1 traffic-selector remote ip-range 0.0.0.0 - 255.255.255.255 port-range 0 - 65535 protocol 0
ikev2 profile set pr1 traffic-selector local ip-range 0.0.0.0 - 255.255.255.255 port-range 0 - 65535 protocol 0

Now we are ready to configure initiator and start a connection.

Initiator

ikev2 profile add pr1
ikev2 profile set pr1 auth rsa-sig cert-file server-cert.pem
set ikev2 local key server1/client-key.pem
ikev2 profile set pr1 id local fqdn roadwarrior.vpn.example.com
ikev2 profile set pr1 id remote fqdn vpp.home
ikev2 profile set pr1 traffic-selector local ip-range 0.0.0.0 - 255.255.255.255 port-range 0 - 65535 protocol 0
ikev2 profile set pr1 traffic-selector remote ip-range 0.0.0.0 - 255.255.255.255 port-range 0 - 65535 protocol 0

ikev2 profile set pr1 responder GigabitEthernet0/3/0 192.168.0.123
ikev2 profile set pr1 ike-crypto-alg aes-cbc 256  ike-integ-alg sha1-96  ike-dh modp-2048
ikev2 profile set pr1 esp-crypto-alg aes-cbc 256  esp-integ-alg sha1-96  esp-dh ecp-256
ikev2 profile set pr1 sa-lifetime 3600 10 5 0

ikev2 initiate sa-init pr1

Results

Encap trace

DBGvpp# show ikev2 sa
iip 192.168.0.122 ispi 4c28e1c804fd1947 rip 192.168.0.123 rspi 399dc6c103195aaf
encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048
 nonce i:3d3efa1c7e22b2d8a71cee9a25dc9865b7f9390cc5779951c853f54d3c43f8a4        r:a5d1349d0c3361f4b83928a4ed7d830c0bb30ce1c24eec0fad8f1246d5aa3d13
 SK_d    6801b1efb1b2b1af7716aa59110232e11f6ab14d21a5bbed78a5e3df780accfd
 SK_a  i:ef80c745b9b00687b790c8733ef1259051792d5a
        r:88197e468c0bb1547da1ba83a615fda8bddafe70
 SK_e  i:e03c29186cb043aab949345b4b082d52be0a55a917f0871055e8201b4a82bbe6
        r:47ce9e6ca78758d0ca55b49c95db412f41f4d82473f183276b09a4aeca4acabf
 SK_p  i:5e97db586a7e3f2f0532c8ecbd360cb9a8b9894bc1f7bcccb253878b299a3689
        r:a9346e5827ccf6927acaa5fff0d9cc4461649154f4e01ceed410cdbb1985a596
 identifier (i) fqdn roadwarrior.vpn.example.com
 identifier (r) fqdn vpp.home
 child sa 0:
   encr:aes-cbc-256 integ:sha1-96 esn:yes 
    spi(i) df7eeb0c spi(r) 244bc72d
   SK_e  i:d5cdc8129b666eb0ef40111d9a78c4d8b053e28b2d28846d421c47f27f00d9fd
         r:3ee2ab1bbfce8b0714d735e2e13a18d44d274c9a214b88ff9a7d47170f364f94
   SK_a  i:ba24a4dabc09eb1da586437e2d28841c67043d33
         r:1f289f029b601b371f7946e93c14df252dd9fcc4
   traffic selectors (i):
     0 type 7 protocol_id 0 addr 0.0.0.0 - 255.255.255.255 port 0 - 65535
   traffic selectors (r):
     0 type 7 protocol_id 0 addr 0.0.0.0 - 255.255.255.255 port 0 - 65535
 iip 192.168.0.122 ispi 4c28e1c804fd1947 rip 192.168.0.123 rspi 399dc6c103195aaf

Renew certificates

If we want to renew certificates on both sides we need to do the following.

Responder

ikev2 profile set pr1 auth rsa-sig cert-file client-cert.pem
set ikev2 local key server-key.pem

Initiator

ikev2 initiate del-child-sa df7eeb0c
ikev2 profile set pr1 auth rsa-sig cert-file server-cert.pem
set ikev2 local key client-key.pem
ikev2 initiate sa-init pr1

References

Learning VPP: IPsec GRE over VxLAN

logo_fdio-300x184

Overview

The goal is to create a layer-2 encrypted tunnel and hide inner network IP addresses.

To achieve this goal, the traffic will be encapsulated in GRE, protected with IPsec and encapsulated into VxLAN.

GRE is a tunneling protocol developed by Cisco. The GRE frame looks as follows.

GRE frame

VXLAN tunnel is an L2 overlay on top of an L3 network underlay. It uses the UDP protocol to traverse the network. The VXLAN frame looks as follows.

VXLAN frame

IPsec supports tunnel and transport modes. As far as our tunnel is based on GRE, the transport mode will be used. In this mode, only a payload of the IP packet is encrypted and/or authenticated and the IP header is not touched. The resulting frame looks as follows.

IPSEC frame in transport mode

Setup

Two Ubuntu VMs with VPP ver. 19.01 and two Ubuntu VMs representing hosts.

VXLAN setup (1)

VPP configuration

In terms of VPP we need to create two loopbacks. One loopback will be bridged with GRE-IPsec tunnel, while another will be bridged with VxLAN tunnel. And using routing we will direct traffic into the first loopback where it will be encapsulated into GRE header and encrypted with IPsec. Then the traffic will be routed into a second loopback where it will receive VxLAN header.

Router1

ipsec sa add 10 spi 1001 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec sa add 20 spi 1000 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58

loopback create mac 1a:ab:3c:4d:5e:7f
set interface ip address loop0 10.100.0.7/31
set int mtu 1360 loop0
set int l2 learn loop0 disable
create ipsec gre tunnel src 10.101.0.7 dst 10.101.0.6 local-sa 10 remote-sa 20
set int state ipsec-gre0 up
create bridge-domain 12 learn 0 forward 1 uu-flood 1 flood 1 arp-term 1
set bridge-domain arp entry 12 10.100.0.7 1a:ab:3c:4d:5e:7f
set int l2 bridge loop0 12 bvi
set int l2 bridge ipsec-gre0 12 1

loopback create mac 1a:2b:3c:4d:5e:7f
set interface ip address loop1 10.101.0.7/31
create vxlan tunnel src 192.168.31.76 dst 192.168.31.47 vni 13
create bridge-domain 13 learn 0 forward 1 uu-flood 1 flood 1 arp-term 1
set bridge-domain arp entry 13 10.101.0.7 1a:2b:3c:4d:5e:7f
set interface l2 bridge vxlan_tunnel0 13 1
set interface l2 bridge loop1 13 bvi

ip route add 10.10.10.0/24 via 10.100.0.6

Router2

ipsec sa add 10 spi 1000 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec sa add 20 spi 1001 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58

loopback create mac 1a:ab:3c:4d:5e:6f
set interface ip address loop0 10.100.0.6/31
set int mtu 1360 loop0
set int l2 learn loop0 disable
create ipsec gre tunnel src 10.101.0.6 dst 10.101.0.7 local-sa 10 remote-sa 20
set int state ipsec-gre0 up
create bridge-domain 12 learn 0 forward 1 uu-flood 1 flood 1 arp-term 1
set bridge-domain arp entry 12 10.100.0.6 1a:ab:3c:4d:5e:6f
set int l2 bridge loop0 12 bvi
set int l2 bridge ipsec-gre0 12 1

loopback create mac 1a:2b:3c:4d:5e:6f
set interface ip address loop1 10.101.0.6/31
create vxlan tunnel src 192.168.31.47 dst 192.168.31.76 vni 13
create bridge-domain 13 learn 0 forward 1 uu-flood 1 flood 1 arp-term 1
set bridge-domain arp entry 13 10.101.0.6 1a:2b:3c:4d:5e:6f
set interface l2 bridge vxlan_tunnel0 13 1
set interface l2 bridge loop1 13 bvi

ip route add 20.20.20.0/24 via 10.100.0.7

Results

Encap trace

00:04:26:418264: dpdk-input
  GigabitEthernet0/8/0 rx queue 0
  buffer 0xddb4: current data 0, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x0
                 ext-hdr-valid 
                 l4-cksum-computed l4-cksum-correct 
  PKT MBUF: port 1, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr 0x8e376d80
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
  ICMP: 20.20.20.2 -> 10.10.10.2
    tos 0x00, ttl 64, length 84, checksum 0x66a0
    fragment id 0x97e7, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x2ea5
00:04:26:418338: ethernet-input
  frame: flags 0x3, hw-if-index 2, sw-if-index 2
  IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
00:04:26:418355: ip4-input-no-checksum
  ICMP: 20.20.20.2 -> 10.10.10.2
    tos 0x00, ttl 64, length 84, checksum 0x66a0
    fragment id 0x97e7, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x2ea5
00:04:26:418369: ip4-lookup
  fib 0 dpo-idx 14 flow hash: 0x00000000
  ICMP: 20.20.20.2 -> 10.10.10.2
    tos 0x00, ttl 64, length 84, checksum 0x66a0
    fragment id 0x97e7, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x2ea5
00:04:26:418385: ip4-rewrite
  tx_sw_if_index 3 dpo-idx 14 : ipv4 via 10.100.0.6 loop0: mtu:1360 1aab3c4d5e6f1aab3c4d5e7f0800 flow hash: 0x00000000
  00000000: 1aab3c4d5e6f1aab3c4d5e7f08004500005497e740003f0167a0141414020a0a
  00000020: 0a0208002ea555ec004d8f3afb5d0000000025b60400000000001011
00:04:26:418393: loop0-output
  loop0
  IP4: 1a:ab:3c:4d:5e:7f -> 1a:ab:3c:4d:5e:6f
  ICMP: 20.20.20.2 -> 10.10.10.2
    tos 0x00, ttl 63, length 84, checksum 0x67a0
    fragment id 0x97e7, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x2ea5
00:04:26:418417: l2-input
  l2-input: sw_if_index 3 dst 1a:ab:3c:4d:5e:6f src 1a:ab:3c:4d:5e:7f
00:04:26:418423: l2-fwd
  l2-fwd:   sw_if_index 3 dst 1a:ab:3c:4d:5e:6f src 1a:ab:3c:4d:5e:7f bd_index 1 result [0x1020000000004, 4] none
00:04:26:418428: l2-output
  l2-output: sw_if_index 4 dst 1a:ab:3c:4d:5e:6f src 1a:ab:3c:4d:5e:7f data 08 00 45 00 00 54 97 e7 40 00 3f 01
00:04:26:418434: ipsec-gre0-output
  ipsec-gre0
  00000000: 1aab3c4d5e6f1aab3c4d5e7f08004500005497e740003f0167a0141414020a0a
  00000020: 0a0208002ea555ec004d8f3afb5d0000000025b6040000000000101112131415
  00000040: 161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435
  00000060: 36370000000000000000000000000000000000000000000000000000
00:04:26:418436: ipsec-gre0-tx
  GRE: tunnel 0 len 122 src 10.101.0.7 dst 10.101.0.6 sa-id 10
00:04:26:418440: esp4-encrypt
  esp: spi 1001 seq 134 crypto aes-cbc-128 integrity sha1-96
00:04:26:418468: ip4-lookup
  fib 0 dpo-idx 19 flow hash: 0x00000000
  IPSEC_ESP: 10.101.0.7 -> 10.101.0.6
    tos 0x00, ttl 254, length 168, checksum 0xa74d
    fragment id 0x0000
00:04:26:418471: ip4-rewrite
  tx_sw_if_index 5 dpo-idx 19 : ipv4 via 10.101.0.6 loop1: mtu:9000 1a2b3c4d5e6f1a2b3c4d5e7f0800 flow hash: 0x00000000
  00000000: 1a2b3c4d5e6f1a2b3c4d5e7f0800450000a800000000fd32a84d0a6500070a65
  00000020: 0006000003e9000000870573ff85554266537fd108913fe1aba4e3fc
00:04:26:418485: loop1-output
  loop1
  IP4: 1a:2b:3c:4d:5e:7f -> 1a:2b:3c:4d:5e:6f
  IPSEC_ESP: 10.101.0.7 -> 10.101.0.6
    tos 0x00, ttl 253, length 168, checksum 0xa84d
    fragment id 0x0000
00:04:26:418488: l2-input
  l2-input: sw_if_index 5 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f
00:04:26:418489: l2-fwd
  l2-fwd:   sw_if_index 5 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f bd_index 2 result [0x1020000000006, 6] none
00:04:26:418491: l2-output
  l2-output: sw_if_index 6 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f data 08 00 45 00 00 a8 00 00 00 00 fd 32
00:04:26:418493: vxlan4-encap
  VXLAN encap to vxlan_tunnel0 vni 13
00:04:26:418499: ip4-rewrite
  tx_sw_if_index 1 dpo-idx 18 : ipv4 via 192.168.31.47 GigabitEthernet0/3/0: mtu:9000 08002768d11e0800275a18a50800 flow hash: 0x00000001
  00000000: 08002768d11e0800275a18a50800450000da00000000fd11fd46c0a81f4cc0a8
  00000020: 1f2f12b512b500c600000800000000000d001a2b3c4d5e6f1a2b3c4d
00:04:26:418500: GigabitEthernet0/3/0-output
  GigabitEthernet0/3/0
  IP4: 08:00:27:5a:18:a5 -> 08:00:27:68:d1:1e
  UDP: 192.168.31.76 -> 192.168.31.47
    tos 0x00, ttl 253, length 218, checksum 0xfd46
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 198, checksum 0x0000
00:04:26:418502: GigabitEthernet0/3/0-tx
  GigabitEthernet0/3/0 tx queue 0
  buffer 0x1c332: current data -50, length 232, free-list 0, clone-count 0, totlen-nifb 0, trace 0x0
  PKT MBUF: port 65535, nb_segs 1, pkt_len 232
    buf_len 2176, data_len 232, ol_flags 0x0, data_off 78, phys_addr 0x8e70cd00
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:5a:18:a5 -> 08:00:27:68:d1:1e
  UDP: 192.168.31.76 -> 192.168.31.47
    tos 0x00, ttl 253, length 218, checksum 0xfd46
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 198, checksum 0x0000

Decap trace

00:04:26:419224: dpdk-input
  GigabitEthernet0/3/0 rx queue 0
  buffer 0x1afa: current data 0, length 232, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1
                 ext-hdr-valid 
                 l4-cksum-computed l4-cksum-correct 
  PKT MBUF: port 0, nb_segs 1, pkt_len 232
    buf_len 2176, data_len 232, ol_flags 0x0, data_off 128, phys_addr 0x8dc6bf00
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:68:d1:1e -> 08:00:27:5a:18:a5
  UDP: 192.168.31.47 -> 192.168.31.76
    tos 0x00, ttl 253, length 218, checksum 0xfd46
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 198, checksum 0x0000
00:04:26:419299: ethernet-input
  frame: flags 0x3, hw-if-index 1, sw-if-index 1
  IP4: 08:00:27:68:d1:1e -> 08:00:27:5a:18:a5
00:04:26:419313: ip4-input-no-checksum
  UDP: 192.168.31.47 -> 192.168.31.76
    tos 0x00, ttl 253, length 218, checksum 0xfd46
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 198, checksum 0x0000
00:04:26:419320: ip4-lookup
  fib 0 dpo-idx 5 flow hash: 0x00000000
  UDP: 192.168.31.47 -> 192.168.31.76
    tos 0x00, ttl 253, length 218, checksum 0xfd46
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 198, checksum 0x0000
00:04:26:419328: ip4-local
    UDP: 192.168.31.47 -> 192.168.31.76
      tos 0x00, ttl 253, length 218, checksum 0xfd46
      fragment id 0x0000
    UDP: 4789 -> 4789
      length 198, checksum 0x0000
00:04:26:419334: ip4-udp-lookup
  UDP: src-port 4789 dst-port 4789
00:04:26:419345: vxlan4-input
  VXLAN decap from vxlan_tunnel0 vni 13 next 1 error 0
00:04:26:419358: l2-input
  l2-input: sw_if_index 6 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f
00:04:26:419365: l2-learn
  l2-learn: sw_if_index 6 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f bd_index 2
00:04:26:419375: l2-fwd
  l2-fwd:   sw_if_index 6 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f bd_index 2 result [0x700000005, 5] static age-not bvi 
00:04:26:419381: ip4-input
  IPSEC_ESP: 10.101.0.6 -> 10.101.0.7
    tos 0x00, ttl 253, length 168, checksum 0xa84d
    fragment id 0x0000
00:04:26:419385: ip4-lookup
  fib 0 dpo-idx 8 flow hash: 0x00000000
  IPSEC_ESP: 10.101.0.6 -> 10.101.0.7
    tos 0x00, ttl 253, length 168, checksum 0xa84d
    fragment id 0x0000
00:04:26:419387: ip4-local
    IPSEC_ESP: 10.101.0.6 -> 10.101.0.7
      tos 0x00, ttl 253, length 168, checksum 0xa84d
      fragment id 0x0000
00:04:26:419390: ipsec-if-input
  IPSec: spi 1000 seq 93
00:04:26:419399: esp4-decrypt
  esp: crypto aes-cbc-128 integrity sha1-96
00:04:26:419421: ipsec-gre-input
  GRE: tunnel -1 len 122 src 10.101.0.6 dst 10.101.0.7
00:04:26:419427: l2-input
  l2-input: sw_if_index 4 dst 1a:ab:3c:4d:5e:7f src 1a:ab:3c:4d:5e:6f
00:04:26:419429: l2-learn
  l2-learn: sw_if_index 4 dst 1a:ab:3c:4d:5e:7f src 1a:ab:3c:4d:5e:6f bd_index 1
00:04:26:419435: l2-fwd
  l2-fwd:   sw_if_index 4 dst 1a:ab:3c:4d:5e:7f src 1a:ab:3c:4d:5e:6f bd_index 1 result [0x700000003, 3] static age-not bvi 
00:04:26:419438: ip4-input
  ICMP: 10.10.10.2 -> 20.20.20.2
    tos 0x00, ttl 63, length 84, checksum 0x82f4
    fragment id 0xbc93
  ICMP echo_reply checksum 0x36a5
00:04:26:419441: ip4-lookup
  fib 0 dpo-idx 26 flow hash: 0x00000000
  ICMP: 10.10.10.2 -> 20.20.20.2
    tos 0x00, ttl 63, length 84, checksum 0x82f4
    fragment id 0xbc93
  ICMP echo_reply checksum 0x36a5
00:04:26:419445: ip4-rewrite
  tx_sw_if_index 2 dpo-idx 26 : ipv4 via 20.20.20.2 GigabitEthernet0/8/0: mtu:9000 0800275467a20800278833fd0800 flow hash: 0x00000000
  00000000: 0800275467a20800278833fd080045000054bc9300003e0183f40a0a0a021414
  00000020: 1402000036a555ec004d8f3afb5d0000000025b60400000000001011
00:04:26:419449: GigabitEthernet0/8/0-output
  GigabitEthernet0/8/0
  IP4: 08:00:27:88:33:fd -> 08:00:27:54:67:a2
  ICMP: 10.10.10.2 -> 20.20.20.2
    tos 0x00, ttl 62, length 84, checksum 0x83f4
    fragment id 0xbc93
  ICMP echo_reply checksum 0x36a5
00:04:26:419455: GigabitEthernet0/8/0-tx
  GigabitEthernet0/8/0 tx queue 0
  buffer 0x1c359: current data 38, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1
  PKT MBUF: port 65535, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x0, data_off 166, phys_addr 0x8e70d6c0
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:88:33:fd -> 08:00:27:54:67:a2
  ICMP: 10.10.10.2 -> 20.20.20.2
    tos 0x00, ttl 62, length 84, checksum 0x83f4
    fragment id 0xbc93
  ICMP echo_reply checksum 0x36a5

References

Learning VPP: VxLAN over IPsec

logo_fdio-300x184

Overview

The goal is to create a layer-2 encrypted tunnel. The traffic will be encapsulated in VxLAN and protected with IPsec.

VXLAN tunnel is an L2 overlay on top of an L3 network underlay. It uses the UDP protocol to traverse the network. The VXLAN frame looks as follows.

VXLAN frame

IPsec supports tunnel and transport modes. As far as our tunnel is based on VxLAN, the transport mode will be used. In this mode, only a payload of the IP packet is encrypted and/or authenticated and the IP header is not touched. The resulting frame looks as follows.

IPSEC frame in transport mode

Setup

Two Ubuntu VMs with VPP ver. 19.01 and two Ubuntu VMs representing hosts.

VXLAN setup (1)

VPP configuration

Router1

loopback create mac 1a:2b:3c:4d:5e:8f
create bridge-domain 13 learn 1 forward 1 uu-flood 1 flood 1 arp-term 0
create vxlan tunnel src 192.168.31.47 dst 192.168.31.76 vni 13
set interface l2 bridge vxlan_tunnel0 13 1
set interface l2 bridge loop0 13 bvi
set interface ip table loop0 0
set interface ip address loop0 10.100.0.6/31
ip route table 0 20.20.20.0/24 via loop0
ipsec sa add 10 spi 1000 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec sa add 20 spi 1001 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec spd add 1
set interface ipsec spd loop0 1
ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
ipsec policy add spd 1 priority 10 outbound action protect sa 10 local-ip-range 10.10.10.0 - 10.10.10.255 remote-ip-range 20.20.20.0 - 20.20.20.255
ipsec policy add spd 1 priority 10 inbound action protect sa 20 local-ip-range 10.10.10.0 - 10.10.10.255 remote-ip-range 20.20.20.0 - 20.20.20.255

Router2

loopback create mac 1a:2b:3c:4d:5e:7f
create bridge-domain 13 learn 1 forward 1 uu-flood 1 flood 1 arp-term 0
create vxlan tunnel src 192.168.31.76 dst 192.168.31.47 vni 13
set interface l2 bridge vxlan_tunnel0 13 1
set interface l2 bridge loop0 13 bvi
set interface ip table loop0 0
set interface ip address loop0 10.100.0.7/31
ip route table 0 10.10.10.0/24 via loop0
ipsec sa add 10 spi 1001 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec sa add 20 spi 1000 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec spd add 1
set interface ipsec spd loop0 1
ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
ipsec policy add spd 1 priority 10 outbound action protect sa 10 local-ip-range 20.20.20.0 - 20.20.20.255 remote-ip-range 10.10.10.0 - 10.10.10.255
ipsec policy add spd 1 priority 10 inbound action protect sa 20 local-ip-range 20.20.20.0 - 20.20.20.255 remote-ip-range 10.10.10.0 - 10.10.10.255

Results

Encap trace

00:01:37:265053: dpdk-input
GigabitEthernet0/8/0 rx queue 0
buffer 0xe663: current data 0, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 1, nb_segs 1, pkt_len 98
buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr 0x91b99940
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
ICMP: 20.20.20.2 -> 10.10.10.2
tos 0x00, ttl 64, length 84, checksum 0x5e94
fragment id 0x9ff3, flags DONT_FRAGMENT
ICMP echo_request checksum 0x5d06
00:01:37:265092: ethernet-input
frame: flags 0x3, hw-if-index 2, sw-if-index 2
IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
00:01:37:265103: ip4-input-no-checksum
ICMP: 20.20.20.2 -> 10.10.10.2
tos 0x00, ttl 64, length 84, checksum 0x5e94
fragment id 0x9ff3, flags DONT_FRAGMENT
ICMP echo_request checksum 0x5d06
00:01:37:265110: ip4-lookup
fib 0 dpo-idx 21 flow hash: 0x00000000
ICMP: 20.20.20.2 -> 10.10.10.2
tos 0x00, ttl 64, length 84, checksum 0x5e94
fragment id 0x9ff3, flags DONT_FRAGMENT
ICMP echo_request checksum 0x5d06
00:01:37:265118: ip4-rewrite
tx_sw_if_index 3 dpo-idx 21 : ipv4 via 10.100.0.6 loop0: mtu:9000 1a2b3c4d5e6f1a2b3c4d5e7f0800 flow hash: 0x00000000
00000000: 1a2b3c4d5e6f1a2b3c4d5e7f0800450000549ff340003f015f94141414020a0a
00000020: 0a0208005d0605030016ba1c935d0000000088930100000000001011
00:01:37:265123: ipsec4-output
spd 1
00:01:37:265131: esp4-encrypt
esp: spi 1001 seq 19 crypto aes-cbc-128 integrity sha1-96
00:01:37:265168: loop0-output
loop0
IP4: 1a:2b:3c:4d:5e:7f -> 1a:2b:3c:4d:5e:6f
IPSEC_ESP: 20.20.20.2 -> 10.10.10.2
tos 0x00, ttl 254, length 136, checksum 0x8022
fragment id 0x0000
00:01:37:265175: l2-input
l2-input: sw_if_index 3 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f
00:01:37:265179: l2-fwd
l2-fwd:   sw_if_index 3 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f bd_index 1 result [0x1010000000004, 4] none
00:01:37:265183: l2-output
l2-output: sw_if_index 4 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f data 08 00 45 00 00 88 00 00 00 00 fe 32
00:01:37:265188: vxlan4-encap
VXLAN encap to vxlan_tunnel0 vni 13
00:01:37:265192: ip4-rewrite
tx_sw_if_index 1 dpo-idx 15 : ipv4 via 192.168.31.47 GigabitEthernet0/3/0: mtu:9000 08002768d11e0800275a18a50800 flow hash: 0x00000001
00000000: 08002768d11e0800275a18a50800450000ba00000000fd11fd66c0a81f4cc0a8
00000020: 1f2f3b6112b500a600000800000000000d001a2b3c4d5e6f1a2b3c4d
00:01:37:265194: GigabitEthernet0/3/0-output
GigabitEthernet0/3/0
IP4: 08:00:27:5a:18:a5 -> 08:00:27:68:d1:1e
UDP: 192.168.31.76 -> 192.168.31.47
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 15201 -> 4789
length 166, checksum 0x0000
00:01:37:265196: GigabitEthernet0/3/0-tx
GigabitEthernet0/3/0 tx queue 0
buffer 0x1aca6: current data -50, length 200, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1
PKT MBUF: port 65535, nb_segs 1, pkt_len 200
buf_len 2176, data_len 200, ol_flags 0x0, data_off 78, phys_addr 0x916b2a00
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
IP4: 08:00:27:5a:18:a5 -> 08:00:27:68:d1:1e
UDP: 192.168.31.76 -> 192.168.31.47
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 15201 -> 4789
length 166, checksum 0x0000

Decap trace

00:01:37:265912: dpdk-input
GigabitEthernet0/3/0 rx queue 0
buffer 0x357c: current data 0, length 200, free-list 0, clone-count 0, totlen-nifb 0, trace 0x2
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 0, nb_segs 1, pkt_len 200
buf_len 2176, data_len 200, ol_flags 0x0, data_off 128, phys_addr 0x918d5f80
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
IP4: 08:00:27:68:d1:1e -> 08:00:27:5a:18:a5
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 62150 -> 4789
length 166, checksum 0x0000
00:01:37:265941: ethernet-input
frame: flags 0x3, hw-if-index 1, sw-if-index 1
IP4: 08:00:27:68:d1:1e -> 08:00:27:5a:18:a5
00:01:37:265945: ip4-input-no-checksum
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 62150 -> 4789
length 166, checksum 0x0000
00:01:37:265947: ip4-lookup
fib 0 dpo-idx 5 flow hash: 0x00000000
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 62150 -> 4789
length 166, checksum 0x0000
00:01:37:265951: ip4-local
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 62150 -> 4789
length 166, checksum 0x0000
00:01:37:265954: ip4-udp-lookup
UDP: src-port 62150 dst-port 4789
00:01:37:265959: vxlan4-input
VXLAN decap from vxlan_tunnel0 vni 13 next 1 error 0
00:01:37:265964: l2-input
l2-input: sw_if_index 4 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f
00:01:37:265967: l2-learn
l2-learn: sw_if_index 4 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f bd_index 1
00:01:37:265971: l2-fwd
l2-fwd:   sw_if_index 4 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f bd_index 1 result [0x700000003, 3] static age-not bvi
00:01:37:265974: ip4-input
IPSEC_ESP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 254, length 136, checksum 0x8022
fragment id 0x0000
00:01:37:265976: ipsec4-input
esp: sa_id 20 spd 1 spi 1000 seq 19
00:01:37:265980: esp4-decrypt
esp: crypto aes-cbc-128 integrity sha1-96
00:01:37:266019: ip4-input
ICMP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 254, length 84, checksum 0x8087
fragment id 0x0000
ICMP echo_reply checksum 0x6506
00:01:37:266021: ip4-lookup
fib 0 dpo-idx 23 flow hash: 0x00000000
ICMP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 254, length 84, checksum 0x8087
fragment id 0x0000
ICMP echo_reply checksum 0x6506
00:01:37:266024: ip4-rewrite
tx_sw_if_index 2 dpo-idx 23 : ipv4 via 20.20.20.2 GigabitEthernet0/8/0: mtu:9000 0800275467a20800278833fd0800 flow hash: 0x00000000
00000000: 0800275467a20800278833fd08004500005400000000fd0181870a0a0a021414
00000020: 14020000650605030016ba1c935d0000000088930100000000001011
00:01:37:266025: GigabitEthernet0/8/0-output
GigabitEthernet0/8/0
IP4: 08:00:27:88:33:fd -> 08:00:27:54:67:a2
ICMP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 253, length 84, checksum 0x8187
fragment id 0x0000
ICMP echo_reply checksum 0x6506
00:01:37:266029: GigabitEthernet0/8/0-tx
GigabitEthernet0/8/0 tx queue 0
buffer 0x1accd: current data 0, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x2
ip4
PKT MBUF: port 65535, nb_segs 1, pkt_len 98
buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr 0x916b33c0
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
IP4: 08:00:27:88:33:fd -> 08:00:27:54:67:a2
ICMP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 253, length 84, checksum 0x8187
fragment id 0x0000
ICMP echo_reply checksum 0x6506

References