Learning VPP: IPsec IKEv2

logo_fdio-300x184

Overview

Internet Key Exchange or IKE is the protocol used to set up IPsec connection using certificates.

Setup

Two Ubuntu 18.04 VMs with VPP 20.05.

VPP IKEv2

Prerequisites

First we need generate private keys and certificates and place them accordingly. To do that we need to install the strongswan and strongswan-pki packages. After that we run the following commands.

ipsec pki --gen  > server-key.der
ipsec pki --self --in server-key.der --dn "CN=vpp.home" > server-cert.der
openssl x509 -inform DER -in server-cert.der -out server-cert.pem
openssl rsa -inform DER -in server-key.der -out server-key.pem

ipsec pki --gen  > client-key.der
ipsec pki --self --in client-key.der --dn "CN=roadwarrior.vpn.example.com" > client-cert.der
openssl x509 -inform DER -in client-cert.der -out client-cert.pem
openssl rsa -inform DER -in client-key.der -out client-key.pem

VPP configuration

We need to configure responder first.

Responder

ikev2 profile add pr1
ikev2 profile set pr1 auth rsa-sig cert-file client-cert.pem
set ikev2 local key server-key.pem
ikev2 profile set pr1 id local fqdn vpp.home
ikev2 profile set pr1 id remote fqdn roadwarrior.vpn.example.com
ikev2 profile set pr1 traffic-selector remote ip-range 0.0.0.0 - 255.255.255.255 port-range 0 - 65535 protocol 0
ikev2 profile set pr1 traffic-selector local ip-range 0.0.0.0 - 255.255.255.255 port-range 0 - 65535 protocol 0

Now we are ready to configure initiator and start a connection.

Initiator

ikev2 profile add pr1
ikev2 profile set pr1 auth rsa-sig cert-file server-cert.pem
set ikev2 local key server1/client-key.pem
ikev2 profile set pr1 id local fqdn roadwarrior.vpn.example.com
ikev2 profile set pr1 id remote fqdn vpp.home
ikev2 profile set pr1 traffic-selector local ip-range 0.0.0.0 - 255.255.255.255 port-range 0 - 65535 protocol 0
ikev2 profile set pr1 traffic-selector remote ip-range 0.0.0.0 - 255.255.255.255 port-range 0 - 65535 protocol 0

ikev2 profile set pr1 responder GigabitEthernet0/3/0 192.168.0.123
ikev2 profile set pr1 ike-crypto-alg aes-cbc 256  ike-integ-alg sha1-96  ike-dh modp-2048
ikev2 profile set pr1 esp-crypto-alg aes-cbc 256  esp-integ-alg sha1-96  esp-dh ecp-256
ikev2 profile set pr1 sa-lifetime 3600 10 5 0

ikev2 initiate sa-init pr1

Results

Encap trace

DBGvpp# show ikev2 sa
iip 192.168.0.122 ispi 4c28e1c804fd1947 rip 192.168.0.123 rspi 399dc6c103195aaf
encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048
 nonce i:3d3efa1c7e22b2d8a71cee9a25dc9865b7f9390cc5779951c853f54d3c43f8a4        r:a5d1349d0c3361f4b83928a4ed7d830c0bb30ce1c24eec0fad8f1246d5aa3d13
 SK_d    6801b1efb1b2b1af7716aa59110232e11f6ab14d21a5bbed78a5e3df780accfd
 SK_a  i:ef80c745b9b00687b790c8733ef1259051792d5a
        r:88197e468c0bb1547da1ba83a615fda8bddafe70
 SK_e  i:e03c29186cb043aab949345b4b082d52be0a55a917f0871055e8201b4a82bbe6
        r:47ce9e6ca78758d0ca55b49c95db412f41f4d82473f183276b09a4aeca4acabf
 SK_p  i:5e97db586a7e3f2f0532c8ecbd360cb9a8b9894bc1f7bcccb253878b299a3689
        r:a9346e5827ccf6927acaa5fff0d9cc4461649154f4e01ceed410cdbb1985a596
 identifier (i) fqdn roadwarrior.vpn.example.com
 identifier (r) fqdn vpp.home
 child sa 0:
   encr:aes-cbc-256 integ:sha1-96 esn:yes 
    spi(i) df7eeb0c spi(r) 244bc72d
   SK_e  i:d5cdc8129b666eb0ef40111d9a78c4d8b053e28b2d28846d421c47f27f00d9fd
         r:3ee2ab1bbfce8b0714d735e2e13a18d44d274c9a214b88ff9a7d47170f364f94
   SK_a  i:ba24a4dabc09eb1da586437e2d28841c67043d33
         r:1f289f029b601b371f7946e93c14df252dd9fcc4
   traffic selectors (i):
     0 type 7 protocol_id 0 addr 0.0.0.0 - 255.255.255.255 port 0 - 65535
   traffic selectors (r):
     0 type 7 protocol_id 0 addr 0.0.0.0 - 255.255.255.255 port 0 - 65535
 iip 192.168.0.122 ispi 4c28e1c804fd1947 rip 192.168.0.123 rspi 399dc6c103195aaf

Renew certificates

If we want to renew certificates on both sides we need to do the following.

Responder

ikev2 profile set pr1 auth rsa-sig cert-file client-cert.pem
set ikev2 local key server-key.pem

Initiator

ikev2 initiate del-child-sa df7eeb0c
ikev2 profile set pr1 auth rsa-sig cert-file server-cert.pem
set ikev2 local key client-key.pem
ikev2 initiate sa-init pr1

References

Learning VPP: OSPF routing protocol

logo_fdio-300x184

Overview

The task at hand is to enable OSPF on VPP router. For this purpose is chosen FRRouting (FRR), which is an IP routing protocol suite for Linux and Unix platforms.

I will use VPP’s router plugin, that implements logic to punt control packets to the Linux network stack and a Netlink-based mechanism that synchronizes the Linux’s routing table into VPP’s FIB.

In order to compile the router plugin with VPP version 20, I had to make few modifications to source code that can be seen on Github.

Topology

I have used 4 VirtualBox VMs for two VPP routers and two hosts.

VPP_OSPF

VPP VM configuration looks as follows.

VPP_VM_config

Host VM configuration looks as follows.

Host_VM_config

Install VPP from the package

For your convenience, I have uploaded prebuilt Debian packages targeted against Ubuntu 18.04. The VPP will be installed and started as a service with a router plugin enabled.

curl -s https://packagecloud.io/install/repositories/emflex/fdio/script.deb.sh | sudo bash
sudo apt-get install vpp vpp-plugin-core vpp-plugin-dpdk vpp-ext-deps

Install VPP from source

Here, I have introduced a script to download and build VPP from sources together with the router plugin.

git clone https://github.com/garyachy/frr-vpp.git
cd frr-vpp
sudo ./build_vpp.sh
sudo dpkg -i vpp/build-root/*.deb

Install FRR from the package

curl -s https://deb.frrouting.org/frr/keys.asc | sudo apt-key add -
FRRVER="frr-stable"
echo deb https://deb.frrouting.org/frr $(lsb_release -s -c) $FRRVER | sudo tee -a /etc/apt/sources.list.d/frr.list
sudo apt update && sudo apt install frr frr-pythontools

Configure FRR

Make the following changes to /etc/frr/daemons

ospfd=yes
ospfd_options=" -A 127.0.0.1 -f /etc/frr/ospfd.conf"

Make the following changes to /etc/frr/ospfd.conf

hostname ospfd
password zebra
log file /var/log/frr/ospfd.log informational
log stdout
!
router ospf
    ospf router-id 10.10.10.1
    network 10.10.10.1/24 area 0.0.0.0
    network 100.100.100.0/24 area 0.0.0.0
!

Configure Netplan

VPP interface need to be added into /etc/netplan/50-cloud-init.yaml and netplan apply executed.

network:
    ethernets:
        enp0s3:
            dhcp4: true
        vpp0:
            addresses:
            - 10.10.10.1/24
        vpp1:
            addresses:
            - 100.100.100.1/24
    version: 2

Results

VPP creates TAP interfaces in Linux.

denys@vpp1:~$ ip addr
5: vpp0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether 08:00:27:4a:5c:a2 brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.1/24 brd 10.10.10.255 scope global vpp0
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe4a:5ca2/64 scope link 
       valid_lft forever preferred_lft forever
6: vpp1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether 08:00:27:7a:4b:93 brd ff:ff:ff:ff:ff:ff
    inet 100.100.100.1/24 brd 100.100.100.255 scope global vpp1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe7a:4b93/64 scope link 
       valid_lft forever preferred_lft forever

You can see that OSPF routes were installed into Linux.

denys@vpp1:~$ ip route
default via 192.168.0.1 dev enp0s3 proto dhcp src 192.168.0.108 metric 100 
10.10.10.0/24 dev vpp0 proto kernel scope link src 10.10.10.1 
20.20.20.0/24 via 100.100.100.2 dev vpp1 proto ospf metric 20 
100.100.100.0/24 dev vpp1 proto kernel scope link src 100.100.100.1 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 
192.168.0.0/24 dev enp0s3 proto kernel scope link src 192.168.0.108 
192.168.0.1 dev enp0s3 proto dhcp scope link src 192.168.0.108 metric 100

Also, hosts can ping each other.

denys@host2:~$ ping 10.10.10.2
PING 10.10.10.2 (10.10.10.2) 56(84) bytes of data.
64 bytes from 10.10.10.2: icmp_seq=2 ttl=62 time=0.769 ms
64 bytes from 10.10.10.2: icmp_seq=3 ttl=62 time=0.547 ms
64 bytes from 10.10.10.2: icmp_seq=4 ttl=62 time=0.593 ms

References

Learning VPP: NAT

logo_fdio-300x184

Overview

We will use NAT feature to enable hosts connected to VPP router access the Internet.

Configuration

We will need NAT extra features that are enabled only in endpoint dependent mode. Also, we need to increase limits for NAT translations that are too small by default.

So we need to add the following lines into startup.conf file.

nat {
    endpoint-dependent
    translation hash buckets 1048576
    translation hash memory 268435456
    user hash buckets 1024
    max translations per user 10000
 }

After VPP is started the following commands will enable NAT on two interfaces.

nat44 add interface address GigabitEthernet0/3/0
nat addr-port-assignment-alg default
set interface nat44 in GigabitEthernet0/8/0 out GigabitEthernet0/3/0 output-feature
nat44 forwarding enable

Bypassing NAT

To access VPP using ssh the following command is applied.

nat44 add static mapping local 192.168.31.130 22 external GigabitEthernet0/3/0 22 tcp

To forbid NAT change source port of the outgoing specific traffic the following command is used.

nat44 add identity mapping 192.168.31.130 udp 4789

References

Learning VPP: ABF

logo_fdio-300x184

Overview

ABF stands for ACL Based Forwarding. ABF is a subset of PBR (Policy Based Routing). ABF is different from normal IP routing in that the lookup by IP destination address is replaced by a match using ACL rules.

Testing

Run VAT.

./build-root/build-vpp_debug-native/vpp/bin/vpp
./vpp/build-root/build-vpp_debug-native/vpp/bin/vpp_api_test

Create ACL rules.

vat# acl_add_replace ipv4 permit dst 8.8.8.8/32
acl_dump
vl_api_acl_add_replace_reply_t_handler:108: ACL index: 0
vat# vl_api_acl_details_t_handler:222: acl_index: 0, count: 1
   tag {}
   ipv4 action 1 src 0.0.0.0/0 dst 8.8.8.8/32 proto 0 sport 0-65535 dport 0-65535 tcpflags 0 mask 0

Create a policy.

DBGvpp# abf policy add id 0 acl 0 via 10.100.0.4 loop2
DBGvpp# show abf policy                                                                                    
abf:[0]: policy:0 acl:0
     path-list:[43] locks:1 flags:shared,no-uRPF, uRPF-list: None
      path:[45] pl-index:43 ip4 weight=1 pref=0 attached-nexthop:  oper-flags:resolved,
        10.100.0.4 loop2
      [@0]: arp-ipv4: via 10.100.0.4 loop2

Bind to an interface.

DBGvpp# abf attach ip4 policy 0 GigabitEthernet0/8/0         
DBGvpp# show abf attach GigabitEthernet0/8/0
ipv4:
 abf-interface-attach: policy:0 priority:0
  [@1]: arp-ipv4: via 10.100.0.4 loop2

Trace without ABF

00:02:27:818282: dpdk-input
  GigabitEthernet0/8/0 rx queue 0
  buffer 0xd886: current data 0, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1
                 ext-hdr-valid 
                 l4-cksum-computed l4-cksum-correct 
  PKT MBUF: port 1, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr 0x8b162200
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
  ICMP: 20.20.20.2 -> 8.8.8.8
    tos 0x00, ttl 64, length 84, checksum 0x85e9
    fragment id 0x7c9a, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x8a71
00:02:27:818326: ethernet-input
  frame: flags 0x3, hw-if-index 2, sw-if-index 2
  IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
00:02:27:818337: ip4-input-no-checksum
  ICMP: 20.20.20.2 -> 8.8.8.8
    tos 0x00, ttl 64, length 84, checksum 0x85e9
    fragment id 0x7c9a, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x8a71
00:02:27:818350: ip4-lookup
  fib 0 dpo-idx 11 flow hash: 0x00000000
  ICMP: 20.20.20.2 -> 8.8.8.8
    tos 0x00, ttl 64, length 84, checksum 0x85e9
    fragment id 0x7c9a, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x8a71
00:02:27:818357: ip4-rewrite
  tx_sw_if_index 1 dpo-idx 11 : ipv4 via 192.168.0.1 GigabitEthernet0/3/0: mtu:9000 98ded060c14f0800275a18a50800 flow hash: 0x00000000
  00000000: 98ded060c14f0800275a18a50800450000547c9a40003f0186e9141414020808
  00000020: 080808008a7106b2069c0292205e00000000707d0e00000000001011
00:02:27:818362: nat44-ed-in2out-output
  NAT44_IN2OUT_ED_FAST_PATH: sw_if_index 2, next index 3, session -1
00:02:27:818369: nat44-ed-in2out-output-slowpath
  NAT44_IN2OUT_ED_SLOW_PATH: sw_if_index 2, next index 0, session 5
00:02:27:818376: GigabitEthernet0/3/0-output
  GigabitEthernet0/3/0
  IP4: 08:00:27:5a:18:a5 -> 98:de:d0:60:c1:4f
  ICMP: 192.168.0.106 -> 8.8.8.8
    tos 0x00, ttl 63, length 84, checksum 0xedec
    fragment id 0x7c9a, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x51a6
00:02:27:818381: GigabitEthernet0/3/0-tx
  GigabitEthernet0/3/0 tx queue 0
  buffer 0xd886: current data 0, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1
                 ext-hdr-valid 
                 l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 l3-hdr-offset 14 
  PKT MBUF: port 1, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr 0x8b162200
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:5a:18:a5 -> 98:de:d0:60:c1:4f
  ICMP: 192.168.0.106 -> 8.8.8.8
    tos 0x00, ttl 63, length 84, checksum 0xedec
    fragment id 0x7c9a, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x51a6

Trace with ABF

From the trace below it is clear that traffic traverses abf-input-ip4 node. As a result, it is encapsulated in VxLAN and forwarded through a tunnel.

00:03:30:398890: dpdk-input
  GigabitEthernet0/8/0 rx queue 0
  buffer 0xcec6: current data 0, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x3
                 ext-hdr-valid 
                 l4-cksum-computed l4-cksum-correct 
  PKT MBUF: port 1, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr 0x8d73b200
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
  ICMP: 20.20.20.2 -> 8.8.8.8
    tos 0x00, ttl 64, length 84, checksum 0xce6e
    fragment id 0x3415, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x1312
00:03:30:398931: ethernet-input
  frame: flags 0x3, hw-if-index 2, sw-if-index 2
  IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
00:03:30:399080: ip4-input-no-checksum
  ICMP: 20.20.20.2 -> 8.8.8.8
    tos 0x00, ttl 64, length 84, checksum 0xce6e
    fragment id 0x3415, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x1312
00:03:30:399382: abf-input-ip4
   next 1 index 12
00:03:30:399535: ip4-rewrite
  tx_sw_if_index 3 dpo-idx 12 : ipv4 via 10.100.0.4 loop2: mtu:1360 020027fd0004020027fd00050800 flow hash: 0x00000000
  00000000: 020027fd0004020027fd0005080045000054341540003f01cf6e141414020808
  00000020: 08080800131206b200096c8b205e0000000091760100000000001011
00:03:30:399713: loop2-output
  loop2
  IP4: 02:00:27:fd:00:05 -> 02:00:27:fd:00:04
  ICMP: 20.20.20.2 -> 8.8.8.8
    tos 0x00, ttl 63, length 84, checksum 0xcf6e
    fragment id 0x3415, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x1312
00:03:30:400172: l2-input
  l2-input: sw_if_index 3 dst 02:00:27:fd:00:04 src 02:00:27:fd:00:05
00:03:30:400389: l2-fwd
  l2-fwd:   sw_if_index 3 dst 02:00:27:fd:00:04 src 02:00:27:fd:00:05 bd_index 1 result [0xffffffffffffffff, -1] static age-not bvi filter learn-event learn-move 
00:03:30:400617: l2-flood
  l2-flood: sw_if_index 3 dst 02:00:27:fd:00:04 src 02:00:27:fd:00:05 bd_index 1
00:03:30:400894: l2-output
  l2-output: sw_if_index 4 dst 02:00:27:fd:00:04 src 02:00:27:fd:00:05 data 08 00 45 00 00 54 34 15 40 00 3f 01
00:03:30:401147: ipsec-gre0-output
  ipsec-gre0
  00000000: 020027fd0004020027fd0005080045000054341540003f01cf6e141414020808
  00000020: 08080800131206b200096c8b205e000000009176010000000000101112131415
  00000040: 161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435
  00000060: 36370000000000000000000000000000000000000000000000000000
00:03:30:401413: ipsec-gre0-tx
  GRE: tunnel 0 len 122 src 10.101.0.5 dst 10.101.0.4 sa-id 1
00:03:30:401757: esp4-encrypt
  esp: spi 5 seq 469 crypto aes-cbc-128 integrity sha-256-128
00:03:30:402114: ip4-lookup
  fib 0 dpo-idx 16 flow hash: 0x00000000
  IPSEC_ESP: 10.101.0.5 -> 10.101.0.4
    tos 0x00, ttl 254, length 172, checksum 0xa74d
    fragment id 0x0000
00:03:30:402450: ip4-rewrite
  tx_sw_if_index 5 dpo-idx 16 : ipv4 via 10.101.0.4 loop3: mtu:9000 020027fe0004020027fe00050800 flow hash: 0x00000000
  00000000: 020027fe0004020027fe00050800450000ac00000000fd32a84d0a6500050a65
  00000020: 000400000005000001d6c6fd0b0258faf7c67f3d2a5c88dd30723e7a
00:03:30:402820: loop3-output
  loop3
  IP4: 02:00:27:fe:00:05 -> 02:00:27:fe:00:04
  IPSEC_ESP: 10.101.0.5 -> 10.101.0.4
    tos 0x00, ttl 253, length 172, checksum 0xa84d
    fragment id 0x0000
00:03:30:403637: l2-input
  l2-input: sw_if_index 5 dst 02:00:27:fe:00:04 src 02:00:27:fe:00:05
00:03:30:404046: l2-fwd
  l2-fwd:   sw_if_index 5 dst 02:00:27:fe:00:04 src 02:00:27:fe:00:05 bd_index 2 result [0xffffffffffffffff, -1] static age-not bvi filter learn-event learn-move 
00:03:30:404494: l2-flood
  l2-flood: sw_if_index 5 dst 02:00:27:fe:00:04 src 02:00:27:fe:00:05 bd_index 2
00:03:30:404950: l2-output
  l2-output: sw_if_index 6 dst 02:00:27:fe:00:04 src 02:00:27:fe:00:05 data 08 00 45 00 00 ac 00 00 00 00 fd 32
00:03:30:405421: vxlan4-encap
  VXLAN encap to vxlan_tunnel0 vni 3
00:03:30:405921: ip4-rewrite
  tx_sw_if_index 1 dpo-idx 17 : ipv4 via 192.168.0.104 GigabitEthernet0/3/0: mtu:9000 08002768d11e0800275a18a50800 flow hash: 0x00000001
  00000000: 08002768d11e0800275a18a50800450000de00000000fd113aecc0a8006ac0a8
  00000020: 006812b512b500ca00000800000000000300020027fe0004020027fe
00:03:30:406389: nat44-ed-in2out-output
  NAT44_IN2OUT_ED_FAST_PATH: sw_if_index 5, next index 3, session -1
00:03:30:406922: nat44-ed-in2out-output-slowpath
  NAT44_IN2OUT_ED_SLOW_PATH: sw_if_index 5, next index 0, session -1
00:03:30:408014: GigabitEthernet0/3/0-output
  GigabitEthernet0/3/0
  IP4: 08:00:27:5a:18:a5 -> 08:00:27:68:d1:1e
  UDP: 192.168.0.106 -> 192.168.0.104
    tos 0x00, ttl 253, length 222, checksum 0x3aec
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 202, checksum 0x0000
00:03:30:408548: GigabitEthernet0/3/0-tx
  GigabitEthernet0/3/0 tx queue 0
  buffer 0x16c54: current data -50, length 236, free-list 0, clone-count 0, totlen-nifb 0, trace 0x3
  PKT MBUF: port 65535, nb_segs 1, pkt_len 236
    buf_len 2176, data_len 236, ol_flags 0x0, data_off 78, phys_addr 0x8d1b1580
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:5a:18:a5 -> 08:00:27:68:d1:1e
  UDP: 192.168.0.106 -> 192.168.0.104
    tos 0x00, ttl 253, length 222, checksum 0x3aec
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 202, checksum 0x0000

References

Learning VPP: Python API

logo_fdio-300x184

Overview

VPP preferred interface is a binary API that is used by Northbound control plane applications like Honeycomb. Furthermore, VPP provides an autogenerated Python wrapper for your convenience.

While Python API channel is significantly slower than native binary API (1500 messages/second versus 450000), it is still a great option.

There are three classes of VPP API methods:

  1. Synchronous request/reply.
  2. Dump functions.
  3. Events.

Install

1. Build VPP.

2. Check JSON API definitions under

vpp/build-root/install-vpp_debug-native/vpp/share/vpp/api/core

3. Set LD_LIBRARY_PATH to point to the location of libvppapiclient.so

4. Install Python package.

cd vpp
cd src/vpp-api/python
sudo python setup.py install

Code

The code below is doing the following.

  1. Connect to VPP.
  2. Print version.
  3. Print all interfaces.
  4. Subscribe on interface statistics updates.
  5. Disconnect from VPP.
#!/bin/env python

from __future__ import print_function
import os
import fnmatch
import time
from vpp_papi import VPP

def papi_event_handler(msgname, result):
    print(msgname)
    print(result)
 
vpp_json_dir = '/usr/share/vpp/api/'
 
jsonfiles = []
for root, dirnames, filenames in os.walk(vpp_json_dir):
    for filename in fnmatch.filter(filenames, '*.api.json'):
        jsonfiles.append(os.path.join(vpp_json_dir, filename))
 
if not jsonfiles:
    print('Error: no json api files found')
    exit(-1)
 
vpp = VPP(jsonfiles)
r = vpp.connect('papi-example')

rv = vpp.api.show_version()
print('VPP version =', rv.version.decode().rstrip('\0x00'))

for intf in vpp.api.sw_interface_dump():
    print(intf.interface_name.decode())

async=True                                                                                                                                                                                                
r=vpp.register_event_callback(papi_event_handler)
pid=os.getpid()
sw_ifs = [2]
r = vpp.api.want_per_interface_simple_stats(enable_disable=True, sw_ifs=sw_ifs, num=len(sw_ifs), pid=pid)
print(r)
                                                                                                                                                                                
time.sleep(60)
r = vpp.api.want_per_interface_simple_stats(enable_disable=False)
 
r = vpp.disconnect()
exit(r)

Output

sudo python vpp.py
VPP version = 19.01.00.01.01-rc0~11-gc069ff9
local0
GigabitEthernet0/3/0
GigabitEthernet0/8/0
want_per_interface_simple_stats_reply(_0=862, context=3, retval=0)
vnet_per_interface_simple_counters
vnet_per_interface_simple_counters(_0=887, count=1, timestamp=785, data=[vl_api_vnet_simple_counter_t(sw_if_index=2, drop=44, punt=0, rx_ip4=62, rx_ip6=0, rx_no_buffer=0, rx_miss=0, rx_error=0, tx_error=44, rx_mpls=0)])
vnet_per_interface_simple_counters
vnet_per_interface_simple_counters(_0=887, count=1, timestamp=795, data=[vl_api_vnet_simple_counter_t(sw_if_index=2, drop=44, punt=0, rx_ip4=72, rx_ip6=0, rx_no_buffer=0, rx_miss=0, rx_error=0, tx_error=44, rx_mpls=0)])
vnet_per_interface_simple_counters

References

Learning VPP: IPsec GRE over VxLAN

logo_fdio-300x184

Overview

The goal is to create a layer-2 encrypted tunnel and hide inner network IP addresses.

To achieve this goal, the traffic will be encapsulated in GRE, protected with IPsec and encapsulated into VxLAN.

GRE is a tunneling protocol developed by Cisco. The GRE frame looks as follows.

GRE frame

VXLAN tunnel is an L2 overlay on top of an L3 network underlay. It uses the UDP protocol to traverse the network. The VXLAN frame looks as follows.

VXLAN frame

IPsec supports tunnel and transport modes. As far as our tunnel is based on GRE, the transport mode will be used. In this mode, only a payload of the IP packet is encrypted and/or authenticated and the IP header is not touched. The resulting frame looks as follows.

IPSEC frame in transport mode

Setup

Two Ubuntu VMs with VPP ver. 19.01 and two Ubuntu VMs representing hosts.

VXLAN setup (1)

VPP configuration

In terms of VPP we need to create two loopbacks. One loopback will be bridged with GRE-IPsec tunnel, while another will be bridged with VxLAN tunnel. And using routing we will direct traffic into the first loopback where it will be encapsulated into GRE header and encrypted with IPsec. Then the traffic will be routed into a second loopback where it will receive VxLAN header.

Router1

ipsec sa add 10 spi 1001 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec sa add 20 spi 1000 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58

loopback create mac 1a:ab:3c:4d:5e:7f
set interface ip address loop0 10.100.0.7/31
set int mtu 1360 loop0
set int l2 learn loop0 disable
create ipsec gre tunnel src 10.101.0.7 dst 10.101.0.6 local-sa 10 remote-sa 20
set int state ipsec-gre0 up
create bridge-domain 12 learn 0 forward 1 uu-flood 1 flood 1 arp-term 1
set bridge-domain arp entry 12 10.100.0.7 1a:ab:3c:4d:5e:7f
set int l2 bridge loop0 12 bvi
set int l2 bridge ipsec-gre0 12 1

loopback create mac 1a:2b:3c:4d:5e:7f
set interface ip address loop1 10.101.0.7/31
create vxlan tunnel src 192.168.31.76 dst 192.168.31.47 vni 13
create bridge-domain 13 learn 0 forward 1 uu-flood 1 flood 1 arp-term 1
set bridge-domain arp entry 13 10.101.0.7 1a:2b:3c:4d:5e:7f
set interface l2 bridge vxlan_tunnel0 13 1
set interface l2 bridge loop1 13 bvi

ip route add 10.10.10.0/24 via 10.100.0.6

Router2

ipsec sa add 10 spi 1000 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec sa add 20 spi 1001 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58

loopback create mac 1a:ab:3c:4d:5e:6f
set interface ip address loop0 10.100.0.6/31
set int mtu 1360 loop0
set int l2 learn loop0 disable
create ipsec gre tunnel src 10.101.0.6 dst 10.101.0.7 local-sa 10 remote-sa 20
set int state ipsec-gre0 up
create bridge-domain 12 learn 0 forward 1 uu-flood 1 flood 1 arp-term 1
set bridge-domain arp entry 12 10.100.0.6 1a:ab:3c:4d:5e:6f
set int l2 bridge loop0 12 bvi
set int l2 bridge ipsec-gre0 12 1

loopback create mac 1a:2b:3c:4d:5e:6f
set interface ip address loop1 10.101.0.6/31
create vxlan tunnel src 192.168.31.47 dst 192.168.31.76 vni 13
create bridge-domain 13 learn 0 forward 1 uu-flood 1 flood 1 arp-term 1
set bridge-domain arp entry 13 10.101.0.6 1a:2b:3c:4d:5e:6f
set interface l2 bridge vxlan_tunnel0 13 1
set interface l2 bridge loop1 13 bvi

ip route add 20.20.20.0/24 via 10.100.0.7

Results

Encap trace

00:04:26:418264: dpdk-input
  GigabitEthernet0/8/0 rx queue 0
  buffer 0xddb4: current data 0, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x0
                 ext-hdr-valid 
                 l4-cksum-computed l4-cksum-correct 
  PKT MBUF: port 1, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr 0x8e376d80
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
  ICMP: 20.20.20.2 -> 10.10.10.2
    tos 0x00, ttl 64, length 84, checksum 0x66a0
    fragment id 0x97e7, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x2ea5
00:04:26:418338: ethernet-input
  frame: flags 0x3, hw-if-index 2, sw-if-index 2
  IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
00:04:26:418355: ip4-input-no-checksum
  ICMP: 20.20.20.2 -> 10.10.10.2
    tos 0x00, ttl 64, length 84, checksum 0x66a0
    fragment id 0x97e7, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x2ea5
00:04:26:418369: ip4-lookup
  fib 0 dpo-idx 14 flow hash: 0x00000000
  ICMP: 20.20.20.2 -> 10.10.10.2
    tos 0x00, ttl 64, length 84, checksum 0x66a0
    fragment id 0x97e7, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x2ea5
00:04:26:418385: ip4-rewrite
  tx_sw_if_index 3 dpo-idx 14 : ipv4 via 10.100.0.6 loop0: mtu:1360 1aab3c4d5e6f1aab3c4d5e7f0800 flow hash: 0x00000000
  00000000: 1aab3c4d5e6f1aab3c4d5e7f08004500005497e740003f0167a0141414020a0a
  00000020: 0a0208002ea555ec004d8f3afb5d0000000025b60400000000001011
00:04:26:418393: loop0-output
  loop0
  IP4: 1a:ab:3c:4d:5e:7f -> 1a:ab:3c:4d:5e:6f
  ICMP: 20.20.20.2 -> 10.10.10.2
    tos 0x00, ttl 63, length 84, checksum 0x67a0
    fragment id 0x97e7, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x2ea5
00:04:26:418417: l2-input
  l2-input: sw_if_index 3 dst 1a:ab:3c:4d:5e:6f src 1a:ab:3c:4d:5e:7f
00:04:26:418423: l2-fwd
  l2-fwd:   sw_if_index 3 dst 1a:ab:3c:4d:5e:6f src 1a:ab:3c:4d:5e:7f bd_index 1 result [0x1020000000004, 4] none
00:04:26:418428: l2-output
  l2-output: sw_if_index 4 dst 1a:ab:3c:4d:5e:6f src 1a:ab:3c:4d:5e:7f data 08 00 45 00 00 54 97 e7 40 00 3f 01
00:04:26:418434: ipsec-gre0-output
  ipsec-gre0
  00000000: 1aab3c4d5e6f1aab3c4d5e7f08004500005497e740003f0167a0141414020a0a
  00000020: 0a0208002ea555ec004d8f3afb5d0000000025b6040000000000101112131415
  00000040: 161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435
  00000060: 36370000000000000000000000000000000000000000000000000000
00:04:26:418436: ipsec-gre0-tx
  GRE: tunnel 0 len 122 src 10.101.0.7 dst 10.101.0.6 sa-id 10
00:04:26:418440: esp4-encrypt
  esp: spi 1001 seq 134 crypto aes-cbc-128 integrity sha1-96
00:04:26:418468: ip4-lookup
  fib 0 dpo-idx 19 flow hash: 0x00000000
  IPSEC_ESP: 10.101.0.7 -> 10.101.0.6
    tos 0x00, ttl 254, length 168, checksum 0xa74d
    fragment id 0x0000
00:04:26:418471: ip4-rewrite
  tx_sw_if_index 5 dpo-idx 19 : ipv4 via 10.101.0.6 loop1: mtu:9000 1a2b3c4d5e6f1a2b3c4d5e7f0800 flow hash: 0x00000000
  00000000: 1a2b3c4d5e6f1a2b3c4d5e7f0800450000a800000000fd32a84d0a6500070a65
  00000020: 0006000003e9000000870573ff85554266537fd108913fe1aba4e3fc
00:04:26:418485: loop1-output
  loop1
  IP4: 1a:2b:3c:4d:5e:7f -> 1a:2b:3c:4d:5e:6f
  IPSEC_ESP: 10.101.0.7 -> 10.101.0.6
    tos 0x00, ttl 253, length 168, checksum 0xa84d
    fragment id 0x0000
00:04:26:418488: l2-input
  l2-input: sw_if_index 5 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f
00:04:26:418489: l2-fwd
  l2-fwd:   sw_if_index 5 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f bd_index 2 result [0x1020000000006, 6] none
00:04:26:418491: l2-output
  l2-output: sw_if_index 6 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f data 08 00 45 00 00 a8 00 00 00 00 fd 32
00:04:26:418493: vxlan4-encap
  VXLAN encap to vxlan_tunnel0 vni 13
00:04:26:418499: ip4-rewrite
  tx_sw_if_index 1 dpo-idx 18 : ipv4 via 192.168.31.47 GigabitEthernet0/3/0: mtu:9000 08002768d11e0800275a18a50800 flow hash: 0x00000001
  00000000: 08002768d11e0800275a18a50800450000da00000000fd11fd46c0a81f4cc0a8
  00000020: 1f2f12b512b500c600000800000000000d001a2b3c4d5e6f1a2b3c4d
00:04:26:418500: GigabitEthernet0/3/0-output
  GigabitEthernet0/3/0
  IP4: 08:00:27:5a:18:a5 -> 08:00:27:68:d1:1e
  UDP: 192.168.31.76 -> 192.168.31.47
    tos 0x00, ttl 253, length 218, checksum 0xfd46
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 198, checksum 0x0000
00:04:26:418502: GigabitEthernet0/3/0-tx
  GigabitEthernet0/3/0 tx queue 0
  buffer 0x1c332: current data -50, length 232, free-list 0, clone-count 0, totlen-nifb 0, trace 0x0
  PKT MBUF: port 65535, nb_segs 1, pkt_len 232
    buf_len 2176, data_len 232, ol_flags 0x0, data_off 78, phys_addr 0x8e70cd00
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:5a:18:a5 -> 08:00:27:68:d1:1e
  UDP: 192.168.31.76 -> 192.168.31.47
    tos 0x00, ttl 253, length 218, checksum 0xfd46
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 198, checksum 0x0000

Decap trace

00:04:26:419224: dpdk-input
  GigabitEthernet0/3/0 rx queue 0
  buffer 0x1afa: current data 0, length 232, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1
                 ext-hdr-valid 
                 l4-cksum-computed l4-cksum-correct 
  PKT MBUF: port 0, nb_segs 1, pkt_len 232
    buf_len 2176, data_len 232, ol_flags 0x0, data_off 128, phys_addr 0x8dc6bf00
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:68:d1:1e -> 08:00:27:5a:18:a5
  UDP: 192.168.31.47 -> 192.168.31.76
    tos 0x00, ttl 253, length 218, checksum 0xfd46
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 198, checksum 0x0000
00:04:26:419299: ethernet-input
  frame: flags 0x3, hw-if-index 1, sw-if-index 1
  IP4: 08:00:27:68:d1:1e -> 08:00:27:5a:18:a5
00:04:26:419313: ip4-input-no-checksum
  UDP: 192.168.31.47 -> 192.168.31.76
    tos 0x00, ttl 253, length 218, checksum 0xfd46
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 198, checksum 0x0000
00:04:26:419320: ip4-lookup
  fib 0 dpo-idx 5 flow hash: 0x00000000
  UDP: 192.168.31.47 -> 192.168.31.76
    tos 0x00, ttl 253, length 218, checksum 0xfd46
    fragment id 0x0000
  UDP: 4789 -> 4789
    length 198, checksum 0x0000
00:04:26:419328: ip4-local
    UDP: 192.168.31.47 -> 192.168.31.76
      tos 0x00, ttl 253, length 218, checksum 0xfd46
      fragment id 0x0000
    UDP: 4789 -> 4789
      length 198, checksum 0x0000
00:04:26:419334: ip4-udp-lookup
  UDP: src-port 4789 dst-port 4789
00:04:26:419345: vxlan4-input
  VXLAN decap from vxlan_tunnel0 vni 13 next 1 error 0
00:04:26:419358: l2-input
  l2-input: sw_if_index 6 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f
00:04:26:419365: l2-learn
  l2-learn: sw_if_index 6 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f bd_index 2
00:04:26:419375: l2-fwd
  l2-fwd:   sw_if_index 6 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f bd_index 2 result [0x700000005, 5] static age-not bvi 
00:04:26:419381: ip4-input
  IPSEC_ESP: 10.101.0.6 -> 10.101.0.7
    tos 0x00, ttl 253, length 168, checksum 0xa84d
    fragment id 0x0000
00:04:26:419385: ip4-lookup
  fib 0 dpo-idx 8 flow hash: 0x00000000
  IPSEC_ESP: 10.101.0.6 -> 10.101.0.7
    tos 0x00, ttl 253, length 168, checksum 0xa84d
    fragment id 0x0000
00:04:26:419387: ip4-local
    IPSEC_ESP: 10.101.0.6 -> 10.101.0.7
      tos 0x00, ttl 253, length 168, checksum 0xa84d
      fragment id 0x0000
00:04:26:419390: ipsec-if-input
  IPSec: spi 1000 seq 93
00:04:26:419399: esp4-decrypt
  esp: crypto aes-cbc-128 integrity sha1-96
00:04:26:419421: ipsec-gre-input
  GRE: tunnel -1 len 122 src 10.101.0.6 dst 10.101.0.7
00:04:26:419427: l2-input
  l2-input: sw_if_index 4 dst 1a:ab:3c:4d:5e:7f src 1a:ab:3c:4d:5e:6f
00:04:26:419429: l2-learn
  l2-learn: sw_if_index 4 dst 1a:ab:3c:4d:5e:7f src 1a:ab:3c:4d:5e:6f bd_index 1
00:04:26:419435: l2-fwd
  l2-fwd:   sw_if_index 4 dst 1a:ab:3c:4d:5e:7f src 1a:ab:3c:4d:5e:6f bd_index 1 result [0x700000003, 3] static age-not bvi 
00:04:26:419438: ip4-input
  ICMP: 10.10.10.2 -> 20.20.20.2
    tos 0x00, ttl 63, length 84, checksum 0x82f4
    fragment id 0xbc93
  ICMP echo_reply checksum 0x36a5
00:04:26:419441: ip4-lookup
  fib 0 dpo-idx 26 flow hash: 0x00000000
  ICMP: 10.10.10.2 -> 20.20.20.2
    tos 0x00, ttl 63, length 84, checksum 0x82f4
    fragment id 0xbc93
  ICMP echo_reply checksum 0x36a5
00:04:26:419445: ip4-rewrite
  tx_sw_if_index 2 dpo-idx 26 : ipv4 via 20.20.20.2 GigabitEthernet0/8/0: mtu:9000 0800275467a20800278833fd0800 flow hash: 0x00000000
  00000000: 0800275467a20800278833fd080045000054bc9300003e0183f40a0a0a021414
  00000020: 1402000036a555ec004d8f3afb5d0000000025b60400000000001011
00:04:26:419449: GigabitEthernet0/8/0-output
  GigabitEthernet0/8/0
  IP4: 08:00:27:88:33:fd -> 08:00:27:54:67:a2
  ICMP: 10.10.10.2 -> 20.20.20.2
    tos 0x00, ttl 62, length 84, checksum 0x83f4
    fragment id 0xbc93
  ICMP echo_reply checksum 0x36a5
00:04:26:419455: GigabitEthernet0/8/0-tx
  GigabitEthernet0/8/0 tx queue 0
  buffer 0x1c359: current data 38, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1
  PKT MBUF: port 65535, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x0, data_off 166, phys_addr 0x8e70d6c0
    packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x0 fdir.hi 0x0 fdir.lo 0x0
  IP4: 08:00:27:88:33:fd -> 08:00:27:54:67:a2
  ICMP: 10.10.10.2 -> 20.20.20.2
    tos 0x00, ttl 62, length 84, checksum 0x83f4
    fragment id 0xbc93
  ICMP echo_reply checksum 0x36a5

References

Learning VPP: ACL

logo_fdio-300x184

Overview

Our goal is to leverage the ACL plugin for traffic classification based on L3/L4 header fields. The ACL plugin does not supply the CLI for configuration but all APIs are covered in VAT CLI.

Integration

First, register as a user of the ACL plugin API.

acl_plugin_exports_init (&acl_plugin);
acl_user_id = acl_plugin.register_user_module ("Test", "label1", "label2");
acl_lc_id = acl_plugin.get_lookup_context_index (acl_user_id, 1, 2);

Second, add ACL rules into the current context.

vec_add1 (acl_vec, 0);
vec_add1 (acl_vec, 1);
acl_plugin.set_acl_vec_for_context (acl_lc_id, acl_vec);
vec_free (acl_vec);

Third, match traffic against the ACL rules.

acl_plugin_fill_5tuple_inline (acl_plugin.p_acl_main,
                               acl_lc_id, b0,
                               is_ip60,
                               /* is_input */ 0,
                               /* is_l2_path */ 1,
                               &pkt_5tuple0);

res = acl_plugin_match_5tuple_inline (acl_plugin.p_acl_main,
                                      acl_lc_id,
                                      &pkt_5tuple0, is_ip60,
                                      &action0, &acl_pos_p0,
                                      &acl_match_p0,
                                      &rule_match_p0,
                                      &trace_bitmap0);
if (res > 0)
{
    printf ("Rule matched! \n");
}

Testing

Build and run VPP. And run VAT.

./build-root/build-vpp_debug-native/vpp/bin/vpp
./vpp/build-root/build-vpp_debug-native/vpp/bin/vpp_api_test

Create ACL rules.

vat# acl_add_replace ipv6 permit dst 2001:db8::1/128, ipv4 permit src 192.0.2.1/32
vl_api_acl_add_replace_reply_t_handler:108: ACL index: 0
vat# acl_add_replace ipv6 permit dst 2001:db8::1/128, ipv4 permit src 10.10.2.1/32
vl_api_acl_add_replace_reply_t_handler:108: ACL index: 1

Check registered ACL users and ACL rules in CLI.

DBGvpp# show acl-plugin lookup context 
index 0:Test label1: 1 label2: 2, acl_indices: 0, 1
DBGvpp# show acl-plugin acl            
acl-index 0 count 2 tag {}
          0: ipv6 permit src ::/0 dst 2001:db8::1/128 proto 0 sport 0-65535 dport 0-65535
          1: ipv4 permit src 192.0.2.1/32 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535
  used in lookup context index: 0
acl-index 1 count 2 tag {}
          0: ipv6 permit src ::/0 dst 2001:db8::1/128 proto 0 sport 0-65535 dport 0-65535
          1: ipv4 permit src 10.10.2.1/32 dst 0.0.0.0/0 proto 0 sport 0-65535 dport 0-65535
  used in lookup context index: 0

References

 

Learning VPP: VxLAN over IPsec

logo_fdio-300x184

Overview

The goal is to create a layer-2 encrypted tunnel. The traffic will be encapsulated in VxLAN and protected with IPsec.

VXLAN tunnel is an L2 overlay on top of an L3 network underlay. It uses the UDP protocol to traverse the network. The VXLAN frame looks as follows.

VXLAN frame

IPsec supports tunnel and transport modes. As far as our tunnel is based on VxLAN, the transport mode will be used. In this mode, only a payload of the IP packet is encrypted and/or authenticated and the IP header is not touched. The resulting frame looks as follows.

IPSEC frame in transport mode

Setup

Two Ubuntu VMs with VPP ver. 19.01 and two Ubuntu VMs representing hosts.

VXLAN setup (1)

VPP configuration

Router1

loopback create mac 1a:2b:3c:4d:5e:8f
create bridge-domain 13 learn 1 forward 1 uu-flood 1 flood 1 arp-term 0
create vxlan tunnel src 192.168.31.47 dst 192.168.31.76 vni 13
set interface l2 bridge vxlan_tunnel0 13 1
set interface l2 bridge loop0 13 bvi
set interface ip table loop0 0
set interface ip address loop0 10.100.0.6/31
ip route table 0 20.20.20.0/24 via loop0
ipsec sa add 10 spi 1000 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec sa add 20 spi 1001 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec spd add 1
set interface ipsec spd loop0 1
ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
ipsec policy add spd 1 priority 10 outbound action protect sa 10 local-ip-range 10.10.10.0 - 10.10.10.255 remote-ip-range 20.20.20.0 - 20.20.20.255
ipsec policy add spd 1 priority 10 inbound action protect sa 20 local-ip-range 10.10.10.0 - 10.10.10.255 remote-ip-range 20.20.20.0 - 20.20.20.255

Router2

loopback create mac 1a:2b:3c:4d:5e:7f
create bridge-domain 13 learn 1 forward 1 uu-flood 1 flood 1 arp-term 0
create vxlan tunnel src 192.168.31.76 dst 192.168.31.47 vni 13
set interface l2 bridge vxlan_tunnel0 13 1
set interface l2 bridge loop0 13 bvi
set interface ip table loop0 0
set interface ip address loop0 10.100.0.7/31
ip route table 0 10.10.10.0/24 via loop0
ipsec sa add 10 spi 1001 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec sa add 20 spi 1000 esp crypto-alg aes-cbc-128 crypto-key 4a506a794f574265564551694d653768 integ-alg sha1-96 integ-key 4339314b55523947594d6d3547666b45764e6a58
ipsec spd add 1
set interface ipsec spd loop0 1
ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
ipsec policy add spd 1 priority 10 outbound action protect sa 10 local-ip-range 20.20.20.0 - 20.20.20.255 remote-ip-range 10.10.10.0 - 10.10.10.255
ipsec policy add spd 1 priority 10 inbound action protect sa 20 local-ip-range 20.20.20.0 - 20.20.20.255 remote-ip-range 10.10.10.0 - 10.10.10.255

Results

Encap trace

00:01:37:265053: dpdk-input
GigabitEthernet0/8/0 rx queue 0
buffer 0xe663: current data 0, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 1, nb_segs 1, pkt_len 98
buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr 0x91b99940
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
ICMP: 20.20.20.2 -> 10.10.10.2
tos 0x00, ttl 64, length 84, checksum 0x5e94
fragment id 0x9ff3, flags DONT_FRAGMENT
ICMP echo_request checksum 0x5d06
00:01:37:265092: ethernet-input
frame: flags 0x3, hw-if-index 2, sw-if-index 2
IP4: 08:00:27:54:67:a2 -> 08:00:27:88:33:fd
00:01:37:265103: ip4-input-no-checksum
ICMP: 20.20.20.2 -> 10.10.10.2
tos 0x00, ttl 64, length 84, checksum 0x5e94
fragment id 0x9ff3, flags DONT_FRAGMENT
ICMP echo_request checksum 0x5d06
00:01:37:265110: ip4-lookup
fib 0 dpo-idx 21 flow hash: 0x00000000
ICMP: 20.20.20.2 -> 10.10.10.2
tos 0x00, ttl 64, length 84, checksum 0x5e94
fragment id 0x9ff3, flags DONT_FRAGMENT
ICMP echo_request checksum 0x5d06
00:01:37:265118: ip4-rewrite
tx_sw_if_index 3 dpo-idx 21 : ipv4 via 10.100.0.6 loop0: mtu:9000 1a2b3c4d5e6f1a2b3c4d5e7f0800 flow hash: 0x00000000
00000000: 1a2b3c4d5e6f1a2b3c4d5e7f0800450000549ff340003f015f94141414020a0a
00000020: 0a0208005d0605030016ba1c935d0000000088930100000000001011
00:01:37:265123: ipsec4-output
spd 1
00:01:37:265131: esp4-encrypt
esp: spi 1001 seq 19 crypto aes-cbc-128 integrity sha1-96
00:01:37:265168: loop0-output
loop0
IP4: 1a:2b:3c:4d:5e:7f -> 1a:2b:3c:4d:5e:6f
IPSEC_ESP: 20.20.20.2 -> 10.10.10.2
tos 0x00, ttl 254, length 136, checksum 0x8022
fragment id 0x0000
00:01:37:265175: l2-input
l2-input: sw_if_index 3 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f
00:01:37:265179: l2-fwd
l2-fwd:   sw_if_index 3 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f bd_index 1 result [0x1010000000004, 4] none
00:01:37:265183: l2-output
l2-output: sw_if_index 4 dst 1a:2b:3c:4d:5e:6f src 1a:2b:3c:4d:5e:7f data 08 00 45 00 00 88 00 00 00 00 fe 32
00:01:37:265188: vxlan4-encap
VXLAN encap to vxlan_tunnel0 vni 13
00:01:37:265192: ip4-rewrite
tx_sw_if_index 1 dpo-idx 15 : ipv4 via 192.168.31.47 GigabitEthernet0/3/0: mtu:9000 08002768d11e0800275a18a50800 flow hash: 0x00000001
00000000: 08002768d11e0800275a18a50800450000ba00000000fd11fd66c0a81f4cc0a8
00000020: 1f2f3b6112b500a600000800000000000d001a2b3c4d5e6f1a2b3c4d
00:01:37:265194: GigabitEthernet0/3/0-output
GigabitEthernet0/3/0
IP4: 08:00:27:5a:18:a5 -> 08:00:27:68:d1:1e
UDP: 192.168.31.76 -> 192.168.31.47
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 15201 -> 4789
length 166, checksum 0x0000
00:01:37:265196: GigabitEthernet0/3/0-tx
GigabitEthernet0/3/0 tx queue 0
buffer 0x1aca6: current data -50, length 200, free-list 0, clone-count 0, totlen-nifb 0, trace 0x1
PKT MBUF: port 65535, nb_segs 1, pkt_len 200
buf_len 2176, data_len 200, ol_flags 0x0, data_off 78, phys_addr 0x916b2a00
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
IP4: 08:00:27:5a:18:a5 -> 08:00:27:68:d1:1e
UDP: 192.168.31.76 -> 192.168.31.47
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 15201 -> 4789
length 166, checksum 0x0000

Decap trace

00:01:37:265912: dpdk-input
GigabitEthernet0/3/0 rx queue 0
buffer 0x357c: current data 0, length 200, free-list 0, clone-count 0, totlen-nifb 0, trace 0x2
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 0, nb_segs 1, pkt_len 200
buf_len 2176, data_len 200, ol_flags 0x0, data_off 128, phys_addr 0x918d5f80
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
IP4: 08:00:27:68:d1:1e -> 08:00:27:5a:18:a5
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 62150 -> 4789
length 166, checksum 0x0000
00:01:37:265941: ethernet-input
frame: flags 0x3, hw-if-index 1, sw-if-index 1
IP4: 08:00:27:68:d1:1e -> 08:00:27:5a:18:a5
00:01:37:265945: ip4-input-no-checksum
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 62150 -> 4789
length 166, checksum 0x0000
00:01:37:265947: ip4-lookup
fib 0 dpo-idx 5 flow hash: 0x00000000
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 62150 -> 4789
length 166, checksum 0x0000
00:01:37:265951: ip4-local
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 186, checksum 0xfd66
fragment id 0x0000
UDP: 62150 -> 4789
length 166, checksum 0x0000
00:01:37:265954: ip4-udp-lookup
UDP: src-port 62150 dst-port 4789
00:01:37:265959: vxlan4-input
VXLAN decap from vxlan_tunnel0 vni 13 next 1 error 0
00:01:37:265964: l2-input
l2-input: sw_if_index 4 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f
00:01:37:265967: l2-learn
l2-learn: sw_if_index 4 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f bd_index 1
00:01:37:265971: l2-fwd
l2-fwd:   sw_if_index 4 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:6f bd_index 1 result [0x700000003, 3] static age-not bvi
00:01:37:265974: ip4-input
IPSEC_ESP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 254, length 136, checksum 0x8022
fragment id 0x0000
00:01:37:265976: ipsec4-input
esp: sa_id 20 spd 1 spi 1000 seq 19
00:01:37:265980: esp4-decrypt
esp: crypto aes-cbc-128 integrity sha1-96
00:01:37:266019: ip4-input
ICMP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 254, length 84, checksum 0x8087
fragment id 0x0000
ICMP echo_reply checksum 0x6506
00:01:37:266021: ip4-lookup
fib 0 dpo-idx 23 flow hash: 0x00000000
ICMP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 254, length 84, checksum 0x8087
fragment id 0x0000
ICMP echo_reply checksum 0x6506
00:01:37:266024: ip4-rewrite
tx_sw_if_index 2 dpo-idx 23 : ipv4 via 20.20.20.2 GigabitEthernet0/8/0: mtu:9000 0800275467a20800278833fd0800 flow hash: 0x00000000
00000000: 0800275467a20800278833fd08004500005400000000fd0181870a0a0a021414
00000020: 14020000650605030016ba1c935d0000000088930100000000001011
00:01:37:266025: GigabitEthernet0/8/0-output
GigabitEthernet0/8/0
IP4: 08:00:27:88:33:fd -> 08:00:27:54:67:a2
ICMP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 253, length 84, checksum 0x8187
fragment id 0x0000
ICMP echo_reply checksum 0x6506
00:01:37:266029: GigabitEthernet0/8/0-tx
GigabitEthernet0/8/0 tx queue 0
buffer 0x1accd: current data 0, length 98, free-list 0, clone-count 0, totlen-nifb 0, trace 0x2
ip4
PKT MBUF: port 65535, nb_segs 1, pkt_len 98
buf_len 2176, data_len 98, ol_flags 0x0, data_off 128, phys_addr 0x916b33c0
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
IP4: 08:00:27:88:33:fd -> 08:00:27:54:67:a2
ICMP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 253, length 84, checksum 0x8187
fragment id 0x0000
ICMP echo_reply checksum 0x6506

References

Learning VPP: Trace with Wireshark

logo_fdio-300x184

Overview

Each node in VPP is equipped with a possibility to trace the packets. This is a great debugging tool to investigate the issues with traffic. But analyzing trace log in a text form is a tiresome exercise.

But not anymore, as soon as the latest Wireshark supports VPP pcap dispatch trace dissector. As a result, you have an amazing tool to analyze all the changes that happen with a packet buffer while travelling through the VPP node graph.

Setup

VPP

Initiate and stop trace recording using the following commands.

pcap dispatch trace on max 1000 file vppcapture buffer-trace dpdk-input 1000
pcap dispatch trace off

Wireshark

Download and build the latest Wireshark on Ubuntu 18.04.

apt-get install -y libgcrypt11-dev flex bison qtbase5-dev qttools5-dev-tools qttools5-dev qtmultimedia5-dev libqt5svg5-dev libpcap-dev qt5-default libc-ares-dev
git clone https://gitlab.com/wireshark/wireshark.git
cd wireshark
mkdir build
cd build
cmake -G Ninja ../
ninja -j 8
sudo ninja install

Open the file /tmp/vppcapture with Wireshark and make the following changes into “Preferences”.

Wireshark_preferences

Results

As a result, you get the following invaluable recording of the journey that the packet buffer took through the VPP node graph. Here you can find all the metadata information that is traveling from node to node.

Wireshark_capture

References

Learning VPP: VXLAN tunnel

logo_fdio-300x184

Overview

VXLAN tunnel is an L2 overlay on top of an L3 network underlay. It uses the UDP protocol to traverse the network. The VXLAN frame looks as follows.

VXLAN frame

Setup

Two Ubuntu VMs with VPP ver. 19.01 and two Ubuntu VMs representing hosts.

VXLAN setup (1)

VPP configuration

Router1

loopback create mac 1a:2b:3c:4d:5e:8f
create bridge-domain 13 learn 1 forward 1 uu-flood 1 flood 1 arp-term 0
create vxlan tunnel src 192.168.31.47 dst 192.168.31.76 vni 13
set interface l2 bridge vxlan_tunnel0 13 1
set interface l2 bridge loop0 13 bvi
set interface ip table loop0 0

Router2

loopback create mac 1a:2b:3c:4d:5e:7f
create bridge-domain 13 learn 1 forward 1 uu-flood 1 flood 1 arp-term 0
create vxlan tunnel src 192.168.31.76 dst 192.168.31.47 vni 13
set interface l2 bridge vxlan_tunnel0 13 1
set interface l2 bridge loop0 13 bvi
set interface ip table loop0 0

Results

Packet trace

00:03:43:444347: dpdk-input
GigabitEthernet0/3/0 rx queue 0
buffer 0x9d811: current data 0, length 148, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x8
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 0, nb_segs 1, pkt_len 148
buf_len 2176, data_len 148, ol_flags 0x0, data_off 128, phys_addr 0x8c1604c0
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
IP4: 08:00:27:68:d1:1e -> 08:00:27:5a:18:a5
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 134, checksum 0xfd9a
fragment id 0x0000
UDP: 28591 -> 4789
length 114, checksum 0x0000
00:03:43:444389: ethernet-input
frame: flags 0x3, hw-if-index 1, sw-if-index 1
IP4: 08:00:27:68:d1:1e -> 08:00:27:5a:18:a5
00:03:43:444399: ip4-input-no-checksum
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 134, checksum 0xfd9a
fragment id 0x0000
UDP: 28591 -> 4789
length 114, checksum 0x0000
00:03:43:444406: ip4-lookup
fib 0 dpo-idx 5 flow hash: 0x00000000
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 134, checksum 0xfd9a
fragment id 0x0000
UDP: 28591 -> 4789
length 114, checksum 0x0000
00:03:43:444414: ip4-local
UDP: 192.168.31.47 -> 192.168.31.76
tos 0x00, ttl 253, length 134, checksum 0xfd9a
fragment id 0x0000
UDP: 28591 -> 4789
length 114, checksum 0x0000
00:03:43:444418: ip4-udp-lookup
UDP: src-port 28591 dst-port 4789
00:03:43:444423: vxlan4-input
VXLAN decap from vxlan_tunnel0 vni 13 next 1 error 0
00:03:43:444429: l2-input
l2-input: sw_if_index 4 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:8f
00:03:43:444436: l2-learn
l2-learn: sw_if_index 4 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:8f bd_index 1
00:03:43:444441: l2-fwd
l2-fwd: sw_if_index 4 dst 1a:2b:3c:4d:5e:7f src 1a:2b:3c:4d:5e:8f bd_index 1 result [0x700000003, 3] static age-not bvi
00:03:43:444446: ip4-input
ICMP: 10.100.0.6 -> 20.20.20.1
tos 0x00, ttl 64, length 84, checksum 0xabb7
fragment id 0x5c73, flags DONT_FRAGMENT
ICMP echo_request checksum 0xd8d2
00:03:43:444449: ip4-lookup
fib 0 dpo-idx 4 flow hash: 0x00000000
ICMP: 10.100.0.6 -> 20.20.20.1
tos 0x00, ttl 64, length 84, checksum 0xabb7
fragment id 0x5c73, flags DONT_FRAGMENT
ICMP echo_request checksum 0xd8d2
00:03:43:444451: ip4-local
ICMP: 10.100.0.6 -> 20.20.20.1
tos 0x00, ttl 64, length 84, checksum 0xabb7
fragment id 0x5c73, flags DONT_FRAGMENT
ICMP echo_request checksum 0xd8d2

Node counters

Count Node Reason
31 null-node blackholed packets
24 dpdk-input no error
9 ip4-udp-lookup no error
4 ip4-input ip4 source lookup miss
267 ip4-input Multicast RPF check failed
1 ip4-arp ARP requests sent
281 vxlan4-input good packets decapsulated
357 vxlan4-encap good packets encapsulated
357 l2-output L2 output packets
281 l2-learn L2 learn packets
1 l2-learn L2 learn misses
638 l2-input L2 input packets
81 l2-flood L2 flood packets
33 GigabitEthernet0/3/0-output interface is down
35 GigabitEthernet0/8/0-output interface is down

References