The way how Pan-Net is improving the throughput of Compute Nodes
Oct 13, 2020 · 457 words · 3 minute read
Pan-Net is using Openstack CMS (Cloud Management System) with OVS (Open vSwitch) as SDN (Software Defined Network) solution. OVS is configured and managed by Neutron (OS project used for networking).
Three main OVS components are:
- ovs-switchd: the main process of OVS, used to make a decision about packet/frame forwarding, deployed in userspace.
- kernel module (datapath): forwarding packets/frames, deployed in the kernel.
- ovsdb-server: DB used to store OVS configuration, deployed in userspace.
OVS supports many features provided by traditional switches:
- VLAN (802.1Q)
- LACP and static bonding
- tunneling protocols (GRE, VxLAN, Geneve, IPSec, etc.)
- QoS
- monitoring functions (NetFlow, SPAN, RSPAN, etc.)
- multicast snooping
- and so on …
The biggest advantage of OVS is automated control using Openflow.
The OVS solution is CPU intensive, affecting system performance and preventing fully utilizing available bandwidth. To improve switching capability many new technologies are invented (here we will note the most important and well known on the market):
technology | speed up(comparing to OVS) | deployment | standard | VM access | dependencies | comment | maturity |
---|---|---|---|---|---|---|---|
OVS DPDK | 2-4 | Host user space | YES | virtIO | OVS | OVS DPDK must use NIC driver | very mature |
XVIO | 20 | APP in host user space requires additional vCPUs | NO Netronome | virtIO | OVS offload SR-IOV on NIC OVS kernel | vendor lock DPDK based application uses 8 vCPUs | not very agile in development |
SR-IOV | 30 | VM and host NIC | YES | NIC driver | NIC vendors | lack of vSwitch policies only for static environment | very mature |
vSwitch offload | N/A | host NIC | not yet | N/A | another acceleration technology | always should come as support (e.g. to SR-IOV) | moderate proprietary |
vDPA DPDK | N/A similar as OVS-DPDK | NIC and host | should be | virtIO | pushed by RH and Mellanox CPU use | early stage | |
vDPA kernel | 30 similar to SR-IOV | NIC control part in host | should be | virtIO | OVS offload | pushed by RH and Mellanox preferred vDPA solution | early stage |
Pan-Net is, as of today, is using Netronome XVIO technology with a strategy to move to Mellanox supported SR-IOV with ASAP. ASAP (Accelerated Switch and Packet Processing) is the way how Mellanox is doing OVS offload.
OVS offload is a way to move forwarding plane from CPU intensive OVS kernel module to hardware resources on the NIC (Network Interface Card). Usually, NICs that supports OVS offloading are called SmartNICs.
Simplified OVS architecture with offloading components is depicted in the figure below.
Figure 1: Simplified OVS offload architecture
The following figures presents frames flow when SR-IOV with ASAP is used:
Figure 2: Frame flow from VM to NIC
Figure 3: Frame flow from NIC to VM
The Pan-Net long term strategy is to use vDPA in the kernel which, from the current perspective, seems to be the best future proof solution. For now, vDPA is not mature enough to be seriously considered for production deployment.