Setting QoS Configuration on Mellanox Connect X-6

After managing to set the maximum performance of the two hosts, and seeing the difference of performance with multiple VFs, we now started to configure QoS and see the performance difference.

1. Baseline Performance.

Before configuring QoS, it was vital to measure the peformance beforehand.

2. Setting up QoS

Setting up QoS was mainly done using the mlnx_qos linux tool from the MLNX_OFED driver.

In order to reinstall the driver, the Kernel needed to be downgraded as the host: as done here

#1. Check current Kernel Version
uname -r

#2. install new kernel
sudo apt-get install aptitude
sudo aptitude search linux-image
sudo aptitude install linux-image-4.15.0-20-generic

sudo vim /etc/default/grub
#cross-out
#GRUB_TIMEOUT_STYLE=hidden
#GRUB_TIMEOUT=0
sudo update-grub

#3. Delete ex-kernel
sudo dpkg -l | grep linux-
sudo apt-get purge linux-headers-5.4.0-84 linux-headers-5.4.0-84-generic
sudo apt-get purge linux-image-5.4.0-84 linux-modules-5.4.0-84-generic
reboot

#4. Check new Kernel Version
uname -r

Then, I reinstalled driver via

# Download Driver
wget https://content.mellanox.com/ofed/MLNX_EN-5.8-1.1.2.1/mlnx-en-5.8-1.1.2.1-ubuntu18.04-x86_64.iso

#Mount Driver
sudo mkdir /mnt/iso
sudo mount -o loop /share/MLNX_OFED_LINUX-4.2-1.0.0.0-ubuntu16.04-x86_64.iso /mnt/iso
cd /mnt/iso

#Install Driver (--guest not supported on Connect X-5)
sudo ./install --dpdk --upstream-libs --without-neohost-backend --force

#reboot
sudo /etc/init.d/openibd restart
reboot

3. Performance Difference