Using the Linux tbf qdisc for rate limiting on local or loopback interfaces

If you have ever played with the Linux tbf (Token Bucket Filter) on either some local interfaces, or on the loopback interface (lo) then you may have run into problems - like the attained rate is only a few hundred kilobits/s or less (zero)....?

tc qdisc add dev lo root tbf rate 10Mbit burst 10kb latency 5ms 

Basically if your interface has TSO/GSO enabled (check using ethtool -k ethX), or you're using the loopback interface - then you'll probably hit a problem. It turns out that the loopback interface has GSO/TSO enabled as default, plus since it is a software interface its default mtu is 16384 (as compared to 1500 for normal Ethernet interface). This matters as the tbf queue checks the size of the incoming 'packets' - which in the case of GSO/TSO are much larger than a normal on-the-wire packet - instead they're up to 9 x iface's mtu. So for normal interfaces it's about 12K, but for loopback it is about 100k!

In which case you'll need to add the 'mtu' argument to the tc command and then it might work( but check update below in case not):

tc q a dev eth0 root tbf rate 10Mbit burst 10kb latency 5ms mtu 100000

Update [29jan20]: It is also important that the burst parameter is greater than the MTU (you will see warnings in dmesg to this effect if your burst size if too small) if running on an interface that has a large MTU like the loopback interface which is now commonly set to 65535 :

tc q a dev eth0 root tbf rate 10Mbit burst 70kb latency 5ms mtu 100000