What are Jumbo frames?
When I do vSphere training sessions, many times the attendees are not clear about the concept of Jumbo frames and there seems to be some confusion about it. So today’s article focuses on just that. We will be starting with understanding jumbo frames and later exploring how to enable it in a vSphere infrastructure.
Typically data is exchanged between a client and server using TCP/IP. When a sender (client or server) sends an IP datagram which is larger than the MTU (maximum transmission unit)size, the datagram is broken up into smaller chunks (fragments) of data. The process of breaking up the IP datagram into smaller fragments is known as fragmentation. At the receiver end the IP fragments need to be re-assembled as they are received.
The MTU is responsible for specifying how large each IP fragment can be. Smaller the MTU more fragments would be created, thus increasing the time taken for fragmentation and re-assembly. A smaller MTU would thus, also reduce the available effective bandwidth (pipe size) over a given network (point-to-point) connection.
Hence, if the MTU size is large it would not only reduce the IP datagram fragmentation/re-assembly time but also would increase the available effective bandwidth. Such large MTU sizes are known as Jumbo frames. Although larger MTU sizes are possible, usually Jumbo frames are configured with 9000 MTU.
Since large MTU size improves the bandwidth available, Jumbo frames are commonly used and recommended for accessing IP based Storage devices (NFS/iSCSI).
How to enable Jumbo frames in vSphere?
It important to note that Jumbo frames are good only if your entire path is configured with jumbo frames. Even if one device in-the-path is not capable of doing Jumbo frames, the entire path with fallback to the lowest MTU size supported by any device in the network path. Also its equally important to note that if there are some network devices not capable of doing Jumbo frames while some are configured to use Jumbo frames the effective bandwidth would be sub-optimal. In fact it might be less than if configured with the lowest possible MTU in the path.
In the any setup all devices including the endpoints (sender/receiver) and in-between network devices (switches, routers) should be configured with Jumbo frames to get the benefit of improved bandwidth.
On vSphere ESXi 5.0, one can get the configured MTU size using the command
esxcli network ip interface list
OR
esxcfg-vmknic -l
One can also use the vSphere Client to get the MTU details:
In the vSphere client, select your host and select the properties for your vSwitch from
Configuration Tab–>Networking–>vSwitch Properties
In the same way you can update the MTU value to 9000 (for Jumbo frames) either using the command line or the vSphere client. Again important to remember, that you will need to configure it not just for the VMKernel Port but also on the vSwitch. Secondly ensure that you have configured all the network devices in the network path with Jumbo frame including the storage device. Only then will will you benefit from increased bandwidth using Jumbo frames.
I believe just enabling Jumbo frames on the network interface of your storage device should be sufficient, however you may want to check with your storage vendor if any additional configuration is required to enable Jumbo frames.
On vSphere you would not need to do any additional configurations for VMs.
That’s it folks.
.
