Think you’ll be able to use Link Aggregation Groups (LAG) to turn your four Gigabit links into a single 4 Gigabit link? Well, Link Aggregation might not do the job you think it will do. Listen, I get it. There are a ton of topics that you have to know about to be an effective Virtualization solution designer, and network intricacies might seem pretty far down the list. So here’s the deal.
Link Aggregation, when discussed in a virtualization context, is a process of configuring multiple NIC ports to be able to send and receive traffic for a given network configuration of a vSwitch. To the virtualization host, this means the vSwitch can be sent packets on any of the pNIC ports in the aggregation (when the destination is one of the vNICs connected to the vSwitch). The process of outbound packets is a bit different. Currently, the ESXi hypervisor (5.x) creates a hash of the origin and destination addresses and chooses one of the outbound pNICs based on that hash. Yes, one of the pNICs. It doesn’t alternate or load balance between the pNICs for any single origin-destination pair.
At the same time, the network device on the other side of the Link Aggregation (usually a physical switch) needs to make the same configuration as far as accepting packets from the aggregated link. When it sends packets down the aggregated link, it has it’s own rule about which connection it uses. Typically, this is a hash of MAC or IP addresses (Layer 2 or 3). Again, this process typically chooses a single link for traffic involving a given origin-destination pair. Again, no load balancing or bandwidth aggregation for a single origin-destination pair.
What are the implications? Well, vSphere doesn’t support a round-robin use of pNICs in Link Aggregation on the broadcast side, so two or more aggregated links won’t behave as a linear addition of the ports. In fact, for any given origin-destination pair, the host will max out at a single NIC’s worth of (outbound) bandwidth. If you only ever have a single origin-destination pair, then you’d get no bandwidth benefit from link aggregation.
What if there were many endpoints connecting to the vSphere host across a link aggregation? Well, the more pairs, the better chance that the connections are evenly distributed across the ports. In that case, there’s a potential bandwidth benefit.
That’s quite different from a simple addition of the two or more ports together.
image sources
- 20131016_155815: Owned by the author
