The native VMware vSwitch and Distributed vSwitch do not use MAC-learning. This was removed because the vSwitches would be aware of the VMs attached to them and the MAC addresses in use. As a result, if you nest ESXi under a standard vSwitch and power-on VMs under the nested instance, those VMs will be unable to communicate because their MACs are masked by the virtual host and the vSwitch is not aware of them.
Workaround options:
- Enable Promiscuous mode on the vSwitch.
- This works but should never be used in production. It adds a lot of unnecessary traffic and work to the physical NICs. It makes troubleshooting difficult and is a security risk
- Attach your virtual hosts to a Cisco Nexus 1000V.
- The 1000V retains MAC-learning, so VMs on nested virtual ESXi hosts can successfully communicate because the switch learns the nested MAC addresses.
- If your physical servers support virtual interfaces, you can create additional “physical” interfaces and pass them through to the virtual instances. This allows you to place the virtual hosts on the same switch as the physical hosts if you choose. There is obviously a finite amount of virtual interfaces you can create in the service profile, but I think this is a clean, low-overhead solution for environments using Cisco UCS or HP C7000 or similar.
Conclusion
The Nexus 1000V brings back important functionality for nested ESXi environments, especially those environments that do not have access to features like virtual interfaces and service profiles.
Helpful links:
Standing Up The Cisco Nexus 1000v In Less Than 10 Minutes by Kendrick Coleman
Brian Ragazzi 