Use Cisco Nexus 1000V for virtual hosts in nested ESXi

The native VMware vSwitch and Distributed vSwitch do not use MAC-learning. This was removed because the vSwitches would be aware of the VMs attached to them and the MAC addresses in use. As a result, if you nest ESXi under a standard vSwitch and power-on VMs under the nested instance, those VMs will be unable to communicate because their MACs are masked by the virtual host and the vSwitch is not aware of them.

Workaround options:

1. Enable Promiscuous mode on the vSwitch.
2. This works but should never be used in production.  It adds a lot of unnecessary traffic and work to the physical NICs.  It makes troubleshooting difficult and is a security risk
3. Attach your virtual hosts to a Cisco Nexus 1000V.
4. The 1000V retains MAC-learning, so VMs on nested virtual ESXi hosts can successfully communicate because the switch learns the nested MAC addresses.
5. If your physical servers support virtual interfaces, you can create additional “physical” interfaces and pass them through to the virtual instances.  This allows you to place the virtual hosts on the same switch as the physical hosts if you choose.  There is obviously a finite amount of virtual interfaces you can create in the service profile, but I think this is a clean, low-overhead solution for environments using Cisco UCS or HP C7000 or similar.

Conclusion

The Nexus 1000V brings back important functionality for nested ESXi environments, especially those environments that do not have access to features like virtual interfaces and service profiles.

Helpful links:

Standing Up The Cisco Nexus 1000v In Less Than 10 Minutes by Kendrick Coleman

Advertisement

vCloud Suite 5.1 upgrade suggestions

This information will probably be outdated quickly, but for now (9/14/2012), here are my suggestions and tips concerning vCloud Suite 5.1

• If you plan to use Microsoft SQL Server for your SSO database, update the service configuration to listen on a static port.  vCenter Server and VUM use the .NET native SQL client, so they can deal with dynamic TCP ports on the SQL service, but SSO uses the Java SQL driver, which will require a static port (TCP 1433 by default)
• Many administrators have had problems logging into their vCenter Server after installing SSO and upgrading vCenter Server to 5.1.  It appears that if the vCenter Server is a member of the domain, it may still try to use local credentials as the initial/default SSO source.  If your vCenter Server is a member of an Active Directory domain, be sure to locate/create a local admin account before upgrading vCenter Server to 5.1 so you can log on if you are affected by this issue
• If you are running EMC Powerpath/VE on your ESXi hosts, do NOT upgrade to ESXi 5.1 yet.  The version of Powerpath/VE that works with ESXi 5.1 has not yet been released.
• Despite the naming similarities, VMware View 5.1 is not compatible with vSphere 5.1.  Do not yet upgrade to vSphere 5.1 in your View environment.
• The vCloud API for vCloud Director 5.1 is different from the API for vCloud Director 1.5.1.  If you have developed anything that consumes the vCloud API, you will probably have to make significant changes.  If you have vCO packages/workflows that use the vCloud API, you may be affected and should upgrade vCO and vCloud in a lab/test environment where you can update the objects appropriately first.
• There is a new version of the Cisco Nexus 1000V for vSphere 5.1 that should be factored into your upgrade plan if your environment uses that component.
• In vCenter Server 2.5-4.0, host passwords are encrypted in the database using an encryption key of 512-to-1024 bits. However, in 5.1, the encryption key must be 2048 bits. If it is not, the vCenter Services may not start. Check this KB article to see how to check and correct this issue.