Category Archives: VMware

Quick Fix: vCloud Director SP None of the Cells have a vCenter Proxy Service Running. SSL Protocol Fix

vCloud Director SP 8.20 was released a few weeks ago and I wanted to highlight an issue I ran into while testing of the BETA. I hadn’t come across this issue in previous versions of vCD and even though it relates to the fact I had a vCenter 5.5 I thought it worth a post now that 8.20 has GA’ed.

After I upgraded my cells I got the fairly common error message under the Cloud Cells section of the Manage & Monitor menu telling me that I didn’t have a vCenter Proxy service running. It’s something all vCD administrators would have seen over the years, so I did the usual troubleshooting step of going of reconnecting the vCenter under vSphere Resources. This didn’t work, so I did what comes naturally and cleared the Quartz Tables in the vCD database without any success.

Failed to connect to the vCenter. Please check if this is a valid vCenter server and the credentials are correct.

The NestedESXi lab was running vCenter 5.5 U3b and after a bit of searching I came across a post in the vCloud BETA forums relating to this issue:

Starting with VDC 8.20, the SSL protocol ‘TLSv1’ is no longer supported by default in the product for security reasons (as a server to serve the REST API request, but also as a client when talking to vCenter).
The version of vCenter you are running (please confirm which version), is older and probably only supports TLSv1.

Which explains the errors I also had been observing. Note that from 5.5 Update 3e and 6.0 Update 3 and later TLS v1.0 has been disabled and should be disabled.

Due to security concerns in the TLSv1.0 protocol, both Payment Card Industry (PCI) and BSI organizations have suggested to implement and enable TLSv1.1 or TLSv1.2, and move away from the use of TLSv1.0 as soon as possible

Even though it’s not suggested I needed to enable TLS v1 so that vCD SP 8.20 could connect to the vCenter. The following steps where done to enable TLSv1 which was based off this VMwareKB outlining why cells no longer enable SSL v3 by default and talks about a cell management tool command that configures the allowed SSL Protocols vCD uses during the handshake process with vCenter.

The SSL V3 protocol has serious vulnerability, described in CVE-2014-3566. As of vCloud Director 5.5.3, cells no longer enable SSL V3 by default for internal and external HTTPS connections. The vCloud Director cell management tool has been updated with a new subcommand that enables the system administrator to configure the set of SSL protocols that the cell offers to use during the SSL handshake process. This new subcommand has been made available in vCloud Director 5.5.3

Run the following command on the vCD cell in /opt/vmware/vcloud/bin/

./cell-management-tool ssl-protocols -d SSLv3,SSLv2Hello

After that is done restart the cell and check to make sure you have a listener and that vCenter is connected. If you run the ssl-protocols command with a -l flag it will show you what ssl-protocols are allowed. By default you should now only have TLS v1.1 and 1.2 enabled, but in my case I also needed v1.

Finally, it’s worth repeating that TLS v1 shouldn’t be used in production, but if you are still running older versions of 5.5 and 6.0 in your labs then this will help.

References:

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2112282

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2145796

Released: vCenter and ESXi 6.0 Update 3 – What’s in It for Service Providers

Last month I wrote a blog post on upgrading vCenter 5.5 to 6.0 Update 2 and during the course of writing that blog post I conducted a survey on which version of vSphere most people where seeing out in the wild…overwhelmingly vSphere 6.0 was the most popular version with 5.5 second and 6.5 lagging in adoption for the moment. It’s safe to assume that vCenter 6.0 and ESXi 6.0 will be common deployments for some time in brownfield sites and with the release of Update 3 for vCenter and ESXi I thought it would be good to again highlight some of the best features and enhancements as I see them from a Service Provider point of view.

vCenter 6.0 Update 3 (Build 5112506)

This is actually the eighth build release of vCenter 6.0 and includes updated TLS support for v1.0 1.1 and 1.2 which is worth a look in terms of what it means for other VMware products as it could impact connectivity…I know that vCloud Director SP now expects TLSv 1.1 by default as an example. Other things listed in the What’s New include support for MSSQL 2012 SP3, updated M2VCSA support, timezone updates and some changes to the resource allocation for the platform services controller.

Looking through the Resolved Issue there are a number of networking related fixes in the release plus a few annoying problems relating to vMotion. The ones below are the main ones that could impact on Service Provider operations.

  • Upgrading vCenter Server from version 6.0.0b to 6.0.x might fail. 
    Attempts to upgrade vCenter Server from version 6.0.0b to 6.0.x might fail. This issue occurs while starting service An error message similar to the following is displayed in the run-updateboot-scripts.log file.
    “Installation of component VCSServiceManager failed with error code ‘1603’”
  • Managing legacy ESXi from the vCenter Server with TLSv1.0 disabled is impacted.
    vCenter Server with TLSv1.0 disabled supports management of legacy ESXi versions in 5.5.x and 6.0.x. ESXi 5.5 P08 and ESXi 6.0 P02 onwards is supported for 5.5.x and 6.0.x respectively.
  • x-VC operations involving legacy ESXi 5.5 host succeeds.
    x-VC operations involving legacy ESXi 5.5 host succeeds. Cold relocate and clone have been implicitly allowed for ESXi 5.5 host.
  • Unable to use End Vmware Tools install option using vSphere Client.
    Unable to use End VMware Tools install option while installing VMware Tools using vSphere Client. This issue occurs after upgrading to vCenter Server 6.0 Update 1.
  • Enhanced vMotion fails to move the vApp.VmConfigInfo property to destination vCenter Server.
    Enhanced vMotion fails to move the vApp.VmConfigInfo property to destination vCenter Server although virtual machine migration is successful.
  • Storage vMotion fails if the VM is connected with a CD ISO file.
    If the VM is connected with a CD ISO file, Storage vMotion fails with an error similar to the following:
  • Unregistering an extension does not delete agencies created by a solution plug-in.
    The agencies or agents created by a solution such as NSX, or any other solution which uses EAM is not deleted from the database when the solution is unregistered as an extension in vCenter Server.

ESXi 6.0 Update 3 (Build 5050593)

The what’s new in ESXi is a lot more exciting than what’s new with vCenter highlighted by a new Host Client and fairly significant improvements in vSAN performance along with similar TLS changes that are included in the vCenter update 3. With regards to the Host Client the version is now 1.14.0. and includes bug fixes and brings it closer to the functionality provided by the vSphere Client. It’s also worth mentioning that new versions of the Host Client continue to be released through the VMware Labs Flings site. but, those versions are not officially supported and not recommended for production environments.

For vSAN, multiple fixes have been introduced to optimize I/O path for improved vSAN performance in All Flash and Hybrid configurations and there is a seperate VMwareKB that address the fixes here.

  • More Logs Much less Space vSAN now has efficient log management strategies that allows more logging to be packed per byte of storage. This prevents the log from reaching its assigned limit too fast and too frequently. It also provides enough time for vSAN to process the log entries before it reaches it’s assigned limit thereby avoiding unnecessary I/O operations
  • Pre-emptive de-staging vSAN has built in algorithms that de-stages data on periodic basis. The de-staging operations coupled with efficient log management significantly improves performance for large file deletes including performance for write intensive workloads
  • Checksum  Improvements vSAN has several enhancements that made the checksum code path more efficient. These changes are expected to be extremely beneficial and make a significant impact on all flash configurations, as there is no additional read cache look up. These enhancements are expected to provide significant performance benefits for both sequential and random workloads.

As with vCenter, I’ve gone through and picked out the most significant bug fixes as they relate to Service Providers. The first one listed below is important to think about as it should significantly reduce the number of failures that people have been seeing with ESXi installed on SD-Flash Card and not just for VDI environments as the release notes suggest.

  • High read load of VMware Tools ISO images might cause corruption of flash media  In VDI environment, the high read load of the VMware Tools images can result in corruption of the flash media.
    You can copy all the VMware Tools data into its own ramdisk. As a result, the data can be read from the flash media only once per boot. All other reads will go to the ramdisk. vCenter Server Agent (vpxa) accesses this data through the /vmimages directory which has symlinks that point to productLocker.
  • ESXi 6.x hosts stop responding after running for 85 days
    When this problem occurs, the /var/log/vmkernel log file displays entries similar to the followingARP request packets might drop.
  • ARP request packets between two VMs might be dropped if one VM is configured with guest VLAN tagging and the other VM is configured with virtual switch VLAN tagging, and VLAN offload is turned off on the VMs.
  • Physical switch flooded with RARP packets when using Citrix VDI PXE boot
    When you boot a virtual machine for Citrix VDI, the physical switch is flooded with RARP packets (over 1000) which might cause network connections to drop and a momentary outage. This release provides an advanced option /Net/NetSendRARPOnPortEnablement. You need to set the value for /Net/NetSendRARPOnPortEnablementto 0 to resolve this issue.
  • Snapshot creation task cancellation for Virtual Volumes might result in data loss
    Attempts to cancel snapshot creation for a VM whose VMDKs are on Virtual Volumes datastores might result in virtual disks not getting rolled back properly and consequent data loss. This situation occurs when a VM has multiple VMDKs with the same name and these come from different Virtual Volumes datastores.
  • VMDK does not roll back properly when snapshot creation fails for Virtual Volumes VMs
    When snapshot creation attempts for a Virtual Volumes VM fail, the VMDK is tied to an incorrect data Virtual Volume. The issue occurs only when the VMDK for the Virtual Volumes VM comes from multiple Virtual Volumes datastores.
  • ESXi host fails with a purple diagnostic screen due to path claiming conflicts
    An ESXi host displays a purple diagnostic screen when it encounters a device that is registered, but whose paths are claimed by a two multipath plugins, for example EMC PowerPath and the Native Multipathing Plugin (NMP). This type of conflict occurs when a plugin claim rule fails to claim the path and NMP claims the path by default. NMP tries to register the device but because the device is already registered by the other plugin, a race condition occurs and triggers an ESXi host failure.
  • ESXi host fails with a purple diagnostic screen due to path claiming conflicts
    An ESXi host displays a purple diagnostic screen when it encounters a device that is registered, but whose paths are claimed by a two multipath plugins, for example EMC PowerPath and the Native Multipathing Plugin (NMP). This type of conflict occurs when a plugin claim rule fails to claim the path and NMP claims the path by default. NMP tries to register the device but because the device is already registered by the other plugin, a race condition occurs and triggers an ESXi host failure.
  • ESXi host fails to rejoin VMware Virtual SAN cluster after a reboot
    Attempts to rejoin the VMware Virtual SAN cluster manually after a reboot might fail with the following error:
    Failed to join the host in VSAN cluster (Failed to start vsantraced (return code 2)
  • Virtual SAN Disk Rebalance task halts at 5% for more than 24 hours
    The Virtual SAN Health Service reports Virtual SAN Disk Balance warnings in the vSphere Web Client. When you click Rebalance disks, the task appears to halt at 5% for more than 24 hours.

It’s also worth reading through the Known Issues section as there is a fair bit to be aware of especially if running NFS 4.1 and worth looking through the general storage issues.

Happy upgrading!

References:

http://pubs.vmware.com/Release_Notes/en/vsphere/60/vsphere-vcenter-server-60u3-release-notes.html

http://pubs.vmware.com/Release_Notes/en/vsphere/60/vsphere-esxi-60u3-release-notes.html

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2149127

Released: vCloud Director SP 8.20 with HTML5 Goodness!

This week, VMware released vCloud Director SP version 8.20 (build 5070630) which marks the 8th Major Release for vCloud Director since 1.0 was released in 2010. Ever since 2010 the user interface give or take a few minor modifications and additions has been the same. It also required flash and java which has been a pain point for a long time and in someways unfairly contributed towards a negative perception around vCD on a whole.  It’s been a long time coming but vCloud Director finally has a new web UI built on HTML5 however this new UI is only exposed when accessing the new NSX integration which is by far and away the biggest addition in this release.

This NSX integration has been in the works for a while now and has gone through a couple of iterations within the vCloud product team. Initially announced as Advanced Networking Services which was a decoupled implementation of NSX integration we now have a fully integrated solution that’s part of the vCloud Director installer. And while the UI additions only extend to NSX for the moment it’s brilliant to see what the development team have done with the Clarity UI (tbc). I’m going to take a closer look at the new NSX features in another post, but for the moment here are the release highlights of vCD SP 8.20.

New Features:

  • Advanced Edge Gateway and Distributed Firewall Configuration – This release introduces the vCloud Director Tenant Portal with an initial set of controls that you can use to configure Edge Gateways and NSX Distributed Firewalls in your organization.
  • New vCloud Director API for NSX – There is a new a proxy API that enables vCloud API clients to make requests to the NSX API. The vCloud Director API for NSX is designed to address NSX objects within the scope of a vCloud Director tenant organization.
  • Role Administration at the Organization Level – From this release role objects exist in each organization. System administrators can use the vCloud Director Web Console or the vCloud API to create roles in any organization. Organization administrators can use the vCloud API to create roles that are local to their organization.
  • Automatic Discovery and Import of vCenter VMs – Organization VDCs automatically discover vCenter VMs that exist in any resource pool that backs the vDC. A system administrator can use the vCloud API to specify vCetner resource pools for the vDC to adopt. vCenter VMs that exist in an adopted resource pool become available as discovered vApps in the new vDC.
  • Virtual Machine Host Affinity – A system administrator can create groups of VMs in a resource pool, then use VM-Host affinity rules to specify whether members of a VM group should be deployed on members of a vSphere host DRS Group.
  • Multi-Cell Upgrade – The upgrade utility now supports upgrading all the cells in your server group with a single operation.

You can see above that this release has some major new features that are more focused on tenant usability and allow more granular and segmented controls of networks, user access and VM discovery. The Automatic VM discovery and Import is a significant feature that goes along with the 8.10 feature of live VM imports and helps administrators import VM work loads into vCD from vCenter.

“VMware vCloud Director 8.20 is a significant release that adds enhanced functionality.  Fully integrating VMware NSX into the platform allows edge gateways and distributed firewalls to be easily configured via the new HTML5 interface.  Additional enhancements such as seamless cell upgrades and vCenter mapping illustrate VMware is committed to the platform and to vCloud Air Network partners.”

A list of known issues can be found in the release notes and i’d like to highlight the note around Virtual Machine memory for the vCD Cells…I had my NestedESXi lab instances crash due to memory pressures due to the fact the VMs where configured with only 5GB of RAM. vCloud Director SP 8.20 needs at least 6GB so ensure your cells are modified before you upgrade.

Well done the the vCloud Director Product and Development team for this significant release and I’ll look to dig into some of the new feature in detail in upcoming posts. You can also read the offical vCloud Blog release post here. I’m looking forward to what’s coming in the next release now…hopefully more functionality placed into the HTML5 UI and maybe integration with VMwareonAWS 😉

References:

http://pubs.vmware.com/Release_Notes/en/vcd/8-20/rel_notes_vcloud_director_8-20.html

https://www.vmware.com/support/pubs/vcd_sp_pubs.html

https://blogs.vmware.com/vcloud/2017/02/vmware-announces-general-availability-vcloud-director-8-20.html

vExpert’s of 2017 – Listen Up! It’s about the Advocacy

Overnight Cory Romero announced the intake of the 2017 VMware vExperts. As a now six time returning vExpert it would be easy for me to sit back enjoy a perceived sense of entitlement that comes with being a vExpert…but times have changed. The award has changed and the way people feel about the program has changed…when I first become a vExpert back in 2012 there was approximately 300 world wide…fast forward to 2017 and there are now 1463 give or take which is an increase of about 100 from 2016.

Over the past few years there are always comments and questions around the swelling of the numbers and how there should be a more stringent approval and acceptance structure. I myself shared those thoughts in previous posts…however my opinions around this have changed mainly because I have come to understand what the vExpert program (and other vendor programs) are all about and where myself, and VMware can achieve maximum value.

The vExpert program is designed to aid in your success and help amplify your internal and or external personal brands and channels. So whether you are a external evangelist or a internal champion we want to be sure you have the resources needed for the program so you can be more successful. Make no mistake that this program exists to help VMware push it’s products and services through the advocacy of the people in the group. The reward is given to those who in previous 12 months have shown themselves to be active in that advocacy. That doesn’t always mean that you need to be an active blogger or present at events, but it does mean that in your day to day role within the IT Industry you should be championing VMware as a company and break that down to champion VMware products that you use or sell.

This doesn’t mean that you can’t be involved in looking at and advocating other vendor technologies (many others hold multiple program memberships) but as Corey mentioned, the criteria used to have achieved the award implies that those activities need to be VMware focused.

Once you have the title it’s important to understand that there is a responsibility associated with it…it’s not just about the free gear though as I have stated before you should accept that as a perk of being part of the program and you shouldn’t feel like a “vendor whore” for accepting that shirt or coffee mug. Going back to responsibility, what I mean by that is that you should wear the badge proudly…understand that you have taken the time to apply/reapply for the award because you believed yourself worth of filling the selection criteria and use the award as a stepping stone to improve on the activities that got you there the year before.

Don’t rest on your laurels and expect the award to come to you every year…the vExpert team put a lot load of effort into keeping the program running and as a group we get significant exposure and opportunity from VMware and their partners…make it count and don’t waste it! Make sure you engage with others in the community through Twitter, LinkedIn or the Slack vExpert Channel or get down to your local VMUG or VMware event and engage directly.

NOTE: Content First Posted in 2016

NSX Bytes: NSX-v 6.3 Host Preparation Fails with Agent VIB module not installed

NSX-v 6.3 was released last week with an impressive list of new enhancements and I wasted no time in looking to upgrades my NestedESXi lab instance from 6.2.5 to 6.3 however I ran into an issue that at first I thought was related to a previous VIB upgrade issue caused by VMware Update Manager not being available during NSX Host upgrades…in this case it presented with the same error message in the vCenter Events view:

VIB module for agent is not installed on host <hostname> (_VCNS_xxx_Cluster_VMware Network Fabri)

After ensuring that my Update Manager was in a good state I was left scratching my head…that was until some back and forth in the vExpert Slack #NSX channel relating to a new VMwareKB that was released the same day as NSX-v 6.3.

https://kb.vmware.com/kb/2053782

This issue occurs if vSphere Update Manager (VUM) is unavailable. EAM depends on VUM to approve the installation or uninstallation of VIBs to and from the ESXi host.

Even though my Upgrade Manager was available I was not able to upgrade through Host Preparation. It seem’s like vSphere 6.x instances might be impacted by this bug but the good news is there is a relatively easy workaround as mentioned in the VMwareKB that bypasses the VUM install mechanism. To enable the workaround you need to enter into the Managed Object Browser of the vCenter EAM by going to the following URL and entering in vCenter admin credentials.

https://vCenter_Server_IP/eam/mob/ 

Once logged in you are presented with a (or list of) agencies. In my case I had more than one, but I selected the first one in the list which was agency-11

The value that needs to be changed is the bypassVumEnabled boolean value as shown below.

To set that flag to True enter in the following URL:

https://vCenter_Server_IP/eam/mob/?moid=agency-x&method=Update

Making sure that the agency number matches your vCenter EAM instance. From there you need to change the existing configuration for that value by removing all the text in the value box and invoking the value listed below:

Once invoked you should be able to go back into the Web Client and click on Resolve under the Cluster name in the Host Preparation Tab of the NSX Installation window.

Once done I was in an all Green state and all hosts where upgraded to 6.3.0.5007049. Once all hosts have been upgraded it might be a useful idea to reverse the workaround and wait for an official fix from VMware.

References:

https://kb.vmware.com/kb/2053782

NSX Bytes: NSX for vSphere 6.3 and NSX-T 1.1 Release Information

VMware’s NSX has been in the wild for almost three years and while the initial adoption was slow, of recent times there has been a calculated push to make NSX more mainstream. The change in licensing that happened last year has not only been done to help drive adoption by traditional VMware customers running vSphere that previously couldn’t look at NSX due to price but also the Transformers project has looked to build on Nicira’s roots in the heterogeneous hypervisor market and offer network virutalization beyond vSphere and beyond Open source platforms and into the public cloud space. The vision for VMware with NSX is to manage security and connectivity for heterogeneous end points through:

  • Security
  • Automation
  • Application Continuity

NSX has seen significant growth for VMware over the past twelve to eighteen months driven mostly from customer demand focusing around micro-segmentation, IT automation and efficiency and also the need to have extended multiple data centre locations that can be pooled together. To highlight the potential that remains with NSX-v less that 5% of the total available vSphere install base has NSX-v installed…and while that could have something to do with the initial restrictions and cost of the software it still represents enormous opportunity for VMware and their partners.

Last week the NSX vExpert group was given a first look at what’s coming in the new releases…below is a summation of what to expect from both NSX-v 6.3 and NSX-T 1.1. Note that we where not given an indication on vSphere 6.5 support so, like the rest of you we are all waiting for the offical release notes.

[Update] vSphere 6.5 will be supported with NSX-v 6.3

Please note that VMware vSphere 6.5a is the minimum supported version with NSX for vSphere 6.3.0. For the most up-to-date information, see the VMware Product Interoperability Matrix. Also, see 2148841.

NSX for vSphere 6.3 Enhancements:

Security:

  • NSX Pre-Assessment Tool based on vRealize Network Insight
  • Micro-Segmentation Planning and application visibility
  • New Security Certifications around ICSA, FIPS, Common Criteria and STIG
  • Linux Guest VM Introspection
  • Increase performance in service chaining
  • Larger scalability of VDI up to 50K desktops
  • NSX IDFW for VDI
  • Active Directory Integration for VDI at scale

Automation:

  • Routing Enhancements
  • Centralized Dashboard for service and ops
  • Reduced Upgrade windows with rebootless upgrades
  • Integration with vRA 7.2 enhancing LB,NAT
  • vCloud Director 8.20 support with advanced routing, DFW, VPN
  • VIO Updates to include multi-vc deployments
  • vSphere Integrated Container Support
  • New Automation Frameworks for PowerNSX, PyNSXv, vRO

Application Continuity:

  • Multi-DC deployments with Cross VC NSX enhancements for security tags
  • Operations enhancements with improved availability
  • L2VPN performance enhancements for cross DC/Cloud Connectivity

Where does NSX-T Fit:

Given there was some confusion about NSX-v vs. NSX-t in terms of everything going to a common code base starting from the transformers release it was highlighted that VMware’s primary focus for 2017 hasn’t shifted away from NSX for vSphere and will still be heavily invested in to add new capabilities in and beyond 6.3 and that there will be a robust roadmap of new capabilities in future releases with support extended will into the future.

NSX-t’s main drivers related to new data centre and cloud architectures with more hetrogeneality driving a different set of requirements to that of vSphere that focuses around multi-domain environments leading to a multi-hypervisor NSX platform. NSX-t is highly extensible and will address more endpoint heterogeneity in future releases including containers, public clouds and other hypervisors. As you can see before the existing use cases for NSX-t are mainly focused around devops, micro-segmentation and multi-tenant infrastructure.

NSX-T 1.1 Brief Overview:

Again the focus is around private IaaS and multi-hypervisor support for development teams using dev clouds and employing more devops methodologies. There isn’t too much to write home about in the 1.1.0 release but there is some extended hypervisor support for KVM and ESXi, more single or multi-tenant support and some performance and resiliency optimizations.

Conclusion:

There is a lot to like about where VMware is taking NSX and both product streams offer strong network virtualization capabilities for customers to take advantage of. There is no doubt in my mind that the release of NSX-v 6.3 will continue to build on the great foundation laid by the previous NSX versions. When the release notes are made available I will do take a deeper look into all the new features and enhancements and tie them into what’s most useful for service providers.

Upgrading Windows vCenter 5.5 to 6.0 In-Place: Issues and Fixes

Yes that’s not a typo…this post is focusing on upgrading Windows vCenter 5.5 to 6.0 via an in-place upgrade. There is the option to use the vSphere 6.0 Update2M build with the included Migrate to VCSA tool to achieve this and move away from Windows, but I thought it was worth documenting my experiences with a mature vCenter that’s at version 5.5 Update 2 and upgrade that to 6.0 Update 2. Eventually this vCenter will need to move off the current Windows 2008 RTM server which will bring into play the VCSA migration however for the moment it’s going to be upgraded to 6.0 on the same server.

With VMware releasing vSphere 6.5 in November there should be an increased desire for IT shops to start seriously thinking about moving on from there existing vSphere versions and upgrading to the latest 6.5 release however many people I know where still running vSphere 5.5, so the jump to 6.5 directly might not be possible due to internal policies or other business reasons. Interestingly in the rough numbers, I’ve got an active Twitter Poll out at the moment which after 100 votes shows that vSphere 5.5 makes up 53% of the most common vCenter version, followed by 6.0 with 44% and 6.5 with only 3%.

Upgrade Options:

You basically have two options to upgrade a Windows based 5.5 vCenter:

My approach for this particular environment (which is a NestedESXi lab environment) was to ensure a smooth upgrade to vSphere 6.0 Update 2 and then look to upgrade again to 6.5 once is thaws outs in the market. That said, I haven’t read too many issues with vSphere 6.5 and VMware have been excellent in ensuring that the 6.5 release was the most stable for years. The cautious approach will still be undertaken by many and a stepped upgrade to 6.5 and migration to the VCSA will be common place. For those that wish to move away from their Windows vCenter, there is nothing stopping you from going down the Migrate2VCSA path, and it is possible to migrate directly from 5.5 to 6.5.

Existing Component Versions:

  • vCenter 5.5 (2001466)
  • ESXi 5.5 (3116895)

SQL Version Requirements:

vCenter 6.0 Update 2 requires at least SQL Server 2008 R2 SP1 or higher, so if you are running anything lower than that you will need to upgrade to a later service pack or upgrade to later versions of SQL Server. For a list of all compatible databases click here.

vCenter Upgrade Pre-Upgrade Checks:

First step is to make sure you have a backup of the vCenter environment meaning VM state (Snapshot) and vCenter database backup. Once that’s done there are a few pre-requisites that need to be met and that will be checked by the upgrade process before the actual upgrade occurs. The first thing the installer will do after asking for the SSO and VC service account password is run the Pre-Upgrade Checker.

vCenter SSL and SSO SSL System Name Mismatch Error:

A common issue that may pop up from the pre-upgrade checker is the warning below talking about an issue with the system name of the vCenter Server certificate and the SSO certificate. As shown below it’s a hard stop and tells you to replace one or the other certificate so that the same system name is used.

If you have a publicly signed SSL Certificate you will need to generate a new cert request and submit that through the public authority of choice. The quickest way to achieve this for me was to generate a new self signed certificate by following the VMwareKB article here. Once that’s been generated you can replace the existing certificate by following a previous post I did using the VMware SSL Certificate Updater Tool.

After all that, in any case I got the warning below saying that the 5.5 SSL Certificates do not meet security requirements, and so new SSL certificates will need to be generated for vCenter Server 6.0.0.

With that, my suggestion would be to generate a temporary self signed certificate for the upgrade and then apply a public certificate after that’s completed.

Ephemeral TCP Port Error:

Once the SSL mismatch error has been sorted you can run the pre-upgrade checker again. Once that completes successfully you move onto the Configure Ports window. I ran into the error shown below that states that the range of port is too large and the system must be reconfigured to use a smaller ephemeral port range before the install can continue.

The fix is presented in the error message so after running netsh.exe int ipv4 set dynamicportrange tcp 49152 16384 you should be ok to hit Next again and continue the upgrade.

Export of 5.x Data:

During the upgrade the 5.5 data is stored in a directory and then migrated to 6.0. You need to ensure that you have enough room on the drive location to cater for your vCenter instance. While I haven’t seen any offical rules around the storage required, I would suggest having enough storage free and the size of your vCenter SQL database data file.

vCenter Upgrade:

Once you have worked through all the upgrade screens you are ready for upgrade. Confirm the settings, take note of the fact that once updated the vCenter will be in evaluation mode, meaning you need to apply a new vCenter 6.x license once completed, check the checkbox that states you have a backup of the vCenter machine and database and you should be good to go.

Depending on the size of you vCenter instance and the speed of your disks the upgrade can take anywhere from 30 to 60 minutes or longer. If at any time the upgrade process fails during the initial export of the 5.5 data a roll back via the installer is possible…however if there is an issue while 6.0 is being installed the likelihood is that you will need to recover from backups.

Post Upgrade Checks:

Apart from making sure that the upgrade has gone through smoothly by ensuring all core vCenter services are up and running, it’s important to check any VMware or third party services that where registered against the vCenter especially given that the SSL Certificate has been replaced a couple of times. Server applications like NSX-v, vCloud Director and vCO explicitly trust SSL certificates so the registration needs to be actioned again. Also if you are running Veeam Backup & Replication you will need to go through the setup process again to accept the new SSL Certificate otherwise your backup jobs will fail.

If everything has gone as expected you will have a functional vCenter 6.0 Update 2 instance and planning can now take place for the 6.5 upgrade and in my case…the migration from Windows to the VCSA.

References:

http://www.vmware.com/resources/compatibility/sim/interop_matrix.php#db&2=998

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1029944

 

First Look: ManageIQ vCloud Director Orchestration

Welcome to 2017! To kick off the year I thought I’d do a quick post on a little known product (at least in my circles) from Red Hat Inc called ManageIQ. I stumbled across ManageIQ by chance having caught wind that they where soon to have vCloud Director support added to the product. Reading through some of the history behind ManageIQ I found out that in December of 2012 Red Hat acquired ManageIQ and integrated it into its CloudForms cloud management program…they then made it open source in 2014.

ManageIQ is the open source project behind Red Hat CloudForms. The latest product features are implemented in the upstream community first, before eventually making it downstream into Red Hat CloudForms. This process is similar for all Red Hat products. For example, Fedora is the upstream project for Red Hat Enterprise Linux and follows the same upstream-first development model.

CloudForms is a cloud management platform that also manages traditional server virtualization products such as vSphere and oVirt. This broad capability makes it ideal as a hybrid cloud manager as its able to manage both public clouds and on-premises private clouds and virtual infrastructures. This acts as a single management interface into hybrid environments that enables cross platform orchestration to be achieved with relative ease. This is backed by a community that contributes workflows and code to the project.

The supported platforms are shown below.

The October release was the first iteration for the vCloud provider which supports authentication, inventory (including vApps), provisioning, power operations and events all done via the use of the API provided by vCloud Director. First and foremost I see this as a client facing tool rather than an internal orchestration tool for vCAN SPs however given it can go cross platform there can be a use for VM or Container orchestration that SPs could tap into.

While it’s still relatively immature compared to the other platforms it supports, I see great potential in this and I think all vCAN Service Providers running vCloud Director should look at this as a way for their customers to better consume and operate vCD coming from a more modern approach, rather than depending on the UI.

Adding vCloud Director as a Cloud Provider:

Once the Appliance is deployed, head to Compute and Add New Cloud Provider. From the Type dropdown select VMware vCloud

Depending on which version of vCD SP your Service Provider is running, select the appropriate API Version. For vCD SP 8.x it should be vCloud API 9.0

Next add in the URL of the vCloud Director endpoint with it’s port…which is generally 443. For the username, you use the convention of [email protected] which allows you to login specifically to your vCD Organization. If you want to login at an admin enter in [email protected] to get top level access.

Once connected you can add as many vCD endpoints as you have. As you can see below I am connected to four seperate instances of vCloud.

Clicking through you get a Summary of the vCloud Zone with it’s relationships.

Clicking on the Instances you get a list of your VM’s, but this also has views for Virtual Datacenter, vApps and other vCD objects. As you can see below there is detailed views on the VM and it does have basic Power functions in this build.

I’ve just started to look into the power of CloudForms and have been reading through the ManageIQ automation guide. It’s one of those things that needs a little research plus some trial and error to master, but I see this form of cloud consumption where the end user doesn’t have to directly manipulate the various API endpoints as the future. I’m looking forward to how the vCloud Director provider matures and I’ll be keeping an eye on the forums and ManageIQ GitHub page for more examples.

Resources:

http://manageiq.org/docs/get-started/
http://manageiq.org/docs/reference/
https://pemcg.gitbooks.io/mastering-automation-in-cloudforms-and-manageiq/content/chapter1.html

Top Posts 2016

2016 is pretty much done and dusted and it’s been an good year for Virtualization is Life! There was a more modest 70% increase in site visits this year compared to 2015 and a 2600% increase in visits since I began blogging in 2012. In 2016 I managed to produce 124 posts (including this one) which was slightly up on the 110 I produced in 2015 and in doing so passed 300 total blogs since I started here. I was fairly consistent in getting out at least eight blogs per month with June being my most prolific month with sixteen blog posts published.

Looking back through the statistics generate via JetPack, I’ve listed the Top 10 Blog Posts from the last 12 months. This year the opinion pieces seemed to be of interest to my readers and there is still vCloud Director and NSX representation in the top ten with my Veeam articles doing well. Again it was interesting to see that two of the most generic (older posts) and certainly basic posts took out two of the top three spots. It shows that bloggers should not be afraid of blogging around simple topics as there is an audience that will appreciate the content and get value out of the post.

  1. NSX Edge vs vShield Edge: Part 1 – Feature and Performance Matrix
  2. Quick Post: E1000 vs VMXNET3
  3. vSphere 6.0 vCenter Server Appliance: Upgrading from 5.x
  4. ESXi Bugs – VMware Can’t Keep Letting This Happen!
  5. Nutanix Buying PernixData: My Critical Analysis
  6. New NSX License Tier Thoughts and Transformers
  7. CBT Bugs – VMware Can’t Keep Letting This Happen!
  8. Veeam 9 Released: Top New Features
  9. Veeam’s Next Big Thing – Veeam has Arrived!
  10. vCloud Director 8: New Features And A New UI Addition…

I was honoured to have this blog voted #44 in the TopvBlog2016 and even with all the controversy around the voting I still hold that as a significant outcome of which I am very proud and I’d like to thank the readers and supporters of this blog for voting for me! And thanks must also go to my site sponsors who are all listed on the right hand side of this page.

With me moving across to vendor land it’s going to be interesting to see if I can keep up the variety of posts as I “narrow” down my core focus…however I fully intend to keep on pushing this blog by keeping it strong to it’s roots of vCloud Director and core VMware technologies like NSX and vSAN. I have the Home lab and the drive to continue to produce content around the things I am passionate about…and that includes all things hosting and cloud now with a touch of availability 🙂

Stay tuned for an even bigger 2017!

#LongLivevCD

OVFTool: vCloud Director OVA Upload PowerShell Script

Earlier this year I put together a quick and nasty PowerShell Script that exports a vApp from vCloud Director using the OVFTool …for those that don’t know the OVFTool is a command line tool that has a powerful set of functions to import/export VMs and vApps from vCenter, ESXi and vCloud Director weather it be from a vCloud Air or a vCloud Air Network Provider.

You can Download and install the tool from here:

This week I needed to upload an Virtual Machine that was in OVA format and for those that have worked with vCloud Director you would know that the OVA format is not supported using the upload functionality in the current web interface. With that I thought it was a good time to round out the export using OVTTool post with an import using OVFTool post. Again, doing some research I found a bunch of posts relating to importing OVAs into vCloud Director and after working through the Admin Guide and some examples I was ready to build out a basic import command and start work on the PowerShell Script. On Windows you can run the tool from CMD but I would suggest using PowerShell/CLI as in the example below I go through building a variable.

What Info is Required:

  • vCloud URL
  • vCloud Username and Password
  • Org Name
  • vDC Name
  • vApp Name
  • Catalog Name
  • Path to OVA

Command Line Example:

Below is a basic example of how to construct the vCloud String and use it as a variable to execute the tool.

PowerShell Script:

Again, I’ve taken it a step further to make it easier for people to import OVAs into vCloud Director and put together another, slightly improved PowerShell Script that I have coded in to work with my old companies vCloud Zones…though this can be easily modified to use any vCloud Air Network vCD endpoint.

The output of the script can be seen below:

It’s a very basic script that gathers all the required components that make up the vCloud Source Connection String and then exports the OVA into the vCD vApp. I’ve even done a little more PowerShell improvements around password security and added a little colour.

Save the code snippet as a .ps1 into the OFVTool Windows Folder and execute the script from the same location. If there are any errors with the inputs provided the OVFTool will fail with an error, but apart from that it’s a very simple straight forward way to import OVAs into any vCloud Director enabled endpoint.

Additional Reading:

http://www.virtuallyghetto.com/tag/ovftool

http://www.vmwarebits.com/content/import-and-export-virtual-machines-command-line-vmwares-ovf-tool 

« Older Entries Recent Entries »