Monthly Archives: February 2017

Veeam Vault #5: $200 Million Give Away, VCSP Roadshow Plus Vanguard Blog Updates

Welcome to the fifth edition of the Veeam Vault and the second of 2017. It’s been a busy four or so weeks for me since the last update preparing for a number of event’s and webinar’s happening over the next month all focusing on Veeam’s Cloud Business. In this Veeam Vault I am going to talk about an exceptional new promotion that Veeam is running to help drive increased adoption of Veeam Cloud Connect, talk briefly about the ANZ VCSP Roadshow and post a round up of all Veeam Vanguard blog posts since the last update.

Cloud Connect $200 Million Free Cloud Services:

On Valentines day we made public an amazing promotion that Veeam through it’s partners will be giving away $1000 in free cloud services to all existing veeam customers powered by the Veeam Cloud & Service Provider community.

 

This shows just how serious we are about ensuring our customers get the most out of our availability solutions by activating Cloud Connect Backup and Replication services that are included with all Veeam Backup & Replication licenses. A few weeks in and the program has been well received and I am looking forward to this rolling out across EMEA and ANZ over the next few months. For more information about the promotion and information on Cloud Connect Backup and Cloud Connect Replication have a read of my veeam.com Blog Post here.

ANZ VCSP Roadshow 2017:

Last week in Perth we kicked off the ANZ VCSP Roadshow for 2017…this has become an annual event hosted by Veeam ANZ and aims to encourage growth in the VCSP program by presenting to new or existing VCSP partners around Veeam’s Availability Platform that’s anchored by Cloud Connect technologies. If you are in Sydney, Melbourne, Auckland or Adelaide there is still time to register here.

VeeamOn 2017:

VeeamOn 2017 is fast approaching but Veeam is still giving away certification, tickets, flights and accomodation to this years event in May. Our latest competition is a based around our VMCE certification and if you click on the link below you will be taken to the landing page where you need to take a quiz to enter the competition.

Propel your personal career by joining us in New Orleans for a training experience with cutting edge Veeam instructors and complete your VMCE certification. If you are already a VMCE, attend the brand new VMCE-Advanced: Design & Optimization v1.

Take the quiz to win a fully paid trip by taking this quiz by March 20th.

You can register here or you can:

Veeam Vanguard Blog Post Roundup:

Released: vCenter and ESXi 6.0 Update 3 – What’s in It for Service Providers

Last month I wrote a blog post on upgrading vCenter 5.5 to 6.0 Update 2 and during the course of writing that blog post I conducted a survey on which version of vSphere most people where seeing out in the wild…overwhelmingly vSphere 6.0 was the most popular version with 5.5 second and 6.5 lagging in adoption for the moment. It’s safe to assume that vCenter 6.0 and ESXi 6.0 will be common deployments for some time in brownfield sites and with the release of Update 3 for vCenter and ESXi I thought it would be good to again highlight some of the best features and enhancements as I see them from a Service Provider point of view.

vCenter 6.0 Update 3 (Build 5112506)

This is actually the eighth build release of vCenter 6.0 and includes updated TLS support for v1.0 1.1 and 1.2 which is worth a look in terms of what it means for other VMware products as it could impact connectivity…I know that vCloud Director SP now expects TLSv 1.1 by default as an example. Other things listed in the What’s New include support for MSSQL 2012 SP3, updated M2VCSA support, timezone updates and some changes to the resource allocation for the platform services controller.

Looking through the Resolved Issue there are a number of networking related fixes in the release plus a few annoying problems relating to vMotion. The ones below are the main ones that could impact on Service Provider operations.

  • Upgrading vCenter Server from version 6.0.0b to 6.0.x might fail. 
    Attempts to upgrade vCenter Server from version 6.0.0b to 6.0.x might fail. This issue occurs while starting service An error message similar to the following is displayed in the run-updateboot-scripts.log file.
    “Installation of component VCSServiceManager failed with error code ‘1603’”
  • Managing legacy ESXi from the vCenter Server with TLSv1.0 disabled is impacted.
    vCenter Server with TLSv1.0 disabled supports management of legacy ESXi versions in 5.5.x and 6.0.x. ESXi 5.5 P08 and ESXi 6.0 P02 onwards is supported for 5.5.x and 6.0.x respectively.
  • x-VC operations involving legacy ESXi 5.5 host succeeds.
    x-VC operations involving legacy ESXi 5.5 host succeeds. Cold relocate and clone have been implicitly allowed for ESXi 5.5 host.
  • Unable to use End Vmware Tools install option using vSphere Client.
    Unable to use End VMware Tools install option while installing VMware Tools using vSphere Client. This issue occurs after upgrading to vCenter Server 6.0 Update 1.
  • Enhanced vMotion fails to move the vApp.VmConfigInfo property to destination vCenter Server.
    Enhanced vMotion fails to move the vApp.VmConfigInfo property to destination vCenter Server although virtual machine migration is successful.
  • Storage vMotion fails if the VM is connected with a CD ISO file.
    If the VM is connected with a CD ISO file, Storage vMotion fails with an error similar to the following:
  • Unregistering an extension does not delete agencies created by a solution plug-in.
    The agencies or agents created by a solution such as NSX, or any other solution which uses EAM is not deleted from the database when the solution is unregistered as an extension in vCenter Server.

ESXi 6.0 Update 3 (Build 5050593)

The what’s new in ESXi is a lot more exciting than what’s new with vCenter highlighted by a new Host Client and fairly significant improvements in vSAN performance along with similar TLS changes that are included in the vCenter update 3. With regards to the Host Client the version is now 1.14.0. and includes bug fixes and brings it closer to the functionality provided by the vSphere Client. It’s also worth mentioning that new versions of the Host Client continue to be released through the VMware Labs Flings site. but, those versions are not officially supported and not recommended for production environments.

For vSAN, multiple fixes have been introduced to optimize I/O path for improved vSAN performance in All Flash and Hybrid configurations and there is a seperate VMwareKB that address the fixes here.

  • More Logs Much less Space vSAN now has efficient log management strategies that allows more logging to be packed per byte of storage. This prevents the log from reaching its assigned limit too fast and too frequently. It also provides enough time for vSAN to process the log entries before it reaches it’s assigned limit thereby avoiding unnecessary I/O operations
  • Pre-emptive de-staging vSAN has built in algorithms that de-stages data on periodic basis. The de-staging operations coupled with efficient log management significantly improves performance for large file deletes including performance for write intensive workloads
  • Checksum  Improvements vSAN has several enhancements that made the checksum code path more efficient. These changes are expected to be extremely beneficial and make a significant impact on all flash configurations, as there is no additional read cache look up. These enhancements are expected to provide significant performance benefits for both sequential and random workloads.

As with vCenter, I’ve gone through and picked out the most significant bug fixes as they relate to Service Providers. The first one listed below is important to think about as it should significantly reduce the number of failures that people have been seeing with ESXi installed on SD-Flash Card and not just for VDI environments as the release notes suggest.

  • High read load of VMware Tools ISO images might cause corruption of flash media  In VDI environment, the high read load of the VMware Tools images can result in corruption of the flash media.
    You can copy all the VMware Tools data into its own ramdisk. As a result, the data can be read from the flash media only once per boot. All other reads will go to the ramdisk. vCenter Server Agent (vpxa) accesses this data through the /vmimages directory which has symlinks that point to productLocker.
  • ESXi 6.x hosts stop responding after running for 85 days
    When this problem occurs, the /var/log/vmkernel log file displays entries similar to the followingARP request packets might drop.
  • ARP request packets between two VMs might be dropped if one VM is configured with guest VLAN tagging and the other VM is configured with virtual switch VLAN tagging, and VLAN offload is turned off on the VMs.
  • Physical switch flooded with RARP packets when using Citrix VDI PXE boot
    When you boot a virtual machine for Citrix VDI, the physical switch is flooded with RARP packets (over 1000) which might cause network connections to drop and a momentary outage. This release provides an advanced option /Net/NetSendRARPOnPortEnablement. You need to set the value for /Net/NetSendRARPOnPortEnablementto 0 to resolve this issue.
  • Snapshot creation task cancellation for Virtual Volumes might result in data loss
    Attempts to cancel snapshot creation for a VM whose VMDKs are on Virtual Volumes datastores might result in virtual disks not getting rolled back properly and consequent data loss. This situation occurs when a VM has multiple VMDKs with the same name and these come from different Virtual Volumes datastores.
  • VMDK does not roll back properly when snapshot creation fails for Virtual Volumes VMs
    When snapshot creation attempts for a Virtual Volumes VM fail, the VMDK is tied to an incorrect data Virtual Volume. The issue occurs only when the VMDK for the Virtual Volumes VM comes from multiple Virtual Volumes datastores.
  • ESXi host fails with a purple diagnostic screen due to path claiming conflicts
    An ESXi host displays a purple diagnostic screen when it encounters a device that is registered, but whose paths are claimed by a two multipath plugins, for example EMC PowerPath and the Native Multipathing Plugin (NMP). This type of conflict occurs when a plugin claim rule fails to claim the path and NMP claims the path by default. NMP tries to register the device but because the device is already registered by the other plugin, a race condition occurs and triggers an ESXi host failure.
  • ESXi host fails with a purple diagnostic screen due to path claiming conflicts
    An ESXi host displays a purple diagnostic screen when it encounters a device that is registered, but whose paths are claimed by a two multipath plugins, for example EMC PowerPath and the Native Multipathing Plugin (NMP). This type of conflict occurs when a plugin claim rule fails to claim the path and NMP claims the path by default. NMP tries to register the device but because the device is already registered by the other plugin, a race condition occurs and triggers an ESXi host failure.
  • ESXi host fails to rejoin VMware Virtual SAN cluster after a reboot
    Attempts to rejoin the VMware Virtual SAN cluster manually after a reboot might fail with the following error:
    Failed to join the host in VSAN cluster (Failed to start vsantraced (return code 2)
  • Virtual SAN Disk Rebalance task halts at 5% for more than 24 hours
    The Virtual SAN Health Service reports Virtual SAN Disk Balance warnings in the vSphere Web Client. When you click Rebalance disks, the task appears to halt at 5% for more than 24 hours.

It’s also worth reading through the Known Issues section as there is a fair bit to be aware of especially if running NFS 4.1 and worth looking through the general storage issues.

Happy upgrading!

References:

http://pubs.vmware.com/Release_Notes/en/vsphere/60/vsphere-vcenter-server-60u3-release-notes.html

http://pubs.vmware.com/Release_Notes/en/vsphere/60/vsphere-esxi-60u3-release-notes.html

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2149127

Released: vCloud Director SP 8.20 with HTML5 Goodness!

This week, VMware released vCloud Director SP version 8.20 (build 5070630) which marks the 8th Major Release for vCloud Director since 1.0 was released in 2010. Ever since 2010 the user interface give or take a few minor modifications and additions has been the same. It also required flash and java which has been a pain point for a long time and in someways unfairly contributed towards a negative perception around vCD on a whole.  It’s been a long time coming but vCloud Director finally has a new web UI built on HTML5 however this new UI is only exposed when accessing the new NSX integration which is by far and away the biggest addition in this release.

This NSX integration has been in the works for a while now and has gone through a couple of iterations within the vCloud product team. Initially announced as Advanced Networking Services which was a decoupled implementation of NSX integration we now have a fully integrated solution that’s part of the vCloud Director installer. And while the UI additions only extend to NSX for the moment it’s brilliant to see what the development team have done with the Clarity UI (tbc). I’m going to take a closer look at the new NSX features in another post, but for the moment here are the release highlights of vCD SP 8.20.

New Features:

  • Advanced Edge Gateway and Distributed Firewall Configuration – This release introduces the vCloud Director Tenant Portal with an initial set of controls that you can use to configure Edge Gateways and NSX Distributed Firewalls in your organization.
  • New vCloud Director API for NSX – There is a new a proxy API that enables vCloud API clients to make requests to the NSX API. The vCloud Director API for NSX is designed to address NSX objects within the scope of a vCloud Director tenant organization.
  • Role Administration at the Organization Level – From this release role objects exist in each organization. System administrators can use the vCloud Director Web Console or the vCloud API to create roles in any organization. Organization administrators can use the vCloud API to create roles that are local to their organization.
  • Automatic Discovery and Import of vCenter VMs – Organization VDCs automatically discover vCenter VMs that exist in any resource pool that backs the vDC. A system administrator can use the vCloud API to specify vCetner resource pools for the vDC to adopt. vCenter VMs that exist in an adopted resource pool become available as discovered vApps in the new vDC.
  • Virtual Machine Host Affinity – A system administrator can create groups of VMs in a resource pool, then use VM-Host affinity rules to specify whether members of a VM group should be deployed on members of a vSphere host DRS Group.
  • Multi-Cell Upgrade – The upgrade utility now supports upgrading all the cells in your server group with a single operation.

You can see above that this release has some major new features that are more focused on tenant usability and allow more granular and segmented controls of networks, user access and VM discovery. The Automatic VM discovery and Import is a significant feature that goes along with the 8.10 feature of live VM imports and helps administrators import VM work loads into vCD from vCenter.

“VMware vCloud Director 8.20 is a significant release that adds enhanced functionality.  Fully integrating VMware NSX into the platform allows edge gateways and distributed firewalls to be easily configured via the new HTML5 interface.  Additional enhancements such as seamless cell upgrades and vCenter mapping illustrate VMware is committed to the platform and to vCloud Air Network partners.”

A list of known issues can be found in the release notes and i’d like to highlight the note around Virtual Machine memory for the vCD Cells…I had my NestedESXi lab instances crash due to memory pressures due to the fact the VMs where configured with only 5GB of RAM. vCloud Director SP 8.20 needs at least 6GB so ensure your cells are modified before you upgrade.

Well done the the vCloud Director Product and Development team for this significant release and I’ll look to dig into some of the new feature in detail in upcoming posts. You can also read the offical vCloud Blog release post here. I’m looking forward to what’s coming in the next release now…hopefully more functionality placed into the HTML5 UI and maybe integration with VMwareonAWS 😉

References:

http://pubs.vmware.com/Release_Notes/en/vcd/8-20/rel_notes_vcloud_director_8-20.html

https://www.vmware.com/support/pubs/vcd_sp_pubs.html

https://blogs.vmware.com/vcloud/2017/02/vmware-announces-general-availability-vcloud-director-8-20.html

vExpert’s of 2017 – Listen Up! It’s about the Advocacy

Overnight Cory Romero announced the intake of the 2017 VMware vExperts. As a now six time returning vExpert it would be easy for me to sit back enjoy a perceived sense of entitlement that comes with being a vExpert…but times have changed. The award has changed and the way people feel about the program has changed…when I first become a vExpert back in 2012 there was approximately 300 world wide…fast forward to 2017 and there are now 1463 give or take which is an increase of about 100 from 2016.

Over the past few years there are always comments and questions around the swelling of the numbers and how there should be a more stringent approval and acceptance structure. I myself shared those thoughts in previous posts…however my opinions around this have changed mainly because I have come to understand what the vExpert program (and other vendor programs) are all about and where myself, and VMware can achieve maximum value.

The vExpert program is designed to aid in your success and help amplify your internal and or external personal brands and channels. So whether you are a external evangelist or a internal champion we want to be sure you have the resources needed for the program so you can be more successful. Make no mistake that this program exists to help VMware push it’s products and services through the advocacy of the people in the group. The reward is given to those who in previous 12 months have shown themselves to be active in that advocacy. That doesn’t always mean that you need to be an active blogger or present at events, but it does mean that in your day to day role within the IT Industry you should be championing VMware as a company and break that down to champion VMware products that you use or sell.

This doesn’t mean that you can’t be involved in looking at and advocating other vendor technologies (many others hold multiple program memberships) but as Corey mentioned, the criteria used to have achieved the award implies that those activities need to be VMware focused.

Once you have the title it’s important to understand that there is a responsibility associated with it…it’s not just about the free gear though as I have stated before you should accept that as a perk of being part of the program and you shouldn’t feel like a “vendor whore” for accepting that shirt or coffee mug. Going back to responsibility, what I mean by that is that you should wear the badge proudly…understand that you have taken the time to apply/reapply for the award because you believed yourself worth of filling the selection criteria and use the award as a stepping stone to improve on the activities that got you there the year before.

Don’t rest on your laurels and expect the award to come to you every year…the vExpert team put a lot load of effort into keeping the program running and as a group we get significant exposure and opportunity from VMware and their partners…make it count and don’t waste it! Make sure you engage with others in the community through Twitter, LinkedIn or the Slack vExpert Channel or get down to your local VMUG or VMware event and engage directly.

NOTE: Content First Posted in 2016

NSX Bytes: NSX-v 6.3 Host Preparation Fails with Agent VIB module not installed

NSX-v 6.3 was released last week with an impressive list of new enhancements and I wasted no time in looking to upgrades my NestedESXi lab instance from 6.2.5 to 6.3 however I ran into an issue that at first I thought was related to a previous VIB upgrade issue caused by VMware Update Manager not being available during NSX Host upgrades…in this case it presented with the same error message in the vCenter Events view:

VIB module for agent is not installed on host <hostname> (_VCNS_xxx_Cluster_VMware Network Fabri)

After ensuring that my Update Manager was in a good state I was left scratching my head…that was until some back and forth in the vExpert Slack #NSX channel relating to a new VMwareKB that was released the same day as NSX-v 6.3.

https://kb.vmware.com/kb/2053782

This issue occurs if vSphere Update Manager (VUM) is unavailable. EAM depends on VUM to approve the installation or uninstallation of VIBs to and from the ESXi host.

Even though my Upgrade Manager was available I was not able to upgrade through Host Preparation. It seem’s like vSphere 6.x instances might be impacted by this bug but the good news is there is a relatively easy workaround as mentioned in the VMwareKB that bypasses the VUM install mechanism. To enable the workaround you need to enter into the Managed Object Browser of the vCenter EAM by going to the following URL and entering in vCenter admin credentials.

https://vCenter_Server_IP/eam/mob/ 

Once logged in you are presented with a (or list of) agencies. In my case I had more than one, but I selected the first one in the list which was agency-11

The value that needs to be changed is the bypassVumEnabled boolean value as shown below.

To set that flag to True enter in the following URL:

https://vCenter_Server_IP/eam/mob/?moid=agency-x&method=Update

Making sure that the agency number matches your vCenter EAM instance. From there you need to change the existing configuration for that value by removing all the text in the value box and invoking the value listed below:

Once invoked you should be able to go back into the Web Client and click on Resolve under the Cluster name in the Host Preparation Tab of the NSX Installation window.

Once done I was in an all Green state and all hosts where upgraded to 6.3.0.5007049. Once all hosts have been upgraded it might be a useful idea to reverse the workaround and wait for an official fix from VMware.

References:

https://kb.vmware.com/kb/2053782

NSX Bytes: NSX for vSphere 6.3 and NSX-T 1.1 Release Information

VMware’s NSX has been in the wild for almost three years and while the initial adoption was slow, of recent times there has been a calculated push to make NSX more mainstream. The change in licensing that happened last year has not only been done to help drive adoption by traditional VMware customers running vSphere that previously couldn’t look at NSX due to price but also the Transformers project has looked to build on Nicira’s roots in the heterogeneous hypervisor market and offer network virutalization beyond vSphere and beyond Open source platforms and into the public cloud space. The vision for VMware with NSX is to manage security and connectivity for heterogeneous end points through:

  • Security
  • Automation
  • Application Continuity

NSX has seen significant growth for VMware over the past twelve to eighteen months driven mostly from customer demand focusing around micro-segmentation, IT automation and efficiency and also the need to have extended multiple data centre locations that can be pooled together. To highlight the potential that remains with NSX-v less that 5% of the total available vSphere install base has NSX-v installed…and while that could have something to do with the initial restrictions and cost of the software it still represents enormous opportunity for VMware and their partners.

Last week the NSX vExpert group was given a first look at what’s coming in the new releases…below is a summation of what to expect from both NSX-v 6.3 and NSX-T 1.1. Note that we where not given an indication on vSphere 6.5 support so, like the rest of you we are all waiting for the offical release notes.

[Update] vSphere 6.5 will be supported with NSX-v 6.3

Please note that VMware vSphere 6.5a is the minimum supported version with NSX for vSphere 6.3.0. For the most up-to-date information, see the VMware Product Interoperability Matrix. Also, see 2148841.

NSX for vSphere 6.3 Enhancements:

Security:

  • NSX Pre-Assessment Tool based on vRealize Network Insight
  • Micro-Segmentation Planning and application visibility
  • New Security Certifications around ICSA, FIPS, Common Criteria and STIG
  • Linux Guest VM Introspection
  • Increase performance in service chaining
  • Larger scalability of VDI up to 50K desktops
  • NSX IDFW for VDI
  • Active Directory Integration for VDI at scale

Automation:

  • Routing Enhancements
  • Centralized Dashboard for service and ops
  • Reduced Upgrade windows with rebootless upgrades
  • Integration with vRA 7.2 enhancing LB,NAT
  • vCloud Director 8.20 support with advanced routing, DFW, VPN
  • VIO Updates to include multi-vc deployments
  • vSphere Integrated Container Support
  • New Automation Frameworks for PowerNSX, PyNSXv, vRO

Application Continuity:

  • Multi-DC deployments with Cross VC NSX enhancements for security tags
  • Operations enhancements with improved availability
  • L2VPN performance enhancements for cross DC/Cloud Connectivity

Where does NSX-T Fit:

Given there was some confusion about NSX-v vs. NSX-t in terms of everything going to a common code base starting from the transformers release it was highlighted that VMware’s primary focus for 2017 hasn’t shifted away from NSX for vSphere and will still be heavily invested in to add new capabilities in and beyond 6.3 and that there will be a robust roadmap of new capabilities in future releases with support extended will into the future.

NSX-t’s main drivers related to new data centre and cloud architectures with more hetrogeneality driving a different set of requirements to that of vSphere that focuses around multi-domain environments leading to a multi-hypervisor NSX platform. NSX-t is highly extensible and will address more endpoint heterogeneity in future releases including containers, public clouds and other hypervisors. As you can see before the existing use cases for NSX-t are mainly focused around devops, micro-segmentation and multi-tenant infrastructure.

NSX-T 1.1 Brief Overview:

Again the focus is around private IaaS and multi-hypervisor support for development teams using dev clouds and employing more devops methodologies. There isn’t too much to write home about in the 1.1.0 release but there is some extended hypervisor support for KVM and ESXi, more single or multi-tenant support and some performance and resiliency optimizations.

Conclusion:

There is a lot to like about where VMware is taking NSX and both product streams offer strong network virtualization capabilities for customers to take advantage of. There is no doubt in my mind that the release of NSX-v 6.3 will continue to build on the great foundation laid by the previous NSX versions. When the release notes are made available I will do take a deeper look into all the new features and enhancements and tie them into what’s most useful for service providers.