Monthly Archives: April 2018

vSphere 6.7 – What’s in it for Service Providers Part 1

Posted on by

A few weeks ago after much anticipation VMware released vSphere 6.7. Like 6.5 before it, this is a lot more than a point release and represents a major upgrade from vSphere 6.5. There is so much packed into this new release that there is an official page with separate blog posts talking about the features and enhancements. As usual, I will go through some of the key features and enhancements that are included in the latest versions of vCenter and ESXi and as they relate back to the Service Providers that use vSphere as the foundation of their Infrastructure as a Service offerings.

There is a lot go get through though and like the vSphere 6.5 release the “whats new” will not fit into one post so i’ll split the highlights between a couple posts and I’ll cover ESXi specifically in a follow-up. I still feel like it’s important to highlight the base hypervisor as well as the management platform. I’ll also talk about current interoperability with vCloud Director and NSX as well as Veeam supportability for vSphere 6.7.

The major features and enhancements as listed in the What’s New PDF are:

  • Scalability Enhancements
  • VMware vCenter Server Appliance Linked Mode
  • VMware vCenter Server Appliance Back Up Scheduler
  • Single Reboot
  • Quick Boot
  • Support for 4K Native Storage
  • Improved HTML 5 based vSphere Client
  • Security-at-Scale
  • Support for Trusted Platform Module (TPM) 2.0 and virtual TPM
  • Cross-vCenter Encrypted vMotion
  • Support for Microsoft’s Virtualization Based Security (VBS)
  • NVIDIA GRID vGPU Enhancements
  • vSphere Persistent Memory
  • Hybrid Linked Mode
  • Per-VM Enhanced vMotion Compatibility (EVC)
  • Cross-vCenter Mixed Version Provisioning – Simplify provisioning across hybrid cloud environments that have diferent vCenter versions

Below are the ones in red fleshed out in the context of Service Providers.

Enhanced vCenter Server Appliance:

The VCSA has been enhanced significantly in this release. Having used the VCSA exclusively for the past year in all my environments I have a love hate relationship with it. I still feel it’s nowhere as stable as vCenter running ontop of Windows and is prone to more issues than a Windows based vCenter…however this 6.7 release will be the last one supporting or offering a Windows based vCenter. With that VMware have had to work hard on making the VCSA more resilient.

Compared to the 6.5 VCSA, 6.7 offers twice the performance in vCenter operations per second with a three times reduction in memory usage and three times faster DRS operations meaning that power on and other VM operations are performed quicker. This is great on a service provider platform with potentially lots of those operations happening during the course of a day. Hopefully this improves the responsiveness overall of the VCSA which I have felt at times to be poor under load or after an extended period of appliance uptime.

There has also been a number of updates to the APIs offered in vSphere, the VCSA and ESXi. William Lam has a great post on what’s new for APIs here, but all Service Providers should have teams looking at the API Explorer as it’s a great way to explore and learn what’s available.

Single Reboot and Quick Reboot:

For Service Providers who need upgrade their platforms to maintain optimal compatibility, upgrading hosts can be time consuming at scale. vSphere 6.7 reduces ESXi host upgrades, by eliminating one of the two reboots normally required for major version upgrades. This is the single reboot feature. There is also vSphere Quick Boot that restarts the ESXi hypervisor without rebooting the physical host. This skips time-consuming server hardware initialization and post boot operation wait times. Both of these significantly reduce maintenance times.

This blog post covers both features in more detail.

Improved HTML 5 based vSphere Client:

While minor in terms of actual under the hood improvements, the efficiencies that are gained when it comes to a decent user interface are significant. When managing Service Provider platforms at scale, having a reliable client is important and with the decommissioning of the VI client and the often frustrating performance of the Flex client a near complete and workable HTML vSphere Client is a big plus for those who work day to day on vCenter.

The vSphere 6.7 vSphere Client has support for vSAN as well as having Update Manager fully built in. As per the last NSX 6.4 update there is also limited management of that. There is also a new vROps plugin…this plugin is available out-of-the-box once vROps has been linked with vCenter and offers dashboards directly in the vSphere client that can view, cluster view, and alerts for both vCenter and vSAN views. This is extremely handy for Service Providers who use vROps dashboard not needing to go to two different locations to get the info.

vCD and NSX Supportability:

Shifting from new features and enhancements to an important subject to talk about when talking service provider platform…VMware product compatibility. For those VCPP Service Providers running a Hybrid Cloud you should be running a combination of vCloud Director SP or/and NSX-v of which, at the moment there is no support for either in vSphere 6.7.

Looking at vCloud Director, it looks like 9.1 is supported however given the fact you need to be running NSX-v with vCD these days and NSX is not yet supported, it doesn’t make too much sense to suggest that there is total compatability.

I suspect we will see NSX-v come out with a supported build shortly…though I’m only expecting vCloud Director SP to support 6.7 form version 9.1 which will mean upgrades.

Veeam Backup & Replication Supportability: 

Veeam commits to supporting major version releases within 90 days or sooner of GA. So with that, those Service Provider that are also VCSPs using Veeam to backup their infrastructure should not upgrade to vSphere 6.7 until Backup & Replication Update 3a is released. For those that are bleeding edge and have updated your only option at that point is our Agents for Windows and Linux until Update 3a is released.

Wrapping up Part 1:

Rounding off this post, in the Known Issues section there is a fair bit to be aware of for 6.7. it’s worth reading through all the known issues just in case there are any specific issues that might impact you. In upcoming posts around vSphere 6.7 for Service Providers series I will cover more vCenter features as well as ESXi enhancements and what’s new in Core Storage.

Happy upgrading!

References:

https://docs.vmware.com/en/VMware-vSphere/6.7/rn/vsphere-esxi-vcenter-server-67-release-notes.html

https://blogs.vmware.com/vsphere/2018/04/introducing-faster-lifecycle-management-operations-in-vmware-vsphere-6-7.html

Released: vSAN 6.7 – HTML5 Goodness, Enhanced Health Checks and More!

Posted on by

VMware has announced the general availability of vSAN 6.7. As vSAN continues to grow, VMware are very buoyant about how it’s performing in the market. With some 10,000 customers at a run rate of over 600 million they claim to lead the HyperConverged market with a 32% market share. From my point of view it’s great to see vSAN being deployed across 250 cloud providers and have it as the cornerstone storage of the VMware Cloud on AWS solution. vSAN 6.7 is focusing on intuitive operational experience, consistent application experience and holistic support experience.

New Features and Enhancements:

  • HTML5 User Interface
  • Embedded vROPs plugin for HTML5 User Interface
  • Support for Windows Failover Cluster using iSCSI
  • Adaptive Resync Performance Improvements
  • Destaging Performance Improvements
  • More Efficient data placement during Host Decommissioning
  • Improved Space Efficiency
  • Faster Failover with Redundant vSAN Networks
  • Optimized Witness Traffic Seperation
  • Stretched Cluster Improvements
  • Host Affinity for Next-Gen Applications
  • Health Check Enhancements
  • Enhanced Diagnostics
  • vSAN Support Insight
  • 4Kn Device Support
  • Improved FIPS 140-2 Validation Security

There are a lot of enhancements in this release and while not as ground breaking at the 6.6 release last year, there is still a lot to like about how VMware is improving the platform. From the list above, i’ve taken the key ones from my point of view and expanded on them a little.

HTML5 User Interface:

As has been the trend with all VMware products of late, vSAN is getting the Clarity Framework overhaul and is being included in the HTML5 vSphere Web Client with new vSAN tasks and workflows developed from the ground up to simplify the experience. There is also new vSAN functionality that can only be accessed via the HTML5 client.

The legacy Flex client will still be available for use and it’s also worth noting that this is not a direct port of the Flex interface but started from the ground up. This has resulted in a more efficient experience for the user with less clicks and less time to action items. Any new features or enhancements will only be seen in the new HTML5 UI.

Support for Windows Failover Cluster using iSCSI:

A few weeks back I posted around how you could use vSAN as Veeam repository using the iSCSI feature. With vSAN 6.7 there is offical support for Windows Failover Clustering using the vSAN iSCSI service. Lots of people still run MSCS and a lot still use traditional clustering. This supports physical and virtual Guest iSCSI initiators that includes transparent failover of clusters with vSAN iSCSI volumes.

I’m not sure if this now means that iSCSI volumes are supported as Veeam Cloud Repositories…but I will confirm either way.

Adaptive Resync Performance Improvements:

vSAN 6.7 introduces a new Adaptive ReSync feature that will make sure resources are available for VM IO and resync IO. This ensures that under IO stress certain traffic types are not starved of resources and allows more bandwidth to be used when there are periods of less contention. Under contention, resync IO will be guaranteed at least 20% of the bandwidth and if no resync traffic exists, VM IO may consume 100%. This is effectively regulating reads and writes to ensure optimal balance for VM and reync IO.

Destaging Performance Improvements:

vSAN 6.7 looks to be more consistent when talking about data optimizations in the data path. With the faster destaging, data drains more quickly from the write buffer to the capacity tier. This allows the buffer tier to be available for newer IO quicker. This is done via improved in-memory handling of IO during destaging that delivers higher throughput and more consistency which in turn improves the overall performance of VM and resync IO.

More Efficient data placement during Host Decommissioning:

When putting a host in maintenance mode or decommissioning a host you need to select the evacuation type for the objects on that host. This can take time depending on the amount of data. vSAN 6.7 builds on improvements introduced in 6.6 that consolidates replicas living across multiple hosts while maintaining FTT compliance. Is looks for the smallest component to move while results in less data being rebuilt and less temporary space usage. vSAN will provide more intelligence behind the data movement to reduce the time and effort it takes to put a host into maintenance mode.

Improved Space Efficiency:

In previous vSAN versions the VM swap object was always thick provisioned even if the VM it’s self was thin. in vSAN 6.7 this will now be thin by default and also inherit the policy from the VM so that the FTT is the swap object is consistent with the VM which results in more efficient storage. Previous to this, large environments would suffer with a large number of swap files taking up a higher proportionate amount of space.

 

Conclusion:

vSan continues to be improved by VMware and they have addressed some core usability and efficiency features in this 6.7 release. The move to the HTML5 web client was expected, but still good to see while the enhancements in resync and destaging all contributes to platform stability. The enhanced health checks add a new dimension to vSAN troubleshooting and the support insight allows users to get a better view of what’s happening on their instances.

References:

Pre release information and images sourced via VMware EABP

https://blogs.vmware.com/virtualblocks/2018/04/17/whats-new-vmware-vsan-6-7/

 

 

Cloud Connect Subtenants, Veeam Availability Console and Agents!

Posted on by

Cloud Connect Subtenants have gone under the radar for the most but can play an important role in how Service Provider customers consume Cloud Connect services. In a previous post, I described how subtenants work in the context of Cloud Connect Backup.

Subtenants can be configured by either the VCSP or by the tenant consuming a Cloud Connect Backup service. Subtenants are used to carve up and assign a subset of the parent tenant storage quota. This allows individual agents to authenticate against the Cloud Connect service with a unique login allowing backups to Cloud Repositories that can be managed and monitored from the Backup & Replication console.

In this post I’m going to dive into how subtenants are created by the Veeam Availability Console and how they are then used by agents that are managed by VAC. For those that may not know what VAC does, head to this post for a primer.

Automatic Creation of Subtenant Users:

Veeam Availability Console automatically creates subtenant users if a backup policy that is configured to use a cloud repository as a backup target is chosen. When such a backup policy is assigned to an agent, VAC creates a subtenant account on the Cloud Connect Server for each backup agent.

Looking below you can see a list of the Backup Agents under the Discovery Menu.

Looking at the Backup Policy you can see that the Backup Target is a Cloud Repository, which results in the corresponding subtenant account being created.

The backup agents use these subtenant accounts to connect and send data to a Cloud Connect endpoint that are backed by a cloud repository. The name of each subtenant account is created according to the following naming convention:

companyname_computername

At the Cloud Provider end from within the Backup & Replication console under the Cloud Connect Menu and under tenants, clicking on Manage Subtenants will show you the corresponding list of subtenant accounts.

The view above is the same to that seen at the tenant end. A tenant can modify the quota details from the Veeam Backup & Replication console. This will result in a Custom Policy status as shown below. The original policy can be reapplied from VAC to bring it back into line.

The folder structure on the Cloud Repository maps what’s seen above. As you can also see, if you have Backup Protection enable you will also have _RecycleBin objects there.

NOTE: When a new policy is applied to an agent the old subtenant account and data is retained on the Cloud Connect repository. The new policy gets applied and a subtenant account with an _n gets created. Service Providers will need to purge old data manually.

Finally if we look at the endpoint where the agent is installed and managed by VAC you will see the subtenant account configured.

Conclusion:

So there is a deeper look at how subtenants are used as part of the Veeam Availability Console and how they are created, managed and used by the Agent for Windows.

References:

https://helpcenter.veeam.com/docs/vac/provider_admin/create_subtenant_user.html?ver=20

Upgrading Windows Agents with Veeam Availability Console

Posted on by

One of the Veeam Availability Console’s key features is it’s ability to deploy and manage Veeam Agent for Windows. This is done through the VAC Web Console and is achieved through the connectivity of the providers Cloud Connect Gateway to the tenant’s Veeam Backup & Replication instance. Weather this is managed by a service provider or by the tenant, VAC also has the ability to remotely upgrade Windows Agents.

The way that this works is by the Veeam Availability Console periodically connecting to the Veeam Update Server and checks whether a new version of the agent software is available. If a new version is available, VAC displays a warning next to the agents saying that it is outdated as shown below.

Updating the backup agents from the Veeam Update Server is performed via the master agent that sits on-premises. This agent is deployed during the initial Service Provider configuration form the Veeam Backup & Replication server. The master agent downloads the backup agent setup file from the Veeam Update Server and then uploads this setup file to systems selected via the update scope and initiates the update.

To initiate the upgrade, select the agents from the Backup Agents Tab under Clients -> Discovery. Once selected click on the Backup Agent dropdown and click upgrade.

Note: Once you click Upgrade the process will be kicked off…there is no further confirmation. There is also a Patch option which allows you to apply patches to the agents in between major build releases.

Once initiated, all agents will be shown as updating as shown below.

Taking a look at the Resource Monitor of one of the endpoints being updated, you can see that the machine is receiving the update from the local server that has the master agent and that the agent is talking back to the VAC server via Cloud Connect Port 6180.

And you can see the Windows Installer running the agent update msi.

Back to the VAC console, and after a while you will see the update deployment status complete

And the endpoint now has the updated agent version running.

Which is reflected in the VAC Console.

Conclusion:

That’s the very straight forward process of having the Veeam Availability Console upgrade Veeam Windows Agents under it’s management. Again, this can be done by the service provider or it’s a task that can be executed by the tenant through their own console login given the correct permissions. There are a few other options for those that deployed the agents with the help of a 3rd party tool and also for those doing it offline…for a run down of that process, head to the help pages linked below.

References:

https://helpcenter.veeam.com/docs/vac/provider_admin/update_backup_agents.html?ver=20

Released: NSX-v 6.3.6

Posted on by

Last week VMware released NSX-v 6.3.6 (Build 8085122) that doesn’t contain any new features but addresses a number of bug fixes from previous releases. This has been done independently of any updated release of NSX-v 6.4.0 that went GA in January.

This is good to see though interesting to also see that people are still not upgrading to 6.4.0 in droves meaning VMware needs to support both versions. Going through the release notes there are a lot of known issues that should be known and there are more than a few that apply to service providers.

Some key fixes are listed below:

Important Fixes :

  • Network outage of ~40-50 seconds seen on Edge Upgrade – During Edge upgrade, there is an outage of approximately 40-50 seconds
  • After upgrading to 6.3.5, the routing loop between DLR and ESG’s causes connectivity issues in certain BGP configurations –  A routing loop is causing a connectivity issue
  • NSX Manager CPU high due to edge in read-only file system mode – NSX Manager is slow to respond because it keeps 100% CPU and receives a lot of read-only file system events from edge.
  • After upgrade from vCNS edge 5.5.4 to NSX 6.3.6, customers could not configure Health-Check-Monitor port nor make any changes directly from vCD – Customers will not be able to configure Health-Check-Monitor port nor make any changes directly from vCD.
  • Distributed Firewall stays in Publishing state with certain firewall configurations – Distributed Firewall stays in “Publishing” state if you have a security group that contains an IPSet with 0.0.0.0/0 as an EXCLUDE member, an INCLUDE member, or as a part of ‘dynamic membership containing Intersection (AND)’

Those with the correct entitlements can download NSX-v 6.3.6 here.

References:

https://docs.vmware.com/en/VMware-NSX-for-vSphere/6.3/rn/releasenotes_nsx_vsphere_636.html

VeeamOn 2018: Recognizing Innovation and what it means to be Innovative

Posted on by

True innovation is solving a real problem…and though for the most, it’s startups and tech giants that are seen to be the innovators, their customers and partners also have the ability to innovate. Innovation drives competitive advantages and allows companies to differentiate themselves compared to others. In my previous roles I was lucky to be involved with teams of talented people that did great things with great technologies. Like others around the world we where innovating with leading vendor technologies to create new service offerings that add value and compliment the underlying technology.

Innovation requires these teams of people to be experimental at heart and try to build or enhance upon already existing technologies. The Service Provider industry has always found a way to innovate ontop of vendor platforms and successful vendors are those that offer the right tools and guidance for providers to creative innovative solutions ontop of their platforms. The are problem solvers!

Orchestrations, automation, provisioning and billing are driving factors in how service providers can differentiate themselves and gain that competitive advantage in the marketplace. Without innovating ontop of these platforms, service offerings become generic, don’t stand out and are generally operationally expensive to manage and maintain.

Introducing the Veeam Innovation Awards for 2018:

When visiting and talking to different partners across the world it’s amazing to see some of the innovation that’s been built ontop of Veeam technologies and we at Veeam want to reward our customers and partners who have done great things with our technologies.

At VeeamON 2018, we’ll be celebrating some of these innovative solutions, so please let us know how you’ve built upon the Veeam Availability Platform. Nominations can be made from March 29 to April 30, with the winners being recognized during the VeeamON main stage keynote. Self nominations or those from partners, providers, or Veeam field-team members are encouraged — click here to nominate for a Veeam Innovation Award.

I can think of a number of VCSPs that have done great things with building upon Cloud Connect, Backup & Replication IaaS backups and working with Veeam’s API’s and PowerShell to solve customer problems and offer value added services. These guys have brought something new to the industry and we want to reward that.

Having previously come from a successfully innovate company within their own space, being innovative is now something I try to preach to all customers and partners I visit. It is an absolute requirement if you want to win business and stand out in the backup and availability industry…innovation is key and we want to hear about it from you!

References:

https://www.veeam.com/executive-blog/nominations-veeamon-2018-innovation-awards.html