Category Archives: VMware

Released – Runecast Analyser 2.0

Earlier this week, Runecast released into General Availability version 2.0 of their vSphere analyser platform. I’ve been a keen follower of the progress of Runecast since their inception a couple of years ago. There was a space in the market to be filled and they have been able to improve in the initial release by releasing new functionality often. It wasn’t that long ago that they added vSAN support…and more recently NSX support.

This release brings the following new functionalities:

  • Ability to store and display all detected and resolved issues over time for every connected vCenter.
  • The completely new monitoring dashboard with The Most Affected hosts and trending.
  • Automation of PCI-DSS VMware rules and new PCI-DSS profile UI
  • Support for vSphere 6.7 HTML5 plugin
  • Usability, performance and security improvements for increased ease of use.
  • Latest VMware Knowledge Base updates.

First thing to notice in the new release is the new Dashboard that has been improved and for mine is now more logically laid out. But for me the biggest feature added in this release is the enhancement to Historical Trending and a new analysis function. As someone who spent a time managing and operating vSphere platforms over the years, the ability to see trends is crucial in troubleshooting.


Historical Analysis is new in version 2.0 and aims to help isolate the root cause of a reported incident as fast as possible and detect new problems caused by product update or configuration changes. 2.0 will store at least 3 months worth of vCenter, vSAN and NSX-V scan results, including issue description. This provides trending information on the dashboard.

The introduction of PCI-DSS checks is something that will assist in compliancy situations. As someone who has had the pain of going through compliancy, any tool that makes the process easier is welcomed.

Im looking forward to meeting up with the guys at VMworld 2018 in Las Vegas next week and I would recommend and vSphere admin to take a look at Runecast!
You can download Runecase 2.0 from here and take it for a spin: https://runecast.biz/profile

Veeam @VMworld 2018 Edition…

VMworld 2018 is less than a week away, and I can’t wait to fly into Las Vegas for my sixth VMworld and second with Veeam. It’s been an interesting year or so since the last VMworld and the industry has shifted a little when it comes to the backup and recovery market. Data management is the new buzz and lots of vendors (us included) have jumped onto the messaging around data growing at more than exponential rates…sprawling to more platforms than ever before and finally…being more critical than ever. The criticality and power of data is real and VMware still have a lot to say about where an how that data is processed and stored!

VMworld is still a destination event and Veeam recognises VMware’s continued influence in the IT industry by going all in at VMworld 2018. The ecosystem that VMware has built over the past ten to fifteen years is emense and though challenged a few years ago, came back with a bang in 2017. I’m looking forward to seeing VMware’s continues evolution at this years event! Like VMware,

Veeam is evolving as well, and we are building out own own strong ecosystem based on a software first, hardware agnostic platform that results in the greatest flexibility in the backup and recovery market. We continue to support VMware as our number 1 technology partner and this year we look to build on that with support for VMware Cloud on AWS and enhanced VMware features sets built into our core Backup & Replication product as we look to release Update 4 of 9.5 later in the year.

Veeam Sessions @VMworld:

Officially we have two breakout sessions this year, with Danny Allan and Rick Vanover presenting a What’s New in Update 4 for Veeam Backup & Replication and Michael Cade and myself presenting a session on Automation and Orchestration of VMware and Veeam on VMware Cloud on AWS. There are also a couple of vBrownBag Tech Talks where Veeam features including talks from Michael Cade and Michael White while Dave Russell will be presenting a Partner Spotlight session.

https://my.vmworld.com/widget/vmware/vmworld18us/uscatalog?search=Veeam

Veeam @VMworld Solutions Exchange:

This year, as per usual we will have significant presence on the floor, with a Main Booth Area doing demo’s prize, giveaways, having an Experts Bar and acting as sponsor of the opening night hall crawl. We also have an in booth Theatre where I will be presenting on our new vCloud Director integration with Veeam Cloud Connect.

Veeam Community Support @VMworld:

Veeam still gets the community and has been a strong supporter historically of VMworld community based events. This year again, we have come to the party are have gone all-in in terms of being front and center in supporting community events. Special mention goes to Rick Vanover who leads the charge in making sure Veeam is doing what it can to help make these events possible:

  • Opening Acts
  • VMunderground
  • vBrownBag
  • Spousetivities
  • vRockstar Party
  • Vanguard Takeover

Party with Veeam @VMworld:

Finally, it wouldn’t be VMworld without attending Veeam’s seriously legendary party. This year we are looking to top last years event at Hakkasan nightclub by taking over one of the hottest club in Vegas… Omnia Nightclub! If it’s anything like the VeeamON 2015 Party that I attended it’s going to go off!! I know how hard it is to plan evening activities at VMworld and there is no doubt that there are a lot of decent competing parties on the Tuesday night…however whatever you do, you need to make sure that you at least stop by Caesars Casino and party in green. RSVP here.

https://www.eventbrite.com/e/veeams-legendary-vmworld-party-2018-tickets-45869296300

Final Word:

Again, this year’s VMworld is going to be huge and Veeam will be right there front and center of the awesomeness. Please stop by our sessions, visit our stand and attend our community sponsored events and feel free to chase me down for a chat…I’m always keen to meet other members of this great community. Oh, and don’t forget to get to the party!

Automating the Creation of AWS VPC and Subnets for VMware Cloud on AWS

Yesterday I wrote about how to deploy a Single Host SDDC through the VMware Cloud on AWS web console. I mentioned some pre-requisites that where required in order for the deployment to be successful. Part of those is to setup an AWS VPC up with networking in place so that the VMC components can be deployed. While it’s not too hard a task to perform through the AWS console, in the spirit of the work I’m doing around automation I have gotten this done via a Terraform plan.

The max lifetime for a Single Instance deployment is 30 days from creation, but the reality is most people will/should be using this to test the waters and may only want to spin the SDDC up for a couple of hours a day, run some tests and then destroy it. That obviously has it’s disadvantages as well. The main one being that you have to start from scratch every time. Given the nature of the VMworld session around the automation and orchestration of Veeam and VMC, starting from scratch is not an issue however it was desirable to look for efficiencies during the re-deployment.

For those looking to save time and automate parts of the deployment beyond the AWS VPC, there are a number of PowerShell code example and modules available that along with the Terraform plan, reduce the time to get a new SDDC firing.

I’m using a combination of the above scripts to deploy a new SDDC once the AWS VPC has been created. The first one actually deploys the SDDC through PowerShell while the second one is a module that allows some interactivity via commandlets to do things such as export and import Firewall rules.

Using Terraform to Create AWS VPC for VMware Cloud on AWS:

The Terraform plan linked here on GitHub does a couple of things:

  • Creates a new VPC
  • Creates a VPC Network
  • Creates three VPC subnets across different Availability Zones
  • Associates the three VPN subnets to the main route table
  • Creates desired security group rules

https://github.com/anthonyspiteri/vmc_vpc_subnet_create

[Note] Even for the Single Instance Node SDDC it will take about 120 minutes to deploy…so that needs to be factored in in terms of the window to work on the instance.

Creating a Single Host SDDC for VMware Cloud on AWS

While preparing for my VMworld session with Michael Cade on automating and orchestrating the deployment of Veeam into VMware Cloud on AWS, we have been testing against the Single Host SDDC that’s been made available for on demand POCs for those looking to test the waters on VMware Cloud on AWS. The great thing about using the Single Host SDDC is it’s obviously cheaper to run than the four node production version, but also that you can spin it up and destroy the instance as many times as you like.

Single Host SDDC is our low-cost gateway into the VMware Cloud on AWS hybrid cloud solution. Typically purchased as a 4-host service, it is the perfect way to test your first workload and leverage the additional capability and flexibility of VMware Cloud on AWS for 30 days. You can seamlessly scale-up to Production SDDC, a 4-host service, at any time during the 30-days and get even more from the world’s leading private cloud provider running on the most popular public cloud platform.

To get started with the Single Host SDDC, you need to head to this page and sign up…you will get an Activation email and from there be able to go through the account setup. This big thing to note at the moment is that a US Based Credit Card is required.

There are a few pre-requisites before getting an SDDC spun up…mainly around VPC networking within AWS. There is a brilliant blog post here, that describes the networking that needs to be considered before kicking off a fresh deployment. The offical help files are a little less clear on what needs to be put into place from an AWS VPC perspective, but in a nutshell you need:

  • An AWS Account
  • A fresh VPC with a VPC Networking configured
  • At least three VPC Subnets configured
  • A Management Subnet for the VMware Objects to sit on

Once this has been configured in the AWS Region the SDDC will be deployed into the process can be started. First step is to select a region (this is dictated by the choices made at account creation) and then select a deployment type followed by a name for the SDDC.

The next step is to link an existing AWS account. This is not required at the time of setup however it is required to get the most out of the solution. This will go off and launch an AWS CloudFormation template to connect the SDDC to the AWS account. It creates IAM role to allow communication between the SDDC and AWS.

[Note] I ran into an issue initially where the default location for the CloudFormation template to be run out of was not set to the region where the SDDC was to be deployed into. Make sure that when you click on the Launch button you take not the the AWS region and change where appropriate by change the URL to the correct region.

After a minute or so, the VMware Cloud on AWS Create an SDDC page will automatically refresh as shown below

The next step is to select the VPC and the VPC subnets for the raw SDDC components to be deployed into. I ran into a few gotcha’s on this initially and what you need to have configured is the subnets configured to size as listed in the user guides and the post I linked to that covers networking, but you also need to make sure you have at least three subnets configured across different AWS Availability zones within the region. This was not clear, but I was told by support that it was required.

If the AWS side of things is not configured correctly you will see this error.

What you should see…all things being equal is this.

Finally you need to set the Management Subnet which is used for the vCenter, Hosts, NSX Manager and other VMware components being deployed into the SDDC. There is a default, but it’s important to consider that this should not overlap with any existing networks that you may look to extend the SDDC into.

From here, the SDDC can be deployed by clicking on the Deploy SDDC button.

[Note] Even for the Single Instance Node SDDC it will take about 120 minutes to deploy and you can not cancel the process once it’s started.

Once completed we can click into the details of the SDDC, which allows you to see all the relevant information relating to it and also allows you to configure the networking.

Finally, to access the vCenter you need to configure a Firewall rule to allow web access through the management gateway.

Once completed you can login to the vCenter that’s hosted on the VMware Cloud on AWS instance and start to create VMs and have a play around with the environment.

There is a way to automate a lot of what i’ve stepped through above…for that, i’ll go through the tools in another blog post later this week.

References:

Selecting IP Subnets for your SDDC

Workaround – VCSA 6.7 Upgrade Fails with CURL Error: Couldn’t resolve host name

It’s never an issue with DNS! Even when DNS looks right…it’s still DNS! I came across an issue today trying to upgrade a 6.5 VCSA to 6.7. The new VCSA appliance deployment was failing with an OVFTool error suggesting that DNS was incorrectly configured.

Initially I used the FQDN for source and target vCenter’s and let the installer choose the underlying host to deploy the new VCSA appliance to. Even though everything checked out fine in terms of DNS resolution across all systems I kept on getting the failure. I triple checked name resolution on the machine running the update, both vCenter’s and the target hosts. I even tried using IP addresses for the source and target vCenter but the error remained as it still tried to connect to the vCenter controlled host via it’s FQDN resulting in the error.

After doing a quick Google search and finding nothing, I changed the target to be an ESXi host directly and used it’s IP address over it’s FQDN. This time the OVFTool was able to do it’s thing and deploy the new VCSA appliance.

The one caveat when deploying directly to a host over a vCenter is that you need to have the target PortGroup configured as an ephemeral…but that’s a general rule of bootstrapping a VCSA in any case and it’s the only one that will show up from the drop down list.

While very strange given all DNS checked out as per my testing, the workaround did it’s thing and allowed me to continue with the upgrade. This didn’t find the root cause…however when you need to motor on with anupgrade, a workaround is just as good!

Adding Let’s Encrypt SSL Certificate to vCloud Director Keystore

For the longest time the configuring of vCloud Director’s SSL certificate keystore has been the thing that makes vCD admins shudder. There are lots of posts on the process…some good…some not so good. I even have a post from way back in 2012 about fronting vCD with a Citrix NetScaler and if I am honest, I cheated in having HTTPS at the load balancer deal with the SSL certificate while leaving vCD configured with the self signed cert. With the changes to the way the HTML5 Tenant Portal deals with certs and DNS I’m not sure that method would even work today.

I wanted to try and update the self signed certs in both my lab environments to assist in resolving the No Datacenters are available issue that cropped up in vCD 9.1. Instead of generating and using self signed certs I decided to try use Let’s Encrypt signed certs. Most of the process below is curtesy of blog posts from Luca Dell’Oca and it’s worth looking at this blog post from Tom Fojta who has a PowerShell script to automate Let’s Encrypt SSL certs for us on NSX Edge load balancers.

In my case, I wanted to install the cert directly into the vCD Cell Keystore. The manual end to end the process is listed below. I intend to try and automate this process so as to overcome the one constraint with using Let’s Encrypt…that is the 90 day lifespan of the certs. I think that is acceptable and it ensures validity of the SSL cert and a fair caveat given the main use case for this is in lab environments.

Generating the Signed SSL Cert from Let’s Encrypt:

To complete this process you need the ACMESharp PowerShell module. There are a couple of steps to follow which include registering the domain you want to create the SSL cert against, triggering a verification challenge that can be done by creating a domain TXT record as shown in the output of the challenge command. Once submitted, you need to look out for a Valid Status response.

Once complete, there is a script that can be run as show on Luca’s Blog. I’ve added to the script to automatically import the newly created SSL cert into the Local Computer certificate store.

From here, I exported the certificate with the private key so that you are left with a PFX file. I also saved to Base-64 X.509 format the Root and Intermediate certs that form the whole chain. This is required to help resolve the No Datacenters are available error mentioned above. Upload the three files to the vCD cell and continue as shown below.

Importing Signed SSL from Let’s Encrypt into vCD Keystore:

Next, the steps to take on the vCD Cell can be the most complex steps to follow and this is where I have seen different posts do different things. Below shows the commands from start to finish that worked for me…see inline for comments on what each command is doing.

Once that has been done and the vCD services has restarted, the SSL cert has been applied and we are all green and the Let’s Encrypt SSL cert is in play.

Released: vCloud Director 9.1.0.1 – API Tweaks and Resolved Issues

There was a point release of vCloud Director 9.1 (9.1.0.1 Build 8825802) released last week, bringing with it an updated Java Runtime plus new API functions that allow additional configuration of advanced settings for virtual machines. There was also a number of bug fixes from the initial 9.1 release earlier in the year. Some of the issues that are resolved are significant and worth looking into if you have 9.1 GA deployed.

I haven’t been able to find an exact list of the new API functions, however traversing the Org Admin rights API call I did spot something new relating to Latency as show below.

And when I granted this right through the API mechanism I was able to allocate the right to the Org Admin via the administrator web interface.

I’m trying get a list of all the new API rights that where added as part of this release and will update this post when I have them.

Some of the bigger issues that where resolved are listed below:

  • In vCloud Director Tenant Portal, the Configure Services tab is disabled for Advanced Edge Gateway. In vCloud Director Tenant Portal, you cannot configure Advanced Edge Gateway settings as an administrator with any of the Gateway Advanced Services rights.
  • When importing a virtual machine from vCenter Server, vCloud Director relocates it to the primary resource pool. When you import a virtual machine created on a non-primary cluster in vCenter Server to vCloud Director, the machine is always relocated to the primary cluster.
  • In the vCloud Director Tenant Portal, the administrator of one organization can see virtual machines that belong to other vCloud Director organizations. When you configure the organizations in vCloud Director to use an LDAP server for authentication, an administrator of one organization, who is logged in vCloud Director Tenant Portal, can see virtual machines that belong to other organizations.
  • Importing a virtual machine from the vCenter Server deletes the original virtual machine after cloning it. When importing a virtual machine from the vCenter Server to vCloud Director involves changing its datastore, the process consists in cloning the source virtual machine and deleting it, while effectively changing its Managed Object Reference (MoRef).
  • Enabling High Availability for existing edge gateways in a data center with installed NSX Edge 6.4.0 fails.  In a data center with installed NSX Edge 6.4.0, you cannot enable High Availability for existing edge gateways that belong to a datastore cluster with enabled Storage Distributed Resource Scheduler (SDRS).
  • vCloud Director Tenant Portal does not display existing organization virtual data centers. When you use a self-signed SSL certificate for vCloud Director and you log in to vCloud Director Tenant Portal, you do not see a list of the existing organization virtual data centers.

The rest can be found here.

Just to finish up, there is still a lingering issue from the GA release that changed the behaviour of the HTML5 Tenant UI in scenarios where the SSL self signed certificates are used which is covered in this VMwareKB. Even though (as shown above) it’s been listed as resolved…I have run into it again in two different installs.

Obviously, if you are using legit SSL certificates you won’t have the issue, however the work around is not doing it’s thing for me. Hopefully I can resolve this ASAP as I am about to start some validation testing for Veeam and vCloud Director as well as start to test out our new functionality coming in Update 4 of Backup & Replication for Cloud Connect Replication.

For those with the correct entitlements…download here.

#LongLivevCD

References:

https://docs.vmware.com/en/vCloud-Director/9.1/rn/rel_notes_vcloud_director_9-1-0-1.html

vBrownBag TechTalks at VMworld 2018 – The Power to Catapult!

VMworld 2018 is fast approaching and in the last 24 hours, notifications where sent out to those lucky enough to have their session submissions accepted. Having been on the wrong side of that email multiple times I understand the disappointment that comes with not having a session accepted. The great news about VMworld is that there is another way to get your session seen and heard…and that is through the vBrownBag Techtalks.

The TechTalks have been a staple at VMworld’s (and other industry conferences) for a number of years now. Last year saw a stepping up of the vBrownBag game by having the TechTalks listed in the VMworld Content Catalog. I’ve had the pleasure of presenting tech talks at three VMworld’s over the years. The first one was back in 2014 but I remember it being a significant milestone in my career…regardless of the fact it was just a TechTalk it meant a lot to present at VMworld.

Make no mistake…these talks have the power and potential to catapult careers!

While the TechTalks offer the opportunity for folks that have not had sessions accepted, the real power of the talks is in offering a platform for the community to step up and present relevant, thought leading content that generally isn’t driven by marketing. In many ways I see more value in these sessions than in the VMworld sessions proper and there is a lot that can be taken away from the sessions.

That said, it’s great to see a number of vendors sponsoring the TechTalks and as per usual, Veeam is leading the way in our support of community at VMworld. As of last week there where around 50 TechTalks submitted and the team expects to have space for over a hundred TechTslks between the both VMworld conferences.

There is still plenty of time to submit your session, more information is in this post.

Here is a Playlist of the 2017 VMworld TechTalks. For those interested, there is a blog post by the vBrownBag team on what it takes to get get a presentation up live streaming and onto YouTube so fast…I found it a fascinating read.

Released: NSX-v 6.4.1 New Features and Fixes

Last week VMware released NSX-v 6.4.1 (Build 8599035) that contains a some new features and addresses a number of resolved issues from previous releases. I will go through the new features in more detail below however a key mentions is the fact that vSphere 6.7 is now supported, also meaning the vCloud Director can now be used with NSX-v 6.4.1 fully supported on vSphere 6.7. Prior to that only 6.5 was supported by NSX-v meaning you couldn’t upgrade to vSphere 6.7 as vCloud Director is dependant on NSX-v which didn’t support 6.7 until this 6.4.1 release.

There is also a small, but cool automatic backup feature introduced that backs up the state of the NSX Manager locally prior to the upgrade. Going through the release notes there are a lot of known issues that should be looked at and there are more than a few that apply to service providers.

The NSX User Interface continues to be enhanced and additional components added to the HTML5 Web Client. As you can see below, there are a lot more options in the HTML5 Web Client compared to the 6.4 base release…to reference that version menu, click here.

NSX User Interface

As you can see, the following VMware NSX features are now available through the HTML5 vSphere Client. Installation, Groups and Tags, Firewall, Service Composer, Application Rule Manager, SpoofGuard, IPFIX and Flow Monitoring. VMware is maintaining a web page that show the current NSX for vSphere UI Plug-in Functionality.

Other enhancements to the User Interface include:

  • Firewall – UI Enhancements:
    • Improved visibility: status summary, action toolbar, view of group membership details from firewall table
    • Efficient rule creation: in-line editing, clone rules, multi-selection and bulk action support, simplified rule configuration
    • Efficient section management: drag-and-drop, positional insert of sections and rules, section anchors when scrolling
    • Undo operations: revert unpublished rule and section changes on UI client side
    • Firewall Timeout Settings: Protocol values are displayed at-a-glance, without requiring popup dialogs.
  • Application Rule Manager – UI Enhancements:
    • Session Management: View a list of sessions, and their corresponding status (collecting data, analysis complete) and duration.
    • Rule Planning: View summary counts of grouping objects and firewall rules; View recommendations for Universal Firewall Rules
  • Grouping Objects Enhancements:
    • Improved visibility of where the Grouping Objects are used
    • View list of effective group members in terms of VMs, IP, MAC, and vNIC
  • SpoofGuard – UI Enhancements:
    • Bulk action support: Approve or clear multiple IPs at a time

I really like how the HTML5 interface is coming along and i’m now using it as my primary tool over the Flex interface.

Other New Enhancements:

Looking at Security Services are improvements in the Firewall by way of additional layer 7 application context support for Symantec LiveUpdate Traffic, MaxDB SQL Server support and support for web based Git or version control. There is also extended support via the Identity Firewall for user sessions on RDP and application server which now covers Server 2012 and 2012 R2 with specific VMTool versions.

The NSX Load Balance now scales to 256 pool members up from 32 which is a significant enhancement to an already strong feature of the NSX Edges. There are also a number of enhancements to overall operations and troubleshooting pages.

Those with the correct entitlements can download NSX-v 6.4.1 here.

Special Upgrade and Supportability Notes:

  • vSphere 6.7 support: When upgrading to vSphere 6.7, you must first install or upgrade to NSX for vSphere 6.4.1 or later. See Upgrading vSphere in an NSX Environment in the NSX Upgrade Guide and Knowledge Base article 53710 (Update sequence for vSphere 6.7 and its compatible VMware products).
  • NSX for vSphere 6.1.x reached End of Availability (EOA) and End of General Support (EOGS) on January 15, 2017. (See also VMware knowledge base article 2144769.)
  • NSX for vSphere 6.2.x will reach End of General Support (EOGS) on August 20 2018.

References:

https://docs.vmware.com/en/VMware-NSX-for-vSphere/6.4/rn/releasenotes_nsx_vsphere_641.html

 

Quick Fix: vSAN Health Reports iSCSI Target Service Stopped

A few weeks ago I wrote about using iSCSI as a backup repository target. While still running this POC in my environment I came across an error in the vSAN Health Checker stating the vSAN iSCSI target service was in a Failed state. Drilling down into the vSAN Health check tree I could see a Service Runtime status of stopped as shown below against the host.

This host had recently been marked as unreachable in vCenter and required a Management Agent reset to bring it back online. There is a chance that that process stopped the iSCSI Target service but did not start it. In any case there is an easy way to see the status of the services and then get them back online.

Once that’s been done, a re-run of the vSAN Health checker will show that the issue has been resolved and the iSCSI Target Service on the host is now running.

References:

https://kb.vmware.com/s/article/2147603

 

« Older Entries Recent Entries »