Monthly Archives: April 2019

Infrastructure as Code vs RESTful APIs … A Working Example with Terraform and vCloud Director

Last week I wrote an opinion piece on Infrastructure as Code vs RESTful APIs. In a nutshell, I talked about how leveraging IaC instead of trying to code against APIs directly can be more palatable for IT professionals as it acts as a middle man interpreter between yourself and the infrastructure endpoints. IaC can be considered a black box that does the complicated lifting for you without having to deal with APIs directly.

As a follow up to that post I wanted to show an example about the differences between using direct APIs verses using an IaC tool like Terraform. Not surprisingly the example below features vCloud Director…but I think it speaks volumes to the message I was trying to get across in the introduction post.

The vCloud Director Terraform Provider was recently upgraded with the release of vCloud Director 9.7 and now sits at version 2.1 itself.

The Terraform Provider has been developed using Python and GO. It uses Client-Server model inside the hood where the client has been written using GO and server has been written using Python language. The core reason to use two different languages is to make a bridge between Terraform and Pyvcloud API. Pyvcloud is the SDK developed by VMware and provides an medium to talk to vCloud Director. Terraform uses GO to communicate where Pyvcloud has been written in Python3.

The above explanation as to how this provider is doing its thing highlights my previous points around any IaC tools. The abstraction of the infrastructure endpoint is easy to see… and in the below examples you will see it’s benefit for those who have not got the inclination to hit the APIs directly.

The assumption for both examples is that we are starting without any configured Firewall or NAT rules for the NSX Edge Services Gateway. Both methods are connecting as tenant’s of the vCD infrastructure and authenticating with Organisation level access.

The end result will be:

  • Allow HTTP, HTTPS and ICMP access to a VM living in a vDC
    • External IP is 82.221.98.109
    • Internal IP of VM is 172.17.0.240
    • VM Subnet is 172.17.0.0/24
  • Configure DNAT rules to allow HTTP and HTTPS
  • Configure SNAT rule to allow outbound from the VM subnet
Configuring Firewall and NAT Rules with RESTful API:

Firstly, to understand what vCD API operations need to be hit, we need to be familiar with the API Documentation. This will cover initial authentication as either a SYSTEM or Organizational admin and then what calls need to be made to get information relating to the current configuration and schema. Further to this, we need to also be familiar with the NSX API for vCD Documentation which covers how to interact with the network specific API operations possible from the vCD API endpoint.

We are going to be using Postman to execute against the vCD API. Postman is great because you can save your call history and reuse them at a later date. You can also save variable into Workspaces and also insert specific code to assist with things like authentication.

First step is to authenticate against the API and get a session authorization key that will allow you to feed that key back into subsequent requests. This authorization key will only last you a finite amount of time and will need to be regenerated.

Because we are using a decent RESTful API Client like Postman, there is a better way to programatically authenticate using a bearer access token as described in Tom Fojta’s post here when talking to the vCD API.

Once that is done we are authenticated as a vCD Organizational Admin and we can now query the NSX Edge Services Gateway (ESG) Settings for Firewall and NAT rules. I’ll walk through configuring a NAT rule for the purpose of the example, but the same method will be used to configure the Firewall as well.

A summary of the NAT requests can be seen below and found here.

Below we are querying the existing NAT rules using a GET request against the NSX ESG. What we are returned is an empty config in XML.

What needs to be done is to turn that request into a POST and craft an XML payload into the Body of the request so that we can configure the NAT rules as desired.

Redoing the GET request will now show that the NAT rules have been created.

And will be shown in the vCD Tenant UI

From here we can update, append, reset or delete the NAT rules as per the API documentation. Each one of those actions will require a new call to the API and the same process followed as above.

Configuring Firewall and NAT Rules with Terraform:

For a primer on the vCloud Director Terraform Provider, read this post and also head over to Luca’s post on Terraform with vCD. As with the RESTful API example above, I will use Terraform IaC to configure the same Tenant NSX Edge Gateway’s Firewall and NAT rules. What will become clear using Terraform for this is that it is a lot more efficient and elegant that going at it directly against the APIs.

Initially we needs to setup the required configuration items in order for the Terraform Provider to talk to the vCD API endpoint. To do this we need to setup a number of Terraform files that declare the variables required to connect to the vCD Organization and then configure the terraform.tfvars file that contains the specific variables.

We also create a provider .tf file to specifically call out the required Terraform provider and set the main variables.

We contain all this in a single folder (seen in the left pane above) for organization and portability…These folders can be called as Terraform Modules if desired in more complex, reusable plans.

We then create two more .tf files for the Firewall and NAT rules. The format is dictated by the Provider pages which gives examples. We can make things more portable by incorporating some of the variables we declared elsewhere in the code as shown below for the Edge Gateway name and Destination IP address.

Once the initial configuration work is done, all that’s required in order to apply the configuration is to initialize the Terraform Provider, make sure that the Terraform Plan is as expected… and then apply the plan against the Tenant’s Organization.

As the video shows… in less than a minute we have the NSX Firewall and NAT rules configured. More importantly, we now have a desired state which can be modified at any time by simple additions or subtractions to the Terraform code.

Wrapping it up:

From looking at both examples, it’s clear that both methods of configuration do the trick and it really depends on what sort of IT Professional you are in terms of which method is more suited to your day to day. For those that are working as automation engineers, working with APIs directly and/or integrating them into provisioning engines or applications is going to be your preferred method. For those that want to be able to deploy, configure and manager their own infrastructure in a more consumable way, using a Terraform provider is probably a better way

The great thing about Terraform in my eyes is the fact that you have declared the state that you want configured and once that has been actioned, you can easily check that state and modify it by changing the configuration items in the .tf files and reapplying the plan. For me it’s a much more efficient way to programatically configure vCD than doing the same configuration directly against the API.

Ohhh… and don’t forget… you are still allowed to use the UI as well… there is no shame in that!

Infrastructure as Code vs RESTful APIs …Terraform and Everything in Between!

While I was a little late to the game in understanding the power of Infrastructure as Code, I’ve spent a lot of the last twelve months working with Terraform specifically to help deploy and manage various types of my lab and cloud based infrastructure. Appreciating how IaC can fundamentally change the way in which you deploy and configure infrastructure, workloads and applications is not an easy thing to grasp…there can be a steep learning curve and lots of tools to choose from.

In terms of a definition as to what is IaC:

Infrastructure as code (IaC) is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. The IT infrastructure managed by this comprises both physical equipment such as bare-metal servers as well as virtual machines and associated configuration resources. The definitions may be in a version control system. It can use either scripts or declarative definitions, rather than manual processes, but the term is more often used to promote declarative approaches.

As represented above, there are many tools that are in the IaC space and everyone will gravitate towards their own favourite. The post where I borrowed that graphic from actually does a great job or talking about the differences and also why Terraform has become my standout for IT admins and why Hashicorp is on the up. I love how the article talks about the main differences between each one and specifically the part around the Procedural vs Declarative comparison where it states that declarative approach is where “you write code that specifies your desired end state, and the IcC tool itself is responsible for figuring out how to achieve that state.”

You Don’t Need to Know APIs to Survive!:

The statement above is fairly controversial… especially for those that have been preaching about IT professionals having to code in order to remain viable. A lot of that mindshare is centred around the API and the DevOps world…but not everyone needs to be a DevOp! IT is all about trying to solve problems and achieve outcomes… it doesn’t matter how you solve it… as long as the problem/outcome is solved/attained. Being as efficient as possible is also important when achieving that outcome.

My background prior to working with IaC tools like Terraform was working with and actioning outcomes directly against RESTFul APIs. I spent a lot of time specifically with vCloud Director and NSX APIs in order to help productise services in my last two roles so I feel like I know my way around a cURL command or Postman window. Let me point out that there is nothing wrong with having knowledge of APIs and that it is important for IT Professionals to understand the fundamentals of APIs and how they are accessed and used for programatic management of infrastructure and for creating applications.

I’m also not understating the skill that is involved in being able to understand and manipulate APIs directly and also being able to take those resources and create automated provisioning or actual applications that interact directly with APIs and create an outcome of their own. Remembering though that everyones skill set and level is different, and no one should feel any less an IT practitioner if they can’t code at a perceived higher level.

How IaC Tools Bridge the Gap:

In my VMUG UserCon session last month in Melbourne and Sydney I went through the Veeam SDDC Deployment Toolkit that was built with various IaC tooling (Terraform and Chef) as well as PowerShell, PowerCLI and some Bash Scripting. Ultimately putting all that together got us to a point where we could declaratively deploy a fully configured Veeam Backup & Replication server and fully configure it ready for action on any vSphere platform.

That aside, the other main point of the session was taking the audience through a very quick Terraform 101 introduction and demo. In both cities, I asked the crowd how much time they spent working with APIs to do “stuff” on their infrastructure… in both cities there was almost no one that raised their hands. After I went through the basic Terraform demo where I provisioned and then modified a VM from scratch I asked the audience if something like this would help them in their day to day roles… in both cities almost everyone put their hands up.

Therein lies the power of IaC tools like Terraform. I described it to the audience as a way to code without having to know the APIs directly. Terraform Providers act as the middle man or interpreter between yourself and the infrastructure endpoints. Consider it a black box that does the complicated lifting for you… this is the essence of Infrastructure as Code!

There are some that may disagree with me (and that’s fine) but I believe that for the majority of IT professionals that haven’t gotten around yet into transitioning away from “traditional” infrastructure management, configuration and deployment, that looking at a IaC tools like Terraform can help you not only survive…but also thrive!

References:

https://blog.gruntwork.io/why-we-use-terraform-and-not-chef-puppet-ansible-saltstack-or-cloudformation-7989dad2865c

https://en.wikipedia.org/wiki/Infrastructure_as_code

Released: Backup for Office 365 3.0 …Yes! You Still Need to Backup your SaaS

A couple of weeks ago of Veeam Backup for Office 365 version 3.0 (build 3.0.0.422) went GA. This new version builds on the 2.0 release that offered support for SharePoint and OneDrive as well as enhanced self service capabilities for Service Providers. Version 3.0 is more about performance and scalability as well as adding some highly requested features from our customers and partners.

Version 2.0 was released last July and was focused on expansed the feature set to include OneDrive and SharePoint. We also continued to enhanced the automation capability of the platform through a RESTful API service allowing our Cloud & Service Providers to tap into the APIs to create scaleable and efficient service offerings. In version 3.0, there is also an extended set of PowerShell commandlets that have been enhanced from version 2.0.

What’s New in 3.0:

Understanding how best to deal with backing up SaaS based services where a lot of what happens is outside of the control of the backup vendor, there where some challenges around performance with the backing up and restoring of SharePoint and OneDrive in version 2.0. With the release of version 3.0 we have managed to increase the performance of SharePoint and OneDrive incremental backups up to 30 times what was previously seen in 2.0. We have also added support for multi-factor authentication which was a big ask from our customers and partners.

Other key enhancements for me was some optimisations around the repository databases that improves space efficiencies, auto-scaling of repository databases that enable easier storage management for larger environments by overcoming the ESE file size limit of 64 TB. When the limit is reached, a new database will be created automatically in the repository which stops manual intervention.

Apart from the headline new features and enhancements there are also a number of additional ones that have been implemented into Backup for Microsoft Office 365 3.0.

  • Backup flexibility for SharePoint Online. Personal sites within organisations can now be excluded or included from/to a backup in a bulk.
  • Flexible protection of services within your Office 365 organization, including exclusive service accounts for Exchange Online and SharePoint Online.
  • Built-in Office 365 storage and licensing reports.
  • Snapshot-based retention  which extends the available retention types.
  • Extended search options in the backup job wizard that make it is possible to search for objects by name, email alias and office location.
  • On-demand backup jobs to create backup jobs without a schedule and run them upon request.
  • The ability to rename existing organizations to keep a cleaner view on multiple tenant organizations presented in the console

For another look at what’s new, Niels Engelen goes through his top new features in detail here and for service providers out there, it’s worth looking at his Self Service Portal which has also been updated to support 3.0.

Architecture and Components:

 

There hasn’t been much of a change to the overall architecture of VBO and like all things Veeam, you have the ability to go down an all in one design, or scale out depending on sizing requirements. Everything is handled from the main VBO server and the components are configured/provisioned from here.

Proxies are the work horses of VBO and can be scaled out again depending on the size of the environment being backed up. Again, this could be Office 365 or on-premises Exchange or SharePoint instances.

Repositories must be configured on Windows formatted volumes as we use the JetDB database format to store the data. The repositories can be mapped one to one to tenants, or have a many to one relationship.

Installation Notes:

You can download the the latest version of Veeam Backup for Microsoft Office 365 from this location. The download contains three installers that covers the VBO platform and two new versions of the Explorers. Explorer for Microsoft OneDrive for Business is contained within the Explorer for Microsoft SharePoint package and installed automatically.

  • 3.0.0.422.msi for Veeam Backup for Microsoft Office 365
  • 9.6.5.422.msi for Veeam Explorer for Microsoft Exchange
  • 9.6.5.422.msi for Veeam Explorer for Microsoft SharePoint

To finish off…It’s important to read the release notes here as there are a number of known issues relating to specific situations and configurations.

Backup for Office 365 has been a huge success for Veeam with a growing realisation that SaaS based services require an availability strategy. The continuity of data on SaaS platforms like Office 365 is not guaranteed and it’s critical that a backup strategy is put into place.

Links and Downloads:

Cloud Field Day 5 – Recap and Videos #CFD5

Last week I had the pleasure of presenting at Cloud Field Day 5 (a Tech Field Day event). Joined by Michael Cade and David Hill, we took the delegates through Veeam’s cloud vision by showcasing current product and features in the Veeam platform including specific technology that both leverages and protects Public Cloud workloads and services. We also touched on where Veeam is at in terms of market success and also dug into how Veeam enables Service Providers to build services off our Cloud Connect technology.

First off, I would like to thank Stephen Foskett and the guys at Gestalt IT for putting together the event. Believe me there is a lot that goes on behind the scenes and it is impressive how the team are able to setup, tear down and setup agin in different venues while handling the delegates themselves. Also to all the delegates, it was extremely valuable being able to not only present to the group, but also have a chance to talk shop at the offical reception dinner…some great thought provoking conversations where had and I look forward to seeing where your IT journey takes you all next!

Getting back to the recap, i’ve pasted in the YouTube links to the Veeam session below. Michael Cade has a great recap here, where he gives his overview on what was presented and some thoughts about the event.

In terms of what was covered:

We tried to focus on core features relating to cloud and then show a relatable live demo to reinforce the slide decks. No smoke and mirrors when the Veeam Product Strategy Team is doing demos… they are always live!

For those that might not have been up to speed with what Veeam has done over the past couple of years it is a great opportunity to learn about what we have done innovating the Data Protection space, while also looking at the progress we have made in recent times in transitioning to a true software defined, hardware agnostic platform that offers customers absolute choice. We like to say that Veeam was born in the virtual world…but is evolving in the Cloud!

Summary:

Once again, being part of Cloud Field Day 5 was a fantastic experience, and the team executed the event well. In terms of what Veeam set out to achieve, Michael, David and myself where happy with what we where able to present and demo and we where happy with the level of questions being asked by the delegates. We are looking forward to attending Tech Field Day 20 later in the year and maybe as well as continue to show what Veeam can do today…take a look at where we are going in future releases!

References:

Cloud Field Day 5

Disaster Recovery and Resiliency with Veeam Cloud Tier

Yesterday at Cloud Field Day 5, I presented a deep dive on our Cloud Tier feature that was released as a feature for Scale Out Backup Repository (SOBR) in Veeam Backup & Replication Update 4. The section went through an overview of its value proposition as well as deep dive into how we are tiering the backup data into Object Storage repositories via the Capacity Tier Extend of a SOBR. I also covered the space saving and cost saving efficiencies we have built into the feature as well as looking at the full suite of recoverability options still available with data sitting in an Object Storage Repository.

This included a live demo of a situation where a local Backup infrastructure had been lost and what the steps would be to leverage the Cloud Tier to bring that data back at a recovery site.

Quick Overview of Offload Job and VBK Dehydration:

Once a Capacity Tier Extent has been configured, the SOBR Offload Job is enabled. This job is responsible for validating what data is marked to move from the Performance Tier to the Capacity Tier based on two conditions.

  1. The Policy defining the Operational Restore Window
  2. If the backup data is part os a sealed backup chain

The first condition is all about setting a policy on how many days you want to keep data locally on the SOBR Performance Tiers which effectively become your landing zone. This is often dictated by customer requirements and now can be used to better design a more efficient approach to local storage with the understanding that the majority of older data will be tiered to Object storage.

The second is around the sealing of backup chains which means they are no longer under transformation. This is explained in this Veeam Help Document and I also go through it in the CFD#5 session video here.

Once those conditions are met, the job starts to dehydrate the local backup files and offload the data into Object Storage leaving a dehydrated shell with only the metadata.

The importance of this process is that because we leave the shell locally with all the metadata contained, we are able to still perform every Veeam Recovery option including Instant VM Recovery and Restore to Azure or AWS.

Resiliency and Disaster Recovery with Cloud Tier:

Looking at the above image of the offload process you can see that the metadata is replicated to the Object Storage as well as the Archive Index which keeps track of which blocks are mapped to what backup file. In fact for every extent we keep a resilient copy of the archive index meaning that if an extent is lost, there is still a reference.

Why this is relevant is because it gives us disaster recovery options in the case of a loss of whole a whole backup site or the loss of an extent. During the synchronization, we download the backup files with metadata located in the object storage repository to the extents and rebuild the data locally before making it available in the backup console.

After the synchronization is complete, all the backups located in object storage will become available as imported  jobs and will be displayed under the Backups and Imported in the inventory pane. But what better way to see this in action than a live demo…Below, I have pasted in the Cloud Field Day video that will start at the point that I show the demo. If the auto-start doesn’t kick in correctly the demo starts at the 31:30 minute mark.

References:

https://helpcenter.veeam.com/docs/backup/vsphere/capacity_tier_offload_job.html?ver=95u4

Released: Veeam Availability Console v3.0 – Reseller, Licensing and Scalability Enhancements!

A couple of weeks ago, Veeam Availability Console v3.0 (3.0.0.2647) was released. With this major update, VAC is now the central Console providing management and monitoring of Veeam Cloud and Service Provider partner offerings. Not only does it build on the previous releases, but also looks to place VAC as a critical component of any Veeam powered service provider offering. In addition to the existing features the enhancements are a direct response to product feedback and also reduces a lot of the pain points that VCSPs have in managing and monitoring their own partners and customers.

What’s new in v3.0:

If you want to get the complete low down on what’s included in this release, the What’s new document can be accessed here. I’ve listed the key new features and enhancements below and expanded on them in the main body of the post.

  • Reseller Role for more granular access and control
  • Multiple Cloud Connect server support
  • Enhanced licensing management and rental usage reporting
  • Enhanced RESTful APIs
Reseller Role:

Since the introduction of Veeam’s Backup & Management Portal back in 2016 the desire for a reseller tier was top of list for our VCSPs and while it’s taken a few more iterations of the product for it to appear as a feature the addition of the Reseller Role in VAC v3 is significant. This allows the top level VCSP to carve out reseller accounts which allows their reseller partners to manage, monitor and control their own customers.

We also have the ability now to allocate Resellers Site Scopes. Site Scopes are based on Cloud Connect Service instances of which now in v3 we can add multiples (see below).

Resellers have control over their customers and can sub allocate backup resources that have been allocated to them from the VCSP. They also have the power to remotely manage on-premises Backup & Replication servers including the ability to install licenses (see below) which is a huge addition to VACs feature capability. This alone is worth the price of admission!

The Reseller Console contains almost all administrative functions including the ability to brand the portal but obviously does not have access to core Cloud Connect infrastructure or the ability to manage other important VCSP configuration items.

Multiple Cloud Connect Servers and Increased Scalability:

Again, one of the biggest pieces of feedback we received from VCSPs was for a single VAC instance to be able to be connected to multiple Cloud Connect servers. This effectively removes the one to one relationship and have a one to many feature that truely allows VAC to be a central point of management.

Connected Cloud Connect Servers become named Site Scopes which can then be used for Reseller management at a more granular level. In adding the ability to connect to multiple Cloud Connect Servers the scalability of a single VAC instance has been increased to connect to more remote Backup & Replication servers (up to 1000) and manage more Veeam agents (up to 15,000). At the moment VCSPs can add up to 50 Cloud Connect instances which should be more than enough to cover the majority of installs• 50 Veeam Cloud Connect servers

A single VAC instance can also support 500 resellers and up to 1000 active portal users. In line with this there is also a new User Role that allows more granular access to specific Cloud Connect Servers that can be managed by VCSP technical staff.

Enhanced Licensing Management and Reporting:

No one has ever said that managing licenses is fun! It’s a necessary evil in the software world and can always be frustrating. When looking at managing customer licenses for on-premises installs under management that frustration can be multiplied almost exponentially. I would be selling the new licensing features in VAC v3 short if I didn’t say that it was my favourite feature enhancement and almost the single reason why VAC should be installed by all VCSPs.

From the Configuration menu under Licensing there is now the ability to view Cloud Connect and all Backup Servers under management. The view above is the top level VCSP view… a Reseller will see only Backup Servers under their own management, though the functionality is the same.

The Install action can be used to remotely deploy and install a Veeam License either directly to the connected Cloud Connect Servers, or remotely via the Cloud Connect Gateway tunnels to on-premises Backup servers. There is also reporting on the current status of the existing licence and alerts can be configured  to notify VCSPs, Resellers or Customers about an impending license expiration. All this can also be done via new API calls.

In addition to the management of the licenses the reporting has been enhanced to the point where VAC will become the source of truth for all license reporting. The Monthly Usage Report are generated from all connected Backup and Cloud Connect Servers and will produce reports that are downloadable for the VCSP or Reseller while also being available via the API.

Enhanced RESTful APIs:

The RESTful APIs in VAC v3 have been enhanced to include actions to configure and manage areas of the VAC install. Previously the API was used for pulling data and reading configuration items, however there has been a number of actionable items added in this release. Importantly these new POST, PUT and PATCH requests marry up with most of the new features listed above.

Again, the ability to deploy and install license to remote Backup Servers via Cloud Connect from an API is important and I know this will be hugely popular with our VCSPs who have automation capabilities.

Platform Supportability:

This release also delivers full support for all recently shipping Veeam products including Veeam Backup & Replication 9.5 Update 4 (Including Cloud Connect enhancements and vCloud Director support and integration) as well as Veeam Agent forMicrosoft Windows 3.0 and the new ability to create multiple jobs. There are also enhancements to support, Windows Event Logging and notifications while also increasing security.

References and Product Guides:

https://www.veeam.com/vac_3_0_release_notes_rn.pdf

https://www.veeam.com/availability-console-service-providers-faq.html

https://www.veeam.com/veeam_vac_3_0_whats_new_wn.pdf

https://www.veeam.com/blog/managed-backup-service-enhancement.html

Heading to Cloud Field Day 5 #CFD5

I’m currently on the first leg over from Perth to San Fransisco where I’ll head down to Silicon Valley to present at Cloud Field Day 5. This will be my first Tech Field Day event which some may find surprising given my involvement in and around the virtualisation community prior to joining Veeam. I often got asked why I never applied to be a delegate… to be honest I’m not able to answer that question, however I’m really looking forward to presenting with my fellow Product Strategy team members, Michael Cade and David Hill as a sponsor.

cloud field day

Veeam at Tech Field Day Cloud 5

It’s an important event for us at Veeam as we are given the #CFD stage for two hours, first up on the Wednesday morning to talk about how Veeam has evolved from a backup vendor covering vSphere and Hyper-V to one which now offers a platform that extends into the public cloud backed by an incredibly strong Cloud and Service Provider ecosystem.

There is no doubt that the backup industry is currently hot with a number of large investments made into a number of vendors, including us here at Veeam who recently had a $500 million injection for us to pursue acquisitions and increase our development capabilities. There are a number of backup vendors presenting at the CFD#5, including vendors who are not historically backup focused but now, more and more offering inbuilt protection.

For Michael, David and myself we will be focusing on reiterating what Veeam has done in leading the industry for a number of years in innovation while also looking at the progress we have made in recent times in transitioning to a true software defined, hardware agnostic platform that offers customers absolute choice.

Veeam are presenting at 8.30am (Pacific Time) Wednesday 10th April 2019

I am looking forward to presenting to all the delegates as well as those who join via the livestream.

Quick Fix: Upgrading to vCloud Director 9.7 fails During Database Script Updates

While looking to upgrade one of my vCloud Director instances I came across an error during the database upgrade process which is the second step of updating vCloud Director from version to version. I must admit, it the eight to nine years of using vCloud Director, this was the first time that had an error during this process… I was kind of shocked!

Unable to upgrade the database: java.lang.IllegalStateException: Exception encountered while altering idle transaction session timeout in vcloud database:

This was upgrading a PostGreSQL 9.5 database in a single cell all in one setup. Initially I was going from 9.1.x to 9.7 so I decided to roll back and try a 9.5 upgrade first. That worked without issue, however the subsequent upgrade from 9.5 to 9.7 also failed with the same error. Talking in the VMware Cloud Provider Slack Channel I got a few pointers to permissions and/or the version of PostGreSQL being not supported by 9.7.

Looking at the supportability Matrix for vCloud Director against supported databases we see:

vCloud Director 9.7 only supports MSSQL 2017 (last release that will support MSSQL) and PostGreSQL 10 which suggests 10.x. I was running PostGreSQL 9.5, so decided to upgrade to 10.7 using this guide from Yves Sandfort.

After upgrading to 10.7 I tried the upgrade again but still got the same error. Because of the fact that the same instance upgraded successfully from 9.1 to 9.5 I didn’t really consider the database permissions to be a problem so continued to investigate with the help of VMware Support. What we found was an exception that mentioned ownership of the vcloud database from the current user which is vcloud.

Sure enough, logging into PostgreSQL admin it showed that the existing owner of the vcloud database was postgreS its self.

Strangely for me, during the PostgreSQL upgrade and migration the ownership of the database did not carry across. After changing the ownership back to the vcloud user the upgrade worked.

Take Away:

There seems to be two potential triggers for the error during the database upgrade:

  • Being on a non supported version of PostgreSQL (not 10x)
  • Not having the correct ownership permissions against the vcloud database

So if that error pops up during an upgrade to vCloud Director 9.7 check either of the above (or both) and give it another shot.

Update 4 for Service Providers – Targeting vCloud Director with Cloud Connect Replication

When Veeam Backup & Replication 9.5 Update 4 went Generally Available in late January I posted a What’s in it for Service Providers blog. In that post I briefly outlined all the new features and enhancements in Update 4 as it related to our Veeam Cloud and Service Providers. As mentioned each new major feature deserves it’s own seperate post. I’ve covered off the majority of the new feature so far, and for the final post in the series I am looking at something that is close to my heart…vCloud Director Support for Veeam Cloud Connect Replication.

As a reminder here are the top new features and enhancements in Update 4 for VCSPs.

Leveraging the Best of vCloud Director for Stronger DRaaS:

VMware vCloud Director is the de facto standard for service providers who offer Infrastructure as a Service based on VMware and Veeam has had a long history supporting vCloud Director, with the industry’s first support for vCloud Director aware backups released in Veeam Backup & Replication v7 following on with the first stand alone Self Service Backup Portal in v9.5.

With the release of Update 4, we have added support for Veeam Cloud Connect to replicate directly into vCloud Director virtual datacenters, allowing both our Cloud and Service Provider Partners (VCSP) and customers to take advantage of the enhancements VMware has built into the platform. While this has been a long time coming, this support represents a significant enhancement to the way in which our VCSPs offer DRaaS.

With tenants consuming vCloud Director resources, it allows them to take advantage of more powerful features when dealing with full disaster, or the failure of individual workloads. Full and partial failovers will be more transparent with the use of the vCloud Director HTML5 Tenant UI and the vCloud Director HTML5 UI will also allow tenants to see what is happening to workloads as they boot and interact with the guest OS directly. This takes the pressure of the VCSPs helpdesk and gives tenants more control of their replicas once failed over.

Enhanced Edge Networking with NSX:

From a networking point of view, being able to access the NSX Edge Gateway for replicated workloads means that tenants can leverage the advanced networking features available on the NSX Edge Gateway. The Network Extension Appliance did a great job in offering basic network functionality however the NSX Edge offers:

  • Advanced Firewalling and NAT
  • Advanced Dynamic Routing (BGP, OSPF and more)
  • Advanced Load Balancing
  • IPsec and L2VPN
  • SSL VPN
  • SSL Certificate Services

Once a failover has been triggered from the Veeam Backup & Replication Console or via the VCSPs own Portals, the ability to manage and configure everything through the vCloud Director HTML5 UI utilizing NSX via vCloud Director enhances Cloud Connect Replication for both service providers and tenants.

Network Automation During Partial Failovers with the NEA:

There are a number of options that can be used to extend the tenant network to the service provider cloud network when actioning a partial failover. Tenants and service providers can configure:

  • Custom IPsec VPN
  • IPsec or L2VPN via the NSX Edge Gateway
  • NEA to NEA L2 VPN

The Network Extension Appliance is still available for deployment in the same way as before Update 4 and can be used directly from within a vCloud Director virtual datacenter. The NEA’s automate the extension of a tenant network so that the failed over workload can be accessible from the tenant network, even though it resides in the service provider’s environment. The NEA to NEA option is the simplest and most effective way to extend the tenants network to the cloud network.

NOTE: I will be dedicating a seperate blog post to this feature as I believe this is one of the leading innovative features we have as part of Cloud Connect Replication.

vCloud Director 9.7 Compatibility:

Just a quick note to finish that at the time of writing this post, Veeam Backup & Replication 9.5 Update4a does not officially support vCloud Director 9.7. We currently support up to vCloud Director 9.5 but will be looking to add supportability for 9.7 within the next 90 days.

Wrap Up:

DRaaS is something that is only just becoming recognized as something that organizations require as part of their overall data protection strategy. Veeam has had a strong offering delivered through our VCSPs for a while now, with a strong focus on network automation which is typically the hardest part of any DRaaS offering. With Cloud Connect Replication now targeting vCloud Director we now have a very compelling DRaaS product that is simple, flexible and reliable…yet still delivers enterprise grade functionality.

Quick Fix – Incompatible Veeam Backup for Office 365 Server Version

This week Veeam dropped version 3.0 of Backup for Microsoft Office 365, which represents another significant update to the SaaS backup platform and builds on the previous 2.0 and 1.5 releases. For a quick look at some of the highlights, head to my fellow Technologist, Niels Engelen blog post for an overview. Like many out there i’ve been waiting patiently to install the GA and got things updated without any issues…however when looking to browse existing backup points for my Office 365 mailboxes I came across this error.

Incompatible Veeam Backup for Office 365 Server Version (received: 9.6.5.422, expected: 9.6.4.1078).

This is after the Veeam Explorer for Microsoft Exchange has been loaded and it tried to connect to the VBO server. The error is a little misleading in that it’s actually talking about the version of the Explorer rather than the VBO server its self.

If you look inside the VBO v3 downloaded zip file that you will see three installers.

The simple fix is to install the new version of the explorers. The dead giveaway is the new splash screen as seen below.

Once done, relaunching the Explorer session will success and you will be able to see the backed up mailboxes listed.

So there you go… a really simple fix to an error that might stump a few people at first!

« Older Entries