Category Archives: Backup

Veeam Availability Console now available from Azure Marketplace

Last week the Veeam Availability Console Azure Marketplace appliance went live. This allows Veeam Cloud and Service Providers to easily deploy VAC into any Azure region. In it’s previous incarnation the Managed Backup Portal was only available as an Azure marketplace appliance and not available to install by a VCSP. Now that VAC 2.0 is out, VCSPs who don’t have the ability to host Cloud Connect or VAC on their infrastructure can deploy it in Azure and have the service up and running within fifteen minutes.

There are some limitations that come along with deploying VAC into Azure and it won’t be for everyone. The biggest caveat is that you can only have one Cloud Connect Server per VAC instance and as part of the deployment, Cloud Connect services is installed on the same Virtual Machine. You can’t offer Replication services from the Azure instance, and if offering Cloud Connect backup you need to understand it’s own scalability and performance bottlenecks. That said, as a remote management, monitoring, reporting, billing and self service platform there is a lot to like about having VAC in Azure.

Marketplace Deployment Steps:

You can start the deployment by searching for Veeam Availability Console in the Azure Marketplace or you can go direct to the product page here.

Click on Create to start the configuration steps.

The Basics includes VM name, hard disks type, username and password as well as selecting the subscription, the ability to use a new or existing resource group and finally the Azure location you want to deploy into.

In Step 2 you need to choose the Size of the Azure instance. The template provides the recommended configurations. The sizes are relative to the amount of agents and/or Backup & Replication instances you are going to be managing from this instance. You can find sizing guides here for larger environments.

I ended up going with an A2 standard for my instance which removes the load balancing functionality from the configuration and offers a little less IOPS. Step 3 contains some optional extra’s to ensure a higher level of availability for the VM instance and lets you configure the networking. Once that’s done you can review your configuration settings and start the deployment. It took just over 8 minutes for the deployment to succeed.

If you click on the Virtual Machine object in the Azure Portal you will see an overview of the VM and it’s configuration.

Addition Azure Configuration:

If you notice in the image above, a DNS name is listed in the overview. This was something that I had to set manually after the deployment. You set this by going into the Networking of the resource pool and click on IP Configuration. Here, you can enter in a DNS name relative to the Azure zone you are in. You can then use this to connect to the VAC Console, Cloud Connect Service and to RDP to the VM and helps in the event of having a dynamic, rather than a static Azure IP.

Speaking of networking and ports, below is a list of the default port rules created during the deployment. Note that WinRM is open as well.

Finalizing Deployment:

After deploying the Azure Marketplace appliance you can RDP into the VM and complete the setup that includes configuring Cloud Connect and VAC it’s self. A few things have been done for us as part of the deployment, however the first thing you need to do is get a license. This is a BYO license situation, so once you have deployed the Marketplace appliance you will need to source a VAC license from the Veeam Licensing Portal and apply.

Head to the VAC Web Portal and Install the License.

Once done the last step is to configure Cloud Connect from the Backup & Replication Console. Again, you will need a valid Cloud Connect license as you are greeted with the Free Edition when you connect to the console for the first time. As per normal with Cloud Connect, you need to configure the SSL Certificate first and then configure a new Cloud Gateway. Configure the Networking as shown below using the DNS name that was created in the steps above.

Once this is completed you can go into the VAC Console and work through the normal Configuration steps. The only thing you don’t need to do is add the Cloud Connect Server to the VAC instance as this has already been done during the initial deployment process.

It’s worth noting that the versions of Backup & Replication ( and Availability Console ( are up to date and include the latest Hot-Fixes for VAC. The intent is to have the templates as up to date as possible, however once deployed you can upgrade as per usual.


So there you have it…within fifteen minutes you can have a fully working Veeam Availability Console instance running in Azure and ready to be used to offer all the goodness that VAC offers our Cloud and Service Provider partners. For an overview as to what VAC offers, click here and have a read of my GA post on What’s in It for Service Providers.



Office 365 Backups and the Opportunity that Exists for Service Providers

In recent weeks i’ve become reacquainted with an old friend…There was a time where eighty to ninety percent of my day job was working in and around Exchange Server. If I had started this blog in 2005 it would have been dominated with posts around the Hosting of Exchange Server and probably be named Exchange is Life!. I take pride in my Hosted Exchange Org and User creation scripts that I created before Hosting Control Panels where even a thing.

Over the last five or six years my interest in Exchange diminished due to moving roles and also due to some lingering ill feelings about the way in which Microsoft treated their initial Hosting partners as they started what would become, Office 365 back in the late 2000’s. That said I have remained aware of the Exchange landscape and while there is still a lot of on-premises Exchange instances and still a number of decent Hosted Exchange providers out there, there is no stopping Office 365’s growth.

I even jumped on the bandwagon by moving my personal SliemaLabs domain over to an Office 365 Exchange subscription late last year. That domain initially lived on an Exchange Server I ran from home, and then on a Hosted Exchange platform I built and now it’s completed it’s own journey to Office 365.

Having spent a bit of time recently looking at the 1.5 version of our Backup for Microsoft Office 365 product…more specifically the new self service feature that came in Backup & Replication 9.5 Update 3. I’ve had a renewed sense of purpose around the Exchange ecosystem…and that purpose is to ensure that all service providers understand the opportunity that exists around creating offerings for the backing up and availability of Office365 services.

This post follows a post that was released on the blog by Paul Mattes (VP of Global Cloud Group at Veeam) talking about the success of our Backup for Microsoft Office 365 product.

In 2017, more than 25,000 organizations installed our Office 365 backup solution, representing 2.3 million Microsoft Office mailboxes. We saw a staggering 327% quarter-over-quarter growth in Q4 of last year.

And the reasons why all Office 365 users should consider an external backup solution for their data hosted in Microsoft’s SaaS cloud platform.

It’s important to remember that SaaS platform providers, like Microsoft Office 365, take on the responsibility of application uptime and the underlying infrastructure. But it is the customer’s responsibility to manage and protect their vital business data.

This is public cloud in a nutshell…Ultimately the customer has the responsibility to ensure all data is backed up correctly. I won’t go into the technical aspects as to why Office 365 requires additional backups solutions. There a plenty of good online resources, a Gartner report is available here Microsoft’s has an offical page on High Availability and Business Continuity guide. Doing research into the nature of SaaS you understand the need for third party backup solutions.

The Office 365 Opportunity:

From a service provider point of view there is an opportunity to tap into the 85 million user Exchange Online market and offer availability services for organisations using Office 365. This is a multi-billion dollar market that exists today and services based around backup and management of that data are central to tapping into that opportunity. Just breaking down the ANZ market alone, there are approximately 4.25 million Office 365 users of which if only 5% was captured would represent a combined 3.5 to 5 million dollar market.

For those VCSPs who have already deployed Cloud Connect and offering Backup services, the ground work has been laid with regards to having the infrastructure in place to extend that service to offer Veeam Backup for Office 365 aaS.

The billable components of this service are licenses and then storage costs. Managed Service Providers can also build in management fees that offer an end to end solution for their clients. Where it should be seen to be extremely attractive for VCPSs is in the potential for the storage revenue to be significant early and then continue to grow as tenant’s backup and retain more and more mailboxes in addition to new tenants coming on board.

We have given our VCSPs the tools to be able to build a strong service around Office 365 backups with the 1.5 release of Backup for Office 365 focused on scalability and automation. Add to that the self service feature that came in Update 3 for Backup & Replication and there is no excuse to not start thinking about offering this as a service.

Looking beyond Exchange Online, version 2 of Backup for Office 365 will include the ability to backup SharePoint and OneDrive as well…have a think about what that represents in terms of revenue opportunities just on the potential for storage consumption alone.

Again, I want to emphasis that this market is huge and what’s on offer in terms of potential revenue can’t be ignored. I’m excited about the next 12-18 months in being able to see our VCSPs grab this opportunity…don’t let it slip!


The Limitations of Microsoft Office 365 Backup



A Deeper Look at Insider Protection in 9.5 Update 3

With the release of Backup & Replication 9.5 Update 3 we introduced the concept of a Recycle Bin for customers sending offsite cloud backups to VCSPs using Veeam Cloud Connect. This deleted backup protection…or Insider Protection allows the VCSP to enable the deleted backups protection option for specific tenants and looks to add another level of data security for cloud based backups in the case of a malicious user gaining access to the Backup & Replication Console or in the case of accidental deletion by an administrator.

As shown above, this is set by checking a box (Also via PowerShell) in the properties of the tenant account. Once checked the SP will choose the retention period by setting the Keep deleted Backup files for <N> days option. With this option enabled, when a backup or a specific restore point in the backup chain is deleted or aged out from the cloud repository. The actual backup files are not deleted immediately, instead, they are moved to a _RecycleBin folder on the repositories.

Once moved, backup files in the recycle bin do not consume tenant quota however they obviously consume general storage. With that in mind it should be considered by the SP to charge for that used storage. I will release a post shortly detailing some tips on how best to size and charge for the recycle bin storage per client.

At the tenant end those backup files that are moved into the recycle bin are not registered and will not show up in the job information window. They can’t access or do anything with the files in the recycle bin. For the moment if a tenant wants to restore data they must contact the SP to obtain the necessary backup files. Once the retention period has expired all files that fall out of that period are deleted.

Basic Mechanics:

When the option is checked for a tenant a new folder is created under the _RecycleBin\<tenant> folder of the repository. In the case of a Scale Out Backup Repository there is a recycle bin folder created per extent which ensured that any split tenant VM files are processed locally and not between extents.

Once files in the repository start to age out the tenant folder will start to populate with backup files. If there is an event that triggers a change of retention or a VM removed from a job or the deletion of a whole job, any remaining VBK or VIB files in the tenant repository are moved into the recycle bin.

The files remain in the _RecycleBin folder until the retention period has passed or if the service provider moves them out of the folder for recovery purposes.

Working Example:

I have a Cloud Connect Backup account that I am using to back up five VMs that reside on premises, using a standard Backup Job with Forward Incrementals and a Synthetic Full done once a week. I have configured this job to keep two restore points.

I then have configured a secondary destination for the job via a Backup Copy Job to the Cloud Repository and I have set a GFS to happen weekly so I have a full archive offsite. If I hadn’t enabled GFS retention (for those running Update 3) a warning would appear as shown below.

Tip: If the tenant plans to create off-site copies of backed-up data with a backup copy job, it should enable GFS retention settings in the job properties. This way, Veeam Backup & Replication will be able to protect backups created by the job against an attack when a hacker reduces the job’s retention policy and creates a few incremental backups to remove backed-up data from the backup chain.

The Cloud Connect Tenant account has a deleted backup protection setting of 2 days configured as shown in the first image of this post.

Below is the local jobs folder structure:

Looking at the Cloud Connect repository (split over two SOBR extents) you can see that the main repository holds the VM backup files as per the job configuration. Notice the GFS _W files there as well.

Taking a look at the _RecycleBin folder for the tenant after a few days the aged out incremental will start to appear in the folder. Notice that there are no full backup files in the recycle bin at this stage.

Tip: The retention period will look at all backup jobs completed in a 24 hours period and have any expiring or deleted backup files moved into the recycle bin directory. This means that if you are copying up VMs that have a local backup interval of every 4 hours you will have six lots of backup files ageing out daily.

In this example I’m simulating an malicious attack or accidental deletion the VM (TPM03-RMQ-01/VM-120) from the backup. For the sake of this example we are deleting the VM from the Backup & Replication Console under Backups and Cloud. If the Included Archived copies option was chosen then the GFS weekly full backup file is also moved into the recycle bin.

Once the deletion process has been completed the _RecycleBin folder for the tenant will now be populated with the deleted full, plus three incremental files. If the Included Archived copies option was chosen then the GFS weekly full backup file is also moved into the recycle bin.

These will stay in the recycle bin until the retention period is met. From here these files can be transported back to the tenant to be recovered (see here for full process) from within the on-premises Backup & Replication console.


As shown above, deleted backup protection or Insider Protection is an excellent enhancement to Cloud Connect Backup. It goes some way to having an air gapped backup in the cloud and protects against malicious attacks and rogue or clumsy administrators. There is a lot happening behind the scenes to make it work, however the concept is simple and this features extends the 3-2-1 rule by protecting that offsite copy as part of the Cloud Connect solution. VCSP’s should be looking to offer this as a value add to their clients and Veeam customers should be looking to take advantage of Cloud Connect Backup and Replication for their offsite backup and replication needs.


Quick Look: Veeam Agent for Linux 2.0 – Now With Cloud Connect

Just over a year ago Veeam Agent for Linux version 1.0 was released and for me still represents an important milestone for Veeam. During various presentations over the last twelve months I have talked about the fact that Linux backups haven’t really changed for twenty or so years and that the tried and trusted method for backing up Linux systems was solid…yet antiquated. For me, the GitLab backup disaster in Feburary highlighted this fact and the Veeam Agent for Linux takes Linux backups out of the legacy and into the now.

Yesterday, Veeam Agent for Linux 2.0 (Build was released and with it came a number of new features and enhancements improving on the v1.1 build released in May. Most important for me is the ability to now backup straight to a Cloud Connect Repository.

Integration with Veeam Cloud Connect provides the following options:

  • Back up directly to a cloud repository: Veeam Agent for Linux provides a fully integrated, fast and secure way to ship backup files directly to a Cloud Connect repository hosted by one of the many Veeam-powered service providers.
  • Granular recovery from a cloud repository: Volume and file-level recovery can be performed directly from a backup stored within the cloud repository, without having to pull the entire backup on-premises first.
  • Bare-metal recovery from a cloud repository: The updated Veeam Recovery Media allows you to connect to your service provider, select the required restore point from the cloud repository and restore your entire computer to the same or different hardware.
Configuration Overview:

To install, you need to download the relevant Linux Packages from here. For my example below, I’m installing on an Ubuntu machine but we do support a number of popular Linux Distros as explained here.

Once installed you want to apply a Server License to allow backing up to Cloud Connect Repositories.

Before configuring a new job through the Agent for Linux Menu you can add Cloud Providers via the agent CLI. There are a number of cli menu options as shown below.

From here, you can use the cli to configure a new Backup Job but i’ve shown the process though the Agent UI. If you preconfigure the Service Provider with the cli once you select Veeam Cloud Connect Repository you don’t need to enter in the details again.

Once done and the job has run you will see that we have the backup going direct to the Cloud Connect Repository!

From the cli you can also get a quick overview of the job status.

Wrap Up:

I’ve been waiting for this feature for a long time and with the amount of Linux server instances (both physical and virtual) that exist today across on-premises, partner hosts IaaS platforms, or hyper-scale clouds, I hope that Veeam Cloud & Service Providers really hone in on the opportunity that exists with this new feature.

For more on What’s New in 2.0 of Veeam Agent for Linux you click here.


Veeam Backup & Replication 9.5 Update 3 – Top New Features

Earlier today we at Veeam released Update 3 for Veeam Backup & Replication 9.5 (Build and with it comes a couple of very anticipated new features. Back in May at VeeamOn we announced a number of new features that where scheduled to be released as part of the next version of Backup & Replication (v10), however things have worked out such that we have brought some of those features forward into Update 3 for v9.5. It’s a credit to the Product Managers, QA and R&D that we have been able to deliver these ground breaking features into a Update release.

Together with Update 3 we have also released:

Focusing back on Backup & Replication…Update 3 is a fairly significant update and contains a number of enhancements and fixes with a lot of those enhancements aimed at improving the scalability of our flagship Backup & Replication platform. The biggest and most anticipated feature is the built in Agent Management meaning Backup & Replication can now manage virtual, physical and cloud-based workloads from a single console. Further to that we have added offical support for VMware Cloud on AWS and vCloud Director 9.0.

Below are the major features included in Update 3.

  • Built-in agent management
  • Insider protection for Veeam Cloud Connect
  • Data location tagging
  • IBM Spectrum Virtualize Integration
  • Universal Storage Integration API

Other notable enhancements and feature updates include supportability for 4TB virtual disks when using Direct Restore to Azure and support for SQL Server 2017 with that also now a possible database target for the platform. There is extended support for the latest Windows 10, Server and Hyper-V releases. In terms of storage apart from the addition of IBM support and the Universal Storage Integration API we added enhancements to Cisco HyperFlex, Data Domain and HPE 3PAR StoreServ as well as support for Direct NFS to be more efficient with HCI platforms like Nutanix.

For the agents you can now do backup mapping for seeding and restore from backup copies. For VMware there is a significant fix for a condition which reset CBT data for all disks belonging to a VM rather than just the resized disk and there is support again for non encrypted NDB transport.

There is also a lot of new features and enhancements for VCPS and i’ll put together a couple of seperate posts over the next few days outlining those feature…though I did touch on a few of them in the Update 3 RTM post here.

A quick note also for VCSPs that you can upgrade from the RTM to the GA build without issue.

For a full list check out the release notes below and download the update here.



AWS re:Invent – Expectations from a VM Hugger…

Today is the first day offical day of AWS re:Invent 2017 and things are kicking off with the global partner summit. Today also is my first day of AWS re:Invent and I am looking forward to experiencing a different type of big IT conference with all previous experiences being at VMworld or the old Microsoft Tech Eds. Just buy looking at the agenda, schedule and content catalog I can already tell re:Invent is a very very different type of IT conference.

As you may or may not know I started this blog as Hosting is Life! and the first half of my career was spent around hosting applications and web services…in that I gravitated towards looking at AWS solutions to help compliment the hosting platforms I looked after and I was actively using a few AWS services in 2011 and 2012 and attended a couple of AWS courses. After joining Zettagrid my use of AWS decreased and it wasn’t until Veeam announced supportability for AWS storage as part of our v10 announcements that I decided to get back into the swing of things.

Subsequently we announced Veeam Availability for AWS which leverages EBS snapshots to perform agentless backups of AWS instances and more recently we where announced as a launch partner for VMware Cloud on AWS data availability solutions. For me, the fact that VMware have jumped into bed with AWS has obviously raised AWS’s profile in the VMware community and it’s certainly being seen as the cool thing to know (or claim to know) within the ecosystem.

Veeam isn’t the only backup vendor looking to leverage what AWS has to offer by way of extending availability into the hyper-scale cloud and every leading vendor is rushing to claim features that offload backups to AWS cloud storage as well as offering services to protect native AWS workloads…as with IT Pros this is also the in thing!

Apart from backup and availability, my sessions are focused on storage, compute, scalability and scale as well as some sessions on home automation with Alexa and alike. This years re:Invent is 100% a learning experience and I am looking forward to attending a lot of sessions and taking a lot of notes. I might even come out taking the whole serverless thing a little more seriously!

Moving away from the tech the AWS world is one that I am currently removed from…unlike the VMware ecosystem and VMworld I wouldn’t know 95% of the people delivering sessions and I certainly don’t know much about the AWS community. While I can’t fix that by just being here this week, I can certainly use this week as a launching pad to get myself more entrenched with the technology, the ecosystem and the community.

Looking forward to the week and please reach out if you are around.

VCSP Important Notice: 9.5 Update 3 RTM Is Out…With Insider Protection and more!

Earlier this week, Veeam made available to our VCSP partners the RTM of Update 3 for Backup & Replication 9.5 (Build Update 3 is what we term a breaking update, meaning that if a Cloud Connect tenant upgrades from any previous 9.5 version before VCSPs this will break backup or replication functionality. With that in mind the RTM has been made available for our VCSP partners to ensure it is installed and tested before being pushed out to production before the GA release. Veeam Backup & Replication releases from 8.0 (build can write backups to a cloud repository on 9.5 Update 3, and any release from 9.0 (build can write replicas to a cloud host on 9.5 Update 3.

Update 3 is a very significant update and contains a number of enhancements and known issue fixes with a lot of those enhancements aimed at improving the scalability of the Backup & Replication platform that VCSPs can take advantage of. One important note is around new licensing for Cloud Connect Backup that all VCSPs should be aware of. There is a detailed post in the VCSP Forums and there will be emails sent to explains the changes.

We have also pushed out a number new features for our VCSPs with two of them highlighted below. One of which is the new Insider Protection feature or Recycle Bin for Cloud Connect Backups and the other is the a long awaited ask from our providers in the Maintenance Mode for Cloud Connect.

  • Insider protection: Option to hold backups deleted from a tenant’s cloud repository in a “recycle bin” folder for a designated period of time. For more information, see this post in the VCSP forum.

    • Maintenance Mode: Allows you to temporarily stop tenant backup and backup copy tasks from writing to cloud repositories. Already running tenant tasks are allowed to finish, but new tenant tasks fail with an error message indicating that the service provider infrastructure is undergoing maintenance. This is supported at the tenant end in 9.5 Update 3 GA, Agent for Windows 2.1 and Agent for Linux 2.0.

There has also been a lot of work to improve and enhance scalability in the Backup & Replication Cloud Connect functionality to accomodate the increasing usage of Veeam Agent for Windows of which there is a new version (2.1) coming in early December and prepare for the release of Veeam Agent for Linux (2.0) that will include support for backups to be sent to Cloud Connect repositories. For the recently released Veeam Availability Console, Update 3 is 100% compatible with the 2.0 GA (Build released last week and is good from Update 2 or later.


Once again, Update 3 for Veeam Backup & Replication is an important update to apply for VCSPs running Cloud Connect services in preparation for the GA release which will happen in about two weeks. Once released I’ll link to the VeeamKB for a detailed look at the fixes but for the moment, if you have the ability to download the update do so and have it applied to your instances. For more info in the RTM, head to the VCSP Forum post here.

Veeam Availability Console – What’s in it for Service Providers

Today, the Veeam Availability Console was made GA meaning that after a long wait our new multi-tenant service provider management and reporting platform is available for download. VAC is an significant evolution of the Managed Backup Portal that was released in 2016 and acts as a central portal for Veeam Cloud and Service Providers to remotely manage and monitor customer instances of Backup & Replication including the ability to monitor Cloud Connect Backup and Replication jobs and failover plans. It also is the central mechanism to deploy and manage (Windows) agents which includes the ability to install agents onto on-premises machines and apply policies to those agents once deployed.

Veeam® Availability Console is a cloud-enabled platform built specifically for Veeam Cloud & Service Provider (VCSP) partners and resellers looking to launch a managed services business. Through its ability to remotely provision, manage and monitor virtual, physical and cloud-based Veeam environments without any special connectivity requirements, Veeam Availability Console enables you to increase revenue and add value to all your customers.

  • Simplified Setup – now allowing on-premises installs
  • Remote backup agent management and monitoring
  • Remote discovery and deployment with enhanced support for Veeam Cloud Connect
  • Web-based multi-tenant portal
  • Native billing and RESTful APIs
Cloud Connect Requirement:

The Cloud Connect Gateway is central to how the Veeam Availability Console operates and all management traffic is tunneled through the Cloud Connect Gateways. If you are a current VCSP offering Cloud Connect services then you already have the infrastructure in place to facilitate VAC, however if you are not a Cloud Connect partner you can apply for a special key that will enable you to deploy a Gateway without the need for specific Cloud Connect backup or Replication licenses.

For a deeper look at VAC architecture for Service Providers, head to Luca Dell’Oca’s VAC series here.

Designed for Service Providers First:

The Veeam Availability Console was designed from the ground up for Service Providers (there is an Enterprise version available) and contains a rich set of APIs that can be consumed for automation and provisioning purposes. There is also a three tier multi-tenancy design allowing VCSPs the ability to create restricted accounts for their partners or resellers from which in turn, another level of accounts can be created for their customers or tenants.

The multi-tenancy aspect means that partners/resellers and customers can control their own backups centrally from the console. Reporting on backup jobs can be viewed and a mechanism to control those jobs is available allowing retry/stop/start tasks against those jobs. If that’s not enough control or more troubleshooting on failed jobs needs to be done the Remote Console feature introduced in Veeam Backup & Replication Update 2 has been integrated into the console.

VAC also includes built in reporting and billing functionality which enables VCSPs who don’t have the capability for automated reporting and billing to offer that to their customers. The reporting can be accessed via the API meaning that if an existing billing engine is being used there is the possibility to have that interface with VAC to pull out key data points.

The Service Provider Opportunity:

Over the past year I’ve talked a lot about the opportunity that exists for Veeam’s Cloud and Service Providers to take advantage of the opportunity that exists with Veeam’s Agents to capture backups for workloads that previously were out of reach. VAC is central to this and opens up the ability to backup instances that live on-premises (physical or virtual) or in any public cloud hyper-scaler or otherwise.

If you are a reseller looking to cash in on the growing data availability market then you should be looking at how VAC can help you get started by leveraging the features mentioned above . Secondly, if you a reseller and not running Cloud Connect Backup or Replication then the time is right to start looking at getting Cloud Connect deployed and start generating revenue around backup and replication services.

For those existing VCSPs that are offering Cloud Connect services, adding VAC into the mix will allow you to take advantage of the agent opportunity that exists as shown above while also adding value to your existing Managed Backup and Cloud Connect services.

References and Product Guides:

Veeam Vault #9: Backup for Office 365 1.5 GA, Azure Stack and Vanguard Roundup

Welcome to another Veeam Vault! This is the ninth edition and given the last edition was focused around VMware and VMworld I thought just for a change, the focus for this edition will be Microsoft. Reason for that is over the past couple of weeks we have had some significant announcements around Azure Stack and the GA release of Backup for Office 365 1.5. I’ll cover both of those announcements, share some Veeam employee automation work that shows off the power of our new APIs and see what the Veeam Vanguard’s have been blogging about in the last month or so.

Backup for Office 365 1.5 GA:

The early part of my career was dedicated to Exchange Server however I drifted away from that as I made the switch to server virtualization and cloud computing. The old Exchange admin in my is still there however and it’s for that reason that I’m excited about the GA of our Backup for Office 365 product which is now at version 1.5. This release caters specifically for service providers adding scalability and automation enhancements as well as extended support for on-premises and hybrid Exchange setups.

New features and enhancements:

  • Distributed, scalable architecture: Enhanced scalability in distributed environments with several Remote Offices/Branch Offices and in service providers infrastructures
  • Backup proxies: take the workload off the management server, providing flexible throttling policy settings for performance optimization.
  • Support for multiple repositories: Streamlines data backup and restore processes.
  • Support for backup and restore of on-premises and hybrid Exchange organizations: Allows a variety of configurations and usage scenarios and implement those that meet your particular needs.
  • Increased performance: Restore operations allows for up to 5 times faster restores than in v1.0.
  • Restore of multiple datastore mailboxes using Veeam Explorer for Microsoft Exchange: simplifies workflow and minimizes workload for restore operators, as well as 1-Click restore of a mailbox to the original location.
  • RESTful API and PowerShell cmdlets: Helpful for automation of routine tasks and integration into existing or new portals.
  • UI Enhancements: Including main window, wizards, dialogs, and other elements, facilitating administration of the solution.
Examples of the Power of the Veeam APIs:

One of the features of Backup for Office 365 was the addition of a power set of RESTful APIs and PowerShell commandlets that are aimed are service providers automating the setup and management of their offerings around the product. A couple of our employees have written example interfaces for the Backup for Office 365 product and it shows that any service provider with some in house programming skill set can build customer portals that enhances their offerings and increases efficiency through automation.

Special welcome to Niels who this week joined our team. Great to have you on board!

Microsoft Azure Stack Support:

Last week at Microsoft Ignite, we announce our supportability for Azure Stack. This is based around our Windows Agent, Cloud Connect and Availability Console products that combine together to off an availability solution

Key benefits of Veeam’s support for the Azure Stack include:

  • Multi-tenancyVeeam Cloud Connect isolates backup copies for each tenant ensuring security and compliance; 
  • Multiple recovery options: Veeam Backup & Replication supports both granular item level recovery through Veeam Explorers for Microsoft Exchange, SQL Server, Microsoft SharePoint, Microsoft Active Directory and for Oracle, as well as full file level restores for tenant files that were deleted or corrupted;
  • Reporting & Billing: Veeam Availability Console supports real-time monitoring and chargeback on tenant usage, allow either Hosting providers or Enterprise organizations to easily manage and bill their tenants for Availability usage.

Veeam Vanguard Blog Post Roundup:


The One Problem with the VCSA

Over the past couple of months I noticed a trend in my top blog daily reporting…the Quick fix post on fixing a 503 Service Unavailable error was constantly in the top 5 and getting significant views. The 503 error in various forms has been around since the early days of the VCSA which usually manifests it’s self with the following.

503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http20NamedPipeServiceSpecE:0x0000559b1531ef80] _serverNamespace = / action = Allow _pipeName =/var/run/vmware/vpxd-webserver-pipe)

Looking at the traffic stats for that post it’s clear to see an upward trend in the page views since about the end of June.

This to me is both a good and bad thing. It tells me that more people are deploying or migrating to the VCSA which is what VMware want…but it also tells me that more people are running into this 503 error and looking for ways to fix it online.

The Very Good:

The vCenter Server Appliance is a brilliant initiative from VMware and there has been a huge effort in developing the platform over the past three to four years to get it to a point where it not only became equal to vCenter’s deployed on Windows (and relying on MSSQL) but surpassed it in a lot of features especially in the vSphere 6.5 release. Most VMware shops are planning to or have migrated from Windows to the VCSA and for VMware labs it’s a no brainer for both corporate or homelab instances.

Personally I’ve been running VCSA’s in my various labs since the 5.5 release, have deployed key management clusters with the VCSA and more recently have proven that even the most mature Windows vCenter can be upgraded with the excellent migration tool. Being free of Windows and more importantly MSSQL is a huge factor in why the VCSA is an important consideration and the fact you get extra goodies like HA and API UI’s adds to it’s value.

The One Bad:

Everyone who has dealt with storage issues knows that it can lead to Guest OS file systems errors. I’ve been involved with shared hosting storage platforms all my career so I know how fickle filesystems can be to storage latency or loss of connectivity. Reading through the many forums and blog posts around the 503 error there seems to be a common denominator of something going wrong with the underlying storage before a reboot triggers the 503 error. Clicking here will show the Google results for VCSA + 503 where you can read the various posts mentioned above.

As you may or may not know the 6.5 VCSA has twelve VMDKs, up from 2 in the initial release and to 11 in the 6.0 release. There a couple of great posts from William Lam and Mohammed Raffic that go through what each disk partition does. The big advantage in having these seperate partitions is that you can manage storage space a lot more granularly.

The problem as mentioned is that the underlying Linux file system is susceptible to storage issue. Not matter what storage platform you are running you are guaranteed to have issues at one point or another. In my experience Linux filesystems don’t deal will with those issues. Windows file systems seem to tolerate storage issue much better than their Linux counterparts and without starting a religious war I do know about the various tweaks that can be done to help make Linux filesystems more resilient to underlying storage issues.

With that in mind, the VCSA is very much susceptible to those same storage issues and I believe a lot of people are running into problems mainly triggered by storage related events. Most of the symptoms of the 503 relate back to key vCenter services unable to start after reboot. This usually requires some intervention to fix or a recovery of the VCSA from backup, but hopefully all that’s needed is to run an e2fsck against the filesystem(s) impacted.

The Solution:

VMware are putting a lot of faith into the VCSA and have done a tremendous job to develop it up to this point. It is the only option moving forward for VMware based platforms however there needs to be a little more work done into the resiliency of the services to protect against external issues that can impact the guest OS. PhotonOS is now the OS of choice from 6.5 onwards but that will not stop the legacy of susceptibility that comes with Linux based filesystems leading to issues such as the 503 error. If VMware can protect key services in the event of storage issues that will go a long way to improving that resiliency.

I believe it will get better and just this week VMware announced a monthly security patch program for the VCSA which shows that they are serious (not to say they where not before) about ensuring the appliance is protected but I’m sure many would agree that it needs to offer reliability as well…this is the one area where the Windows based vCenter has an advantage still.

With all that said, make sure you are doing everything possible to have the VCSA housed on as reliable as possible storage and make sure that you are not only backing up the VCSA and external dependancies correctly but understand how to restore the appliance including understanding of the inbuilt backup mechanisms for backing up the config and the PostGres database.

I love and would certainly recommend the VCSA…I just want to love it a little more without having to deal with possibility of having the 503 server error lurking around every storage event.


« Older Entries