Tag Archives: Object Storage

v10 Enhancements – Mounting Object Storage Repository for DR

Version 10 of Veeam Backup & Replication isn’t too far away and we are currently at the end of a second private BETA for our customers and partners. There has been a fair bit of content released around v10 functionality and features from our Veeam Vanguard’s over the past couple of weeks and as we move closer to GA, as part of the lead up, I am doing a series on some of the cool new enhancements that are coming as part of the release. These will be quick short takes that give a glimpse into what’s coming as part of the v10 release.

Mounting Object Storage Repository for Streaming Disaster Recovery

The Cloud Tier was introduced in Veeam Backup & Replication 9.5 Update 4 and focused on the offloading of data from local repositories to Object Storage repositories. Essentially looking to reduce the cost and overheads of ever growing local primary repositories. Due to the smarts we built into the feature, the use cases for Cloud Tier started to expand beyond the offloading of data and looked at recovery options.

Because we hold a replicated copy of the VBK metadata as well the actual backup data that is indexed as blocks in Object Storage we have the ability to leverage the data sitting there for recovery purposes. I’ve already shown this a number of times this year, and presented on the recovery and resiliency of the Cloud Tier at Cloud Field Day 5.

With v10, we have made this process even easier by introducing a Mount function that will enable users to import backup restore points for recovery purposes in the case of disaster. This can even be done with the Community Edition which means that the Cloud Tier now becomes a mechanism for recovery from any device to almost any platform.

Quickly going over how this works, the first step is to recreate the Object Storage Repository with the same settings as the one existed in the original location.

At this point we can leverage the new v10 feature that allows you to Import the backup data contained on the Object Storage Repository by right clicking on the repository and selecting Import Backups.

This will store the available restore points in the Backup & Replication database and have them appear under Imported Backups in the console.

It’s important and cool to note, that as this stage we haven’t downloaded the metadata shells that constitute the de-hydrated VBK. One of the extra smart things we have built into this feature is that the metadata and VBK shells are only downloaded once a restore operation has been started, meaning quicker setup and more specific re-syncing of the metadata shells.

On that note, all existing restore operations are available at this point.

Once a restore operation is triggered, only then is the required metadata downloaded and reconstructed into the required shell chain to a temp directory. The example below shows the shells of a full and an incremental triggered by an Instant VM Recovery (IVMR) Operation.

The data required to perform the IVMR is streamed from the Object Storage Repository (Capacity Tier Extent).

Once restore operations have been completed you can go back to the Object Storage Repository, right click and select Detach

This unmounts the Object Storage and removed the restore points from the Imported Backup view and deletes the downloaded contents of the temp folder where the metadata shells where staged.

Wrap Up:

That was a quick look at one of my personal favourite new enhancements in v10. We have improved an operation that was being leveraged in 9.5 Update 4 due to the smarts built into the Cloud Tier for recovery operations and made it quicker and more efficient. This also allows users to effectively restore to any platform from any device that has Veeam Backup & Replication installed and has access to the Object Storage platform!

When put together with the new Copy mode being introduced into v10, we all of a sudden have a solution that can achieve very low RPO and RTO for disaster recovery… more to come on that aspect when v10 launches.

Stay tuned over the next few weeks as I go through some more hidden gems.

Disclaimer: The information and screen shots in this post is based on BETA code and may be subject to change come final GA.

v10 Enhancements – Downloading Object Storage Data per Tenant for SOBR

Version 10 of Veeam Backup & Replication isn’t too far away and we are currently in the middle of a second private BETA for our customers and partners. There has been a fair bit of content released around v10 functionality and features from our Veeam Vanguard’s over the past couple of weeks and as we move closer to GA, as par of the lead up, I am doing a series on some of the cool new enhancements that are coming as part of the release. These will be quick short takes that give a glimpse into what’s coming as part of the v10 release.

Downloading Tenant Data from SOBR Capacity Tier

Cloud Tier was by far the most significant feature of Update 4 for Backup & Replication 9.5 and we have seen the consumption of Object Storage in AWS, Azure and other platforms grow almost exponentially since its release. Our VCSPs have been looking to take advantage of the MOVE functionality that came in Update 4, but have also requested a way to pull back offloaded data from the Capacity Tier back to the Performance Tier on a per tenant basis.

The use case for this might be for tenant off-boarding, or migration of backup data back onsite. In any case our VCSPs needed a way to get the data back and rehydrate the VBK files and remove the data from Object Storage. In this quick post I’ll show how this is achieved through the UI.

First, looking at the image below you can see that there are a couple of dehydrated VBK files that belong to a specific tenant Cloud Connect Backup job are no bigger than 17MB as they site next to ones that are about 1GB.

To start a Download job, we have the option to click on the Download icon in the Tenant ribbon, or right right clicking on the tenant account and select Download

There will be an information box appear letting you know that there is a backup chain on the performance extent and the disk space required to download the other backup data back to the performance tier from the capacity tier The SOBR Download job progress can be tracked
When completed we can see details of the download from Object Storage to the Performance Tier. In the example below a lot of the blocks that where present in the Performance Tier where used to rehydrate the previously offloaded VBKs. This new feature is leveraging the Intelligent Block Recovery to save on egress and also reduce download time. Going back to the file view, the previously smaller 17MB VBKs have been rehydrated to their previous size and we have all the tenant’s data back on the Performance Tier ready to be accessed.

Wrap Up:

That was a quick look at one of the cool smaller enhancements coming in v10. The ability to download data on a per tenant based from the Capacity Tier back to the Performance Tier is one that I know our VCSPs will be happy with.

Stay tuned over the next few weeks as I go through some more hidden gems.

Disclaimer: The information and screen shots in this post is based on BETA code and may be subject to change come final GA.

Public Beta – Backup for Microsoft Office 365 v4

Overnight at Microsoft Ignite, we announced availability of the Public Beta for the next version of Veeam Backup for Microsoft Office 365. This is again a much awaited update for VBO with a ton of enhancements and the introduction of Object Storage Support for Backup Repositories. The product has done extremely well and is one of our fastest growing in the Veeam Availability Platform. The reason for this is due to Organizations now understanding the requirements around the backing up of Office 365 data.

Backup for Office 365 3.0 Yes! You Still Need to Backup your SaaS

While we have enhanced a number of features and added some more reporting and user account management options, the biggest addition is the ability to leverage Object Storage to store longer term backup data. This has been a huge request since around version 1.5 of VBO, mainly due to the amount of data that is required to backup Exchange, SharePoint and OneDrive Organizations.

Similar to Cloud Tier in Backup & Replication 9.5 Update 4, the premise of the feature is to release pressure (be it cost, management and maintenance) on local Backup Repositories and offload the bulk of the data to cheaper Object Storage.

There is support in the beta for:

Though similar in name to Cloud Tier in Backup & Replication, the way in which the data is offloaded, stored and referenced in the VBO implementation is vasty different to that of Cloud Tier. As we get to GA for the v4 release there will be more information forthcoming about the underlying mechanics of that.

The Beta is available now and can be installed on a seperate server for side by side testing against Office 365 Organizations. For those looking to test the new functionality before the offical GA later in the year head to the Beta Download page and try it out!

Cloud Tier Data Migration between AWS and Azure… or anywhere in between!

At the recent Cloud Field Day 5 (CFD#5) I presented a deep dive on the Veeam Cloud Tier which was released as a feature extension of our Scale Out Backup Repository (SOBR) in Update 4 of Veeam Backup & Replication. Since we went GA we have been able to track the success of this feature by looking at Public Cloud Object Storage consumption by Veeam customers using the feature. As of last week Veeam customers have been offloading petabytes of backup data into Azure Blob and Amazon S3…not counting the data being offloaded to other Object Storage repositories.

During the Cloud Field Day 5 presentation, Michael Cade talked about the Portability of Veeam’s data format, around how we do not lock our customers into any specific hardware or format that requires a specific underlying File System. We offer complete Flexibility and Agnosticity where your data is stored and the same is true when talking about what Object Storage platform to choose for the offloading of data with the Cloud Tier.

I had a need recently to setup a Capacity Tier extent that was backed by an Object Storage Repository on Azure Blob. I wanted to use the same backup data that I had in an existing Amazon S3 backed Capacity Tier while still keeping things clean in my Backup & Replication console…luckily we have built in a way to migrate to a new Object Storage Repository, taking advantage of the innovative tech we have built into the Cloud Tier.

Cloud Tier Data Migration:

During the offload process data is tiered from the Performance Tier to the Capacity Tier effectively Dehydrating the VBK files of all backup data only leaving the metadata with an Index that points to where the data blocks have been offloaded into the Object Storage.

This process can also be reversed and the VBK file can be rehydrated. The ability to bring the data back from Capacity Tier to the Performance Tier means that if there was ever a requirement to evacuate or migrate away from a particular Object Storage Provider, the ability to do so is built into Backup & Replication.

In this small example, as you can see below, the SOBR was configured with a Capacity Tier backed by Amazon S3 and using about 15GB of Object Storage.

The first step is to download the data back from the Object Storage and rehydrate the VBK files on the Performance Tier extents.

There are two ways to achieve the rehydration or download operation.

  1. Via the Backup & Replication Console
  2. Via a PowerShell Cmdlet
Rehydration via the Console:

From the Home Menu under Backups right click on the Job Name and select Backup Properties. From here there is a list of the Files contained within the job and also the objects that they contain. Depending on where the data is stored (remembering that the data blocks are only even in one location… the Performance Tier or the Capacity Tier) the icon against the File name will be slightly different with files offloaded represented with a Cloud.

Right Clicking on any of these files will give you the option to Copy the data back to the Performance Tier. You have the choice to copy back the backup file or the backup files and all its dependancies.

Once this is selected, a SOBR Download job is kicked off and the data is moved back to the Performance Tier. It’s important to note that our Intelligent Block Recovery will come into play here and look at the local data blocks to see if any match what is trying to be downloaded from the Object Storage… if so it will copy them from the Performance Tier, saving on egress charges and also speeding up the process.

In the image above you can see the Download Job working and only downloaded 95.5MB from Object Storage with 15.1GB copied from the Performance Tier… meaning the data blocks for the most that are local are able to be used for the rehydration.

The one caveat to this method is that you can’t select bulk files or multiple backup jobs so the process to rehydrate everything from the Capacity Tier can be tedious.

Rehydration via PowerShell:

To solve that problem we can use PowerShell to call the Start-VBRDownloadBackupFile cmdlet to do the bulk of the work for us. Below are the steps I used to get the backup job details, feed that through to variable that contains all the file names, and then kick off the Download Job.

The PowerShell window will then show the Download Job running

Completing the Migration:

No matter which way the Download job is initiated, we can see the progress form the Backup & Replication Console under the Jobs section.

And looking at the Disk and Network sections of Windows Resource Monitor we can see connections to Amazon S3 pulling the required blocks of data down.

Once the Download job has been completed and all VBKs have been rehydrated, the next step is to change the configuration of the SOBR Capacity Tier to point at the Object Storage Repository backed by Azure Blob.

The final step is to initiate an offload to the new Capacity Tier via an Offload Job…this can be triggered via the console or via Powershell (as shown in the last command of the PowerShell code above) and because we have already a set of data that satisfies the conditions for offload (sealed chains and backups outside the operational restore window) data will be dehydrated once again…but this time up to Azure Blob.

The used space shown below in the Azure Blob Object Storage matches the used space initially in Amazon S3 All recovery operations show Restore Points on the Performance Tier and on the Capacity Tier as dictated by the operational restore window policy.
Conclusion:

As mentioned in the intro, the ability for Veeam customers to have control of their data is an important principal revolving around data portability. With the Cloud Tier we have extended that by allowing you to choose the Object Storage Repository of your choice for cloud based storage or Veeam backup data…but also given you the option to pull that data out and shift when and where desired. Migrating data between AWS, Azure or any platform is easily achieved and can be done without too much hassle.

References:

https://helpcenter.veeam.com/docs/backup/powershell/object_storage_data_transfer.html?ver=95u4

Disaster Recovery and Resiliency with Veeam Cloud Tier

Yesterday at Cloud Field Day 5, I presented a deep dive on our Cloud Tier feature that was released as a feature for Scale Out Backup Repository (SOBR) in Veeam Backup & Replication Update 4. The section went through an overview of its value proposition as well as deep dive into how we are tiering the backup data into Object Storage repositories via the Capacity Tier Extend of a SOBR. I also covered the space saving and cost saving efficiencies we have built into the feature as well as looking at the full suite of recoverability options still available with data sitting in an Object Storage Repository.

This included a live demo of a situation where a local Backup infrastructure had been lost and what the steps would be to leverage the Cloud Tier to bring that data back at a recovery site.

Quick Overview of Offload Job and VBK Dehydration:

Once a Capacity Tier Extent has been configured, the SOBR Offload Job is enabled. This job is responsible for validating what data is marked to move from the Performance Tier to the Capacity Tier based on two conditions.

  1. The Policy defining the Operational Restore Window
  2. If the backup data is part os a sealed backup chain

The first condition is all about setting a policy on how many days you want to keep data locally on the SOBR Performance Tiers which effectively become your landing zone. This is often dictated by customer requirements and now can be used to better design a more efficient approach to local storage with the understanding that the majority of older data will be tiered to Object storage.

The second is around the sealing of backup chains which means they are no longer under transformation. This is explained in this Veeam Help Document and I also go through it in the CFD#5 session video here.

Once those conditions are met, the job starts to dehydrate the local backup files and offload the data into Object Storage leaving a dehydrated shell with only the metadata.

The importance of this process is that because we leave the shell locally with all the metadata contained, we are able to still perform every Veeam Recovery option including Instant VM Recovery and Restore to Azure or AWS.

Resiliency and Disaster Recovery with Cloud Tier:

Looking at the above image of the offload process you can see that the metadata is replicated to the Object Storage as well as the Archive Index which keeps track of which blocks are mapped to what backup file. In fact for every extent we keep a resilient copy of the archive index meaning that if an extent is lost, there is still a reference.

Why this is relevant is because it gives us disaster recovery options in the case of a loss of whole a whole backup site or the loss of an extent. During the synchronization, we download the backup files with metadata located in the object storage repository to the extents and rebuild the data locally before making it available in the backup console.

After the synchronization is complete, all the backups located in object storage will become available as imported  jobs and will be displayed under the Backups and Imported in the inventory pane. But what better way to see this in action than a live demo…Below, I have pasted in the Cloud Field Day video that will start at the point that I show the demo. If the auto-start doesn’t kick in correctly the demo starts at the 31:30 minute mark.

References:

https://helpcenter.veeam.com/docs/backup/vsphere/capacity_tier_offload_job.html?ver=95u4

Quick Look: Cloud Tier SOBR Offload Job

With the release of Update 4 for Veeam Backup & Replication 9.5 we introduced the Cloud Tier, which is an extension of the Scale Out Backup Repository (SOBR). The Cloud Tier allows for data to be stripped out of Veeam backup files and offloaded as blocks of data to Object Storage leaving a dehydrated Veeam backup file on the local extents with just the metadata remaining in place. This is done based on a policy that is set against the SOBR that dictates the operational restore window of which local storage is used as the primary landing zone for backup data. The result is a space saving, smaller footprint on the local storage.

Overview of Offload Job:

By default the offload job is run against the data located on the Performance Tier extents of the SOBR every 4 hours. This is a set value that can not be changed. To offload the backup data to the Capacity Tier, the Offload job does the following:

  • Verifies whether backup chains located on the Performance Tier extents satisfy validation criteria and can be offloaded to object storage.
  • Collects verified backup chains from each Performance Tier extent and sends them directly to object storage in the form of data blocks.
  • Saves each session results to the configuration database so that you can review them upon request.

The job and job details can be viewed from the History Menu under System or the Home Menu under Last 24 Hours.

The details of the job will show how much data was offloaded to the Capacity Tier per VM residing on the SOBR. It will show statistics on how much data was processed, read and transferred. Once this job has completed, the local backup files only contain job metadata with the data residing on the Object Storage.

Forcing The Offload Job:

As mentioned, the Offload Job by default is set to run every 4 hours from the creation initial configuration of the Capacity Tier extent on the SOBR. The default value of 4 hours can not be modified however if you want to force the job to run you have two options.

First option is through the UI, under the Backup Infrastructure Menu and under Scale-Out Repositories, do a CONTROL+Click against the SOBR and select the Run Tiering Job Now option. This is hidden by default as an option and will only be shown with the CONTROL+Click

Second option is to run the following PowerShell command:

This tiggers the Offload Job to run.

Note that once the Offload Job has been forced the 4 hours counter is reset to when the job was run…ie the next job will be 4 hours from the time the job was forced.

It’s important to understand that running the job on demand doesn’t necessary mean that you will offload data to the Capacity Tier any quicker. The conditions around operations restore window and sealed backup chains still need to be in place for the job to do its thing. Having the job run six times a day (every 4 hours) is generally going to be more than enough for most instances.

If no data has been offloaded, you will see the following in the job details:

Wrap Up and More Cloud Tier:

To learn more about the Cloud Tier head to my veeam.com post here, and also check our Rhys Hammonds post here. Also look out for a new Veeam White Paper being released in the next month or so which will deep dive into the Cloud Tier in more detail. I will post a few more posts on the Cloud Tier over the next few weeks as well looking at some more use cases and features.

References:

https://helpcenter.veeam.com/docs/backup/vsphere/capacity_tier.html?ver=95u4