Monthly Archives: January 2015

January vCenter and ESXi 5.5 Patch Releases: Critical CBT Bug Fixed

VMware released new builds for vCenter and ESXi 5.5 today. The builds contain mostly bug fixes, but I wanted to point out one fix that had affected those who use CBT as part of their VM backup strategy. Veeam users where initially affected by the bug…though Veeam released a work around in subsequent Veeam 8 builds this ESXi patch officially fixes the issue.

When you use backup software that uses the Virtual Disk Development Kit (VDDK) API call QueryChangedDiskAreas(), the list of allocated disk sectors returned might be incorrect and incremental backups might appear to be corrupt or missing. A message similar to the following is written to vmware.log:

DISKLIB-CTK: Resized change tracking block size from XXX to YYY

For more information, see KB 2090639.

There are also a couple of other interesting fixes in the build:

  • Storage vMotion of thin provisioned virtual machine disk (VMDK) takes longer than the Lazy Zeroed Thick (LZT) disk.
  • When a quiesced snapshot is created on a Windows Server 2003, Windows Server 2003 R2, Windows Server 2008, Windows Server 2008 R2, or a Windows Server 2012 virtual machine, duplicate disks might be created in the virtual machine.
  • On an ESXi 5.5 host, the NFS volumes might not restore after the host reboots. This issue occurs if there is a delay in resolving the NFS server host name to IP address.

For those who backup VMs with disks larger that 128GB I would be looking to deploy this patch ASAP.

Australia Day with @aussievMafia #100

In celebration of Australia Day …and this blogs 100th Post… I have put together an RSS Widget that feeds from the AussievMafia site.

The AussievMafia are a group of Aussie’s who are passionate about Virtualisation…myself included!

The content that’s fed into the page is well worth the follow. Copy and paste the feed code below…otherwise follow @aussievMafia or head to

Platform9 Introduction

I came across Platform9 while wandering the back halls of the VMWorld Solutions Exchange last year in San Francisco…as a fan of the movie District 9 I was drawn to the name without really knowing anything about the tech being shown. After a brief chat with the booth staff going over product I thought to myself that there was potential in a SaaS based Cloud Management/Provisioning Platform…another side of me thought it also threatens part of the work I do in designing and managing Service Provider based Cloud Platforms like vCloud Director.

A few months later Platfrom9 has launched with fresh rounds of VC funding and is ready to go prime time. I spent some time today with Sirish Raghuram Who is the CEO and Co-founder…he also has a very interesting pedigree having previously been with VMware since 2002 working on products like Workstation, SRM and vCloud Director…in fact he was part of the team responsible for bringing maturity to later versions of vCD.

Sirish took me through the basics of what Platform9 offers and how it easily plugs into on-premises Hypervisior resources focusing on KVM with BETA support for ESXi…effectively what you get with Platform9 is your own version of the Openstack Platform and your own Management Portal to control compute, storage and network resources. You then to carve up those resources for use by departments or clients. Like vCloud Director, it abstracts management and provides a mechanism to consume IaaS resources.

What I like about this solution is that its easy…Implementing Openstack is not! …Especially if you choose to deploy and manage it yourself. The time savings along with budget savings compared with deploying and maintaining OpenStack in-house make this an attractive option. It also potentially fills a gap in the market that’s been vacated by VMware’s decision to pull vCD from Enterprise and replace it with the more complicated (dare I say bloated) vRealize Automation. To clarify that comment, think about a situation where a company has a small vCenter Instance with two or three hosts…vRA isn’t the best fit if the company wants to explore Private Cloud…Platform9 fits in nicely and can be retrofitted with ease.

For Service Providers if offers an opportunity to experience Openstack and take advantage of its KVM strengths…dual stacks become a more plausible option and with access to refined APIs it makes it easy for SPs with existing Account/Control Panels to integrate and add to existing offerings. There is also a lot of interest in it’s potential for management of Docker…which is all the rage these days.

For me, this isn’t going to replace vCloud Director SP for vSphere Platforms short to medium term…there are a lot of holes in what Openstack does compared to vCloud Director but there is a future here and there are many use cases where Platform9 makes sense.

Working Through Frustration

The tweet below was posted last night…24 hours later I have found myself in a much better place and certainly free of the thoughts that caused me to vent and I even might have taught myself a lesson…

I’ve mentioned a couple of times in previous posts that I am extremely fortunate to work in an industry that is also a hobby of mine…I love what I do and I love the technology I am exposed to day in and day out. I love getting results and putting runs on the board…I love being on the cutting edge and pushing the envelope…always trying to better myself…In turn this translates to positivity at work.


Sometimes I don’t work on what I love…sometimes I am forced to deviate from the fun things and not focus on technologies and products I have invested myself in over the past few years…sometimes I wonder if I’m wasting my time…all this leads to frustration.

Looking back over the last couple of weeks I’ve stepped outside my comfort zone…worked outside the box, learnt new technologies and appreciated tech I had previously written off…all in all i’ve bettered myself and pretty much done all the things I love about my work…apparently without loving it.

…funny that!

NSX vCloud Retrofit: Logical Network Preparation and Transport Zone Setup

This blog series extends my NSX Bytes Blog Posts to include a more detailed look at how to deploy NSX 6.1.x into an existing vCloud Director Environment. Initially we will be working with vCD 5.5.x which is the non SP Fork of vCD, but as soon as an upgrade path for 5.5.2 -> 5.6.x is released I’ll be including the NSX related improvements in that release.

Part 4: Logical Network Preparation and Transport Zone Setup

In the previous posts we have gone through the process to upgrade the vCNS Manager to the NSX Manager…Configured the NSX Manager so it to talk to vCenter…verified that vCD 5.5 can still deploy/manage traditional vShield Edges and gone through deploying the NSX modules onto ESXi hosts and configure VTEPs for VXLAN transport.

We are now going to prepare for NSX Logical Networks and configure our Transport Zones which define the boundaries of our VXLAN domains. Recently @dkalintsev has released a series of excellent blog posts relating to NSX…the latest goes through Transport Zones in super deep dive detail. If you are not following Dimitri and you are interested in NSX…NIKE!

If you are used to vCloud Director then you know about Provider vDCs and what part they play in abstracting pools of resources for vCD VMs to be consumed via Virtual Datacenters. In it’s simplest form you can think of a PvDC as a NSX Transport Zone and that there is a one to one relationship between the two. With vCD 5.1 the concept of Merging PvDCs first appeared which relied on the vCNS implementation of VXLAN using multicast as a control plane…this opened up the possibility to having vDCs spanning different compute pool resources, possibly in different physical locations. With the NSX Controllers now handling the control plane we can use Unicast and much more easily utilise the Merged PvDCs feature of vCD…using Transport Zones as our network boundaries.

Segment ID Config:

In the Networking & Security Menu go to Installation -> Logical Network Preparation. Under VXLAN you should see the previously configured Cluster and Host details relating to the setting up of the VXLAN Transport Network on each host.

Go to the Segment ID tab and Click on Edit. This is where we are going to configure the scope of the VXLAN Segments that are created. In retrofitting this with vCloud, Segment IDs will be consumed by VXLAN Network Pools in vCD…which in turn translate to Logical Switches

  • You can have 16 million VXLAN segments
  • You can come back here and adjust the number up or down at any time.
  • As we will be using Unicast, leave the Multicast Addressing option unchecked.

Transport Zone Setup:

Go to the Transport Zones Tab and click Add, Enter in the Cluster Name as the name of the new Transport Zone, Select Unicast and Check the desired vSphere Cluster…as mentioned above you can select multiple Clusters to be included in the Transport Zone…this is how you will extend L3 across Providers.

That’s all the ground work done!

The last post in this series will look at how to bring this all together in vCD and leverage some of the power of NSX for vCloud Director.

vCloud Director SP: VM Metrics Database Configuration Part 1

vCloud Director SP 5.6.3 was initially released in October 2014 was the first of the SP Editions that had been forked from the Enterprise 5.x builds that came before it. VMware delivered on their promise to release vCloud Air Features to Network Partners. One of those features was VM Metrics.

  • Virtual machine monitoring: Expose current and historical VM performance metrics to tenants through this tenant visible, multi-tenant safe API. Using the API, tenants can troubleshoot application performance problems, auto-scale their applications, and perform capacity planning.

To facilitate the storing of those VM Metrics a separate database needs to be deployed and and then have vCloud configured to use the new database instance.

No worries right?

The catch here is that VMware have decided to use a linearly scalable and fault tolerant database called KairosDB backed by a Cassandra cluster. To satisfy the requirements as set by the vCloud Director team you need to deploy a three node Cassandra Cluster.

While initially a little annoyed that we would have to deploy and manage another group of servers and services I do understand the decision for the vCloud Dev Team to go with a highly scalable database platform…after all development is done for the vCloud Air Service and then handed down to SPs after. At scale it makes sense to use something like this, however it would have been nice to have the option to use a “light” database option like MSSQL or similar…that would have made sense but lets move on!

There isn’t a lot of sizing around the VM Requirements for this Cassandra cluster so I went with three VMs with 1vCPU and 4GB of vRAM with 100GB of storage to start with. The is no guidance on the growth projections so at this point it’s a case of wait and see. Would be good to have someone from the team give estimates on the size of the cluster relative to the number of VMs in an environment.

In the lab I’m using Ubuntu 14.10 but this would apply to the current 14.04.1 LTS release as well. I’ve linked to the Debian install and config at the end of this post…Below are the quick and nasty steps to download the Cassandra tarball and the required packages to build and run a single server Cassandra instance. In production I would spend a bit more time tweaking the config however it gives you an instance to wrap your head around.

The Video below shows the whole process end to end.

To satisfy the three node Cassandra Requirement we need to repeat the steps on the next two VMs and configure for HA.

Go to the extracted location and the to /conf/cassandra.yaml and edit the config file entries listed below. For the seeds, there is no need to enter in the master IP. For an explanation on the config options the config file is well commented.

Once that’s been configured on every node in the cluster, restart the Cassandra Services. To validate the cluster status use the nodetool command under the bin folder and you should see the following:

In the next post I’ll work through the KairosDB install and configuration as well as tying it all together with vCloud Director and I’ll even attempt to pull some VM stats via the API.

Cassandra Debian Install Guide:

VMware vCD SP 5.6.3 Document Link:

The Internet of Things…Flawed Hype!

Over the last 10-15 years there have been many buzz words and phrases conceived by IT professionals which are then hijacked by marketing spinsters that end up being loathed by people within the IT community. Y2K, ASP, Cloud, Big Data, Single Pane and Glass, The Internet of Things, Hyper Converged, Webscale…and more recently Hybridity (shudder).

Over the years I have forced myself to accept some of these buzz words but for the most I try to steer away from using them in general conversation and in my day to day work. It certainly took a while for me to come to grips with “The Cloud”… At that time I was working for a Hosting Company and The Cloud was a new threat for traditional hosters…hence I used to run with a hashtag of #noCloud

Though it has taken a long long time, Cloud has been mainstreamed and for the most has become an understood industry term and as such has been widely adopted. However there have been a few recent buzz words and phrases that have given myself and others in the industry a cause to vent. Phrases like Big Data and Webscale are examples that have been hijacked by marketing teams and used as generators of hype in an attempt to push products and services. To a certain extent I understand and accept that the riding of waves is a valid sales technique and is part of the IT lifecycle as shown by the Hype Cycle.

One could argue that myself and others are stuck in the Trough of Disillusionment when it comes to these words and phrases however when it comes specifically to The Internet of Things (#IoT) I’m drawing the line.


Have a look at the phrase itself and think about what it suggests?

The Internet…of Things.

The Internet of…Things.

I find this to be a dumbing down of the internet and a simpletons view of great technology…The internet was born of geniuses trying to solve issues with communication from which evolved a tool that today connects billions of people and serves to better our daily lives. To simplify this technological achievement as “things” while trying to position this “concept” as the next great evolution of the internet makes me react kind of like the guy in the picture below.

As with a lot of things in life I have the ability to polarize people with my opinion…and sometimes I think if I am off the mark when it comes to certain feelings towards subjects such as the IoT…However I recently viewed a TED Talk by Nicholas Negroponte where he spends some time going through why he thinks the IoT is “Tragically pathetic” and how it’s being marketed incorrectly.

technology is in some sense moving backward right now, dumbing things down and skewing innovation. Negroponte took an opportunity to bash the “Internet of Things” as it’s presently marketed, rightly noting that it’s often little more than putting “an oven control on your phone.” This is not innovation, he insisted. A “smart” appliance wouldn’t just be a display panel on your phone or tablet. A truly connected smart device can detect, on its own, that you’ve just put a chicken in the oven, and it should then know exactly how to prepare it, perhaps even to your liking. In his view, more work is needed in this area.

Quoting him directly below:

I look today at some of the work being done about the Internet of Things, and I think it’s kind of tragically pathetic, because what has happened is people take the oven panel and put it on your cell phone, or the door key onto your cell phone, just taking it and bringing it to you, and in fact that’s actually what you don’t want. You want to put a chicken in the oven, and the oven says, “Aha, it’s a chicken,” and it cooks the chicken. “Oh, it’s cooking the chicken for Nicholas, and he likes it this way and that way.” So the intelligence, instead of being in the device, we have started today to move it back onto the cell phone or closer to the user, not a particularly enlightened view of the Internet of Things.

Negroponte knows his stuff and he has been on the mark with a lot of technology trends across the last 40+ years…so please people…please stop using The Internet of Things…it needs to be killed off or at the very least modified (and not to “everything“) to better represent the estimated 26 Billion “things” that will be online by 2020.

Marketing Departments…the power is in your collective hands…and Cisco…I’m Looking at you! 🙂



Watch one of the best TED Talks of the last couple of years (an easy feat given the quality of talks of late) below…the bit about the IoT is at the 9 minute mark.

If you feel strongly either way, please share your thoughts below in the comment section



Time flies when you are having fun!

2014 from a work point of view was challenging at times…but overall I felt a lot was achieved. In my last couple of new year posts I’ve talked about work/life balance and looking back at 2014 I can say that overall balance was maintained…I’m pretty lucky to work in an industry where there is constant satisfaction and sense of achievement…best part is I get paid to do it and that makes the tougher times endured on occasion easier to work through. The last year flew by at such a rate of knots and there where lots of personal and work achievements.

From a personal point of view I was able to continue to refine my presentation skills by presenting at the February VMUG User Conferences in Melbourne and Sydney…the Sydney one being one of the stand out moments in my IT Career where I was able to deliver a talk in front of a crowd filled with Industry Experts (no less than 5 VCDXs) on ZettaGrids vCloud 1.5 to 5.1 upgrade.

In addition to the VMUG User Conferences I attended PEX locally here in Sydney and was lucky enough to secure a spot at VMworld 2014 in San Francisco. VMworld was again fantastic and the networking and community opportunities provided by the event are priceless. It was humbling to meet so many industry peers who I would typically interact with via Twitter and to have them validate my online contributions…on the flip side I was constantly awestruck and amazed by the openness of our industries rockstars approachability and had some great conversations with certified legends!

At VMworld I managed to sit and pass the VCAP-DCA 550 which was a fun exam and again validated a lot of the hard work I’ve put into to bettering myself technically over the last couple of years. I’m aiming to add a couple more advanced certs this year around vCloud Director and NSX-v

At ZettaGrid we managed to roll out and complete our vSphere and vCloud Platform upgrades to version 5.5 which was required as a base for a number of new set of products and features for our IaaS Cloud Service. Highlighting that new set of products was the recent production deployment of NSX-v.

Touching on NSX for a moment…Networking has never been my stronger point…I’ve always known enough to be dangerous…but there was something special about NSX that drew me in initially even though in all honesty I didn’t quiet understand it’s complete impact on IT operations, design, implementation and management. VMware made it tough to get initially but through public comment and working with the NSBU Team locally, ZettaGrid was able to sign up as one of the first Service Provider Partners in Australia/ANZ/APJ.

Having spent the last 6 months with it in my labs and working on productising features to add to ZettaGrid’s advanced networking offerings I have come to understand the significant advantages and advances a network overlay product offers…for me its less about Network Virtualization and more about efficiency. VXLAN not only simplifies virtual wire management but also is key in bridging on premises with hosted Cloud based offerings.


So what about the year ahead? Opinions…like Outages are like *holes 🙂 I’m looking at this year as a year of consolidation of previously hyped and unproven technologies and processes. There is a continual challenge working in the IaaS/Cloud/Service Provider industry to stay relevant in the face of the bigger players and their move towards the attempted monopolization of the industry…smaller players will continue to drop off and remain stagnant if the quality of their Cloud offerings are not kept high.

In my newish role as Lead Architect I have been more heavily focused on R&D and the aim for me this year is to continue to improve in my role, continue to participate in the community and to help ZettaGird continue to lead the ANZ vCloud Network Service Provider market.

Here is wishing the greater vExpert/vChampion/vBlogger/Tech Community a great year…it should be fun!

Random Poll:

What is your preferred source for Technical Documentation and Design Guides

  • Subject Matter Expert Blog Posts (75%, 9 Votes)
  • Vendor Documents and Guidlines (25%, 3 Votes)

Total Voters: 12

Loading ... Loading ...