Monthly Archives: September 2016

VCA-CLI for vCloud Director: New Networking Features

There is a lot of talk going around how IT Pros can more efficiently operate and consume Cloud Based Services…AWS has lead the way in offering a rich set of APIs for it’s clients to use to help build out cloud applications and infrastructure and there are a ton of programming libraries and platforms that have seen the rise of the DevOps movement…And while AWS has lead the way, other Public Clouds such as Azure (with PowerShell Packs) and Google have also built self service capability through APIs.

vCloud Director has always had a rich set of APIs (API Online Doco Here) and as I blogged about last year Paco Gomez has been developing a tool called VCA-CLI which is based on pyvcloud which is a Python SDK for vCloud Director and vCloud Air. This is an alternative to Web Based creation and management of vCloud Director vDCs and vApps. Being Python based you have the option of running it pretty much on any OS you like…the posts below show you how to install and configure VCA on a Mac OS X OS and Windows and how to connect up to a vCloud Director based Cloud Org.

Initial releases of VCA-CLI didn’t have the capability to configure the Firewall settings of a vDC Edge Gateway, but since the release of version 16, Firewall rule management has been added. In the below example, I connect up to my vCD Org in Zettagrid, gather some information about my vDC, deploy a SexiLog VM template, set the Syslog setting on the Gateway and then configure a new NAT and Firewall rules to open up port 8080 to the SexiLog Web interface.

And the end result:

Again, this highlights the power of the vCloud Director API and what can be done with the pyvcloud Python SDK. Once perfected the set of commands above can be used to deploy vApps and configure networking in seconds instead of having to work through the vCloud Director UI…and that’s a win win!

References:

https://pypi.python.org/pypi/vca-cli

https://github.com/vmware/vca-cli

http://www.sexilog.fr/

 

The Anatomy of a vBlog Part 2: Plugins, Site Optimizations and Analytics

Part 1 – Building a Blogging Platform

Having looked at hosting platform and operating system suggestions in Part 1, to conclude this two part series I’ll talk about how to make WordPress work harder for you through its plugin ecosystem as well as go through the site optimizations and caching improvements offered by CloudFlare. To finish off I’ll talk about GoSquared which is an external analytics engines that keeps track of site visitors and page views.

WordPress Plugins:

WordPress having been the defacto blogging engine for a number of years now has enabled a whole ecosystem of free and paid for plugins that are used to enhance the usability of your WordPress site. Think about these plugins similar to IOS Apps in that, just like just like the App Store they are easily searchable and installable from the Administration Plugin Menu and for better or worse…they are ultimately what keep you invested the WordPress platform…just like Apps on the iPhone.

In terms of plugin management, the WordPress platform makes it easy to install, configure and upgrade all the plugin from the one menu page. Up to this point I haven’t had any major issues with the plugins I use even. In terms of what plugins I use to help improve the readability, usability and socialability of the site, I’ve listed the plugins I consider core to this site below:

  • CloudFlare: Integrates your blog with the CloudFlare platform.
  • Crayon Syntax Highlighter: A Syntax Highlighter built in PHP and jQuery that supports customizable languages and themes.
  • GoSquared: Add GoSquared tracking code directly to your WordPress site.
  • Image Formatr: A simple plugin that goes through all the content images on posts & pages, and with zero user changes
  • Jetpack: Simplifies managing WordPress sites by giving you visitor stats, security services, speeding up images, and helping you get more traffic. Jetpack is a free plugin
  • Revive Old Post: Helps you to keeps your old posts alive by sharing them and driving more traffic to them from social networks. It also helps you to promote your content.
  • Yoast SEO: Written from the ground up by Joost de Valk and his team at Yoast to improve your site’s SEO on all needed aspects

TIP: Take a look at what features paid for plugins offer over free ones. Just like any software, you will always find an open/free alternative. Some plugins will also come in a lite version with certain features locked to a paid for version.

CloudFlare Optimizations:

As a new blog is starting off the amount of traffic hitting the site is generally small so having the site directly exposed on the internet isn’t usually a problem, however as your site grows you may need to consider fronting the site with a caching or performance engine. Security should also be a consideration to help protect you blog against malicious attacks or code vulnerabilities and exploits.

In the early days of the internet Akamai dominated web geocaching services and a lot of the world’s largest high volume sites used them to improved user experience and protect origin servers from traffic spikes. CloudFlare offers similar services to Akamai, but they do things differently… Their story is worth a read to get an idea of where they came from and what they are trying to achieve. https://www.cloudflare.com/our-story

CloudFlare-powered websites see a significant improvement in performance and a decrease in spam and other attacks. On average, a website on CloudFlare:

  • Loads twice as fast
  • Uses 60% less bandwidth
  • Has 65% fewer requests
  • Is way more secure

CloudFlare can be used regardless of your choice in platform. Setup takes most about five to ten minutes. Adding a website requires your domain’s DNS records to be hosted at CloudFlare (for free) and then make a couple of adjustments to the origin URL’s of your site and have the domain NS records point at CloudFlare’s name servers. A, AAAA, and CNAME records can have their traffic routed through the CloudFlare system. The core service is free and they do offer enhanced services for websites who need extra features like real time reporting or SSL.

As you can see below, CloudFlare offers a number of tweaking options, most of which are available on the free plan.

The efficiency in terms of bandwidth savings is also significant

The Firewall features is also impressive and works to block IP addresses trying to cause issues and launch brute force attacks on sections of the WorpdPress site such as /wp-admin

Having CloudFlare front your site is a no brainier and given that there is a very feature rich’s free version that is extremely effective its something to configure for all blogging sites. Or to add to your existing site. For a look at the specific plan capabilities, click here.

TIP: Comment SPAM can be a significant PITA for bloggers, and in the early days I would spend ten to thirty minutes a week cleaning up unmoderated comments. With CloudFlare in play the amount of comment SPAM has dropped down to almost non-existent levels.

GoSqaured Analytics:

GoSquared takes what JetPack does and elevates it to another level. This is one of the few external services that I have no trouble paying for because, as someone who loves numbers and trend analytics it delivers everything I need to keep tabs of what’s happening on the site. GoSquared offers real time stats on site visitors and as shown below gives you deep insights into not only, where people are visiting you site from, but a lot about what platform they are using to browse.

It works by downloading the WordPress plugin and entering the tracker code that in turn injects a bit of code onto every page from which the live tracking stats are received. They also have a free plan option, but it’s worth looking at the paid plans as your site grows.

https://www.gosquared.com/plans/

TIP: By looking at the site visit graphs you will start to get a feel for when your site is most accessed and from where the site visits occur. From this you will be able to deduct the best time for which to publish a new blog post.

Conclusion:

I hope this two part series has been helpful in breaking down the obvious and less obvious components of a blogging site and more specifically the Virtualization is Life! site that is running WordPress. As mention in Part 1, there is no right answer to what blogging platform is best, however my preference is to keep things under total control all while having a simple and efficient platform from which to create and distribute content. The tools that I have mentioned that go on top of the WordPress site are also vital in keeping things ticking over.

Hope this was useful for some!

The Anatomy of a vBlog Part 1: Building a Blogging Platform

Earlier this week my good friend Matt Crape sent out a Tweet lamenting the fact that he was having issues uploading media to WordPress…shortly after that tweet went out Matt wasn’t short of Twitter and Slack vCommunity advice (follow the Twitter conversation below) and there where a number of options presented to Matt on how best to host his blogging site Matt That IT Guy.

Over the years I have seen that same question of “which platform is best” pop up a fair bit and thought it a perfect opportunity to dissect the anatomy of Virtualization is Life!. The answer to the specific question as to which blogging platform is best doesn’t have a wrong or right answer and like most things in life the platform that you use to host your blog is dependent on your own requirements and resources. For me, I’ve always believed in eating my own dog food and I’ve always liked total end to end control of sites that I run. So while, what I’m about to talk about worked for me…you might like to look at alternative options but feel free to borrow on my example as I do feel it gives bloggers full flexibility and control.

Brief History:

Virtualization is Life! started out as Hosting is Life! back in April of 2012 and I choose WordPress at the time mainly due to it’s relatively simple installation and ease of use. The site was hosted on a Windows Hosting Platform that I had built at Anittel, utilizing WebsitePanel on IIS7.5, running FastCGI to serve the PHP content. Server backend was hosted on a VMware ESX Cluster out of the Anittel Sydney Zones. The cost of running this site was approximately $10 US per month.

Tip: At this stage the site was effectively on a shared hosting platform which is a great way to start off as the costs should be low and maintenance and uptime should be included in the hosters SLA.

Migration to Zettagrid:

When I started at Zettagrid, I had a whole new class of virtual infrastructure at my hands and decided to migrate the blog to one of Zettagrid’s Virtual DataCenter products where I provisioned a vCloud Director vDC and created a vApp with a fresh Ubuntu VM inside. The migration from a Windows based system to Linux went smoother than I thought and I only had a few issues with some character maps after restoring the folder structure and database.

The VM it’s self is configured with the following hardware specs:

  • 2 vCPU (5GHz)
  • 4GB vRAM
  • 20GB Storage

As you can see above the actual usage pulled from vCloud Director shows you how little resource a VM with a single WordPress instance uses. That storage number actually represents the expanded size of a thin provisioned disk…actual used on the file system is less than 3GB, and that is with four and a half years and about 290 posts worth of media and database content  I’ll go through site optimizations in Part 2, but in reality the amount of resources required to get you started is small…though you have to consider the occasional burst in traffic and work in a buffer as I have done with my VM above.

The cost of running this Virtual Datacenter in Zettagrid is approx $120 US per month.

TipEven though I am using a vCloud Director vDC, given the small resource requirements initially needed a VPS or instance based service might be a better bet. Azure/AWS/Google all offer instance based VM instances, but a better bet might be a more boutique provider like DigitalOcean.

Networking and Security:

From a networking point of view I use the vShield/NSX Edge that is part of vCloud Director as my Gateway device. This handles all my DHCP, NAT and Firewall rules and is able to handle the site traffic with ease. If you want to look at what capabilities the vShield/NSX Edges can do, check out my NSX Edge vs vShield Series. Both the basic vShield Edges and NSX Edges have decent Load Balancing features that can be used in high availability situations if required.

As shown below I configured the Gateway rules from the Zettagrid MyAccount Page but could have used the vCloud Director UI. For a WordPress site, the following services should be configured at a minimum.

  • Web (HTTP)
  • Secure Web (HTTPS)
  • FTP (Locked down to only accept connections from specific IPs)
  • SSH (Locked down to only accept connections from specific IPs)

OS and Web Platform Details:

As mentioned above I choose Ubuntu as my OS of choice to run Wordpress though any Linux flavour would have done the trick. Choosing Linux over Windows obviously means you save on the Microsoft SPLA costs associated with hosting a Windows based OS…the savings should be around $20-$50 US a month right there. A Linux distro is a personal choice so as long as you can install the following modules it doesn’t really matter which one you use.

  • SSH
  • PHP
  • MySQL
  • Apache
  • HTOP

The only thing I would suggest is that you use a long term support distro as you don’t want to be stuck on a build that can’t be upgraded or patched to protect against vulnerability and exploits. Essentially I am running a traditional LAMP stack, which is Linux, Apache, MySQL and PHP built on a minimal install of Ubuntu with only SSH enabled. The upkeep and management of the OS and LAMP stack is not much and I would estimate that I have spent about five to ten hours a year since deploying the original server dealing with updates and maintenance. Apache as a web server still performs well enough for a single blog site, though I know many that made the switch to NGINX and use the LEMP Stack.

The last package on this list is a personal favorite of mine…HTOP is an interactive process viewer for Unix systems that can be installed with a quick apt-get install htop command. As shown below it has a detailed interface and is much better than trying to work through standard top.

TipIf you don’t want to deal with installing the OS or installing and configuring the LAMP packages, you can download a number of ready made appliances that contain the LAMP stack. Turnkey Linux offers a number of appliances that can be deployed in OVA format and have a ready made LAMP appliance as well as a ready made WordPress appliance.

That covers off the hosting and platform components of this blog…In Part 2 I will go through my WordPress install in a little more detail and look at themes and plugins as well as talk about how best to optimize a blogging site with the help of free caching and geo-distribution platforms.

References and Guides:

http://www.ubuntu.com/download/server

http://howtoubuntu.org/how-to-install-lamp-on-ubuntu

https://www.digitalocean.com/community/tutorials/how-to-install-linux-nginx-mysql-php-lemp-stack-in-ubuntu-16-04

NSX Bytes: vCloud Director Can’t Deploy NSX Edges

Over the weekend I was tasked with the recovery of a #NestedESXi lab that had vCloud Director and NSX-v components as part of the lab platform. Rather than being a straight forward restore from the Veeam backup I also needed to downgrade the NSX-v version from 6.2.4 to 6.1.4 for testing purposes. That process was relatively straight forward and involved essentially working backwards in terms of installing and configuring NSX and removing all the components from vCenter and the ESXi hosts.

To complete the NSX-v downgrade I deployed a new 6.1.4 appliance and connected it back up to vCenter, configured the hosts, setup VXLAN, transport components and tested NSX Edge deployments through the vCenter Web Client. However, when it came time to test Edge deployments from vCloud Director I kept on getting the following error shown below.

Checking through the NSX Manager logs there was no reference to any API call hitting the endpoint as is suggested by the error detail above. Moving over to the vCloud Director Cells I was able to trace the error message in the log folder…eventually seeing the error generated below in the vcloud-container-info.log file.

As a test I hit the API endpoint referenced in the error message from a browser and got the same result.

This got me thinking that the error was either DNS related or permission related. After confirming that the vCloud Cells where resolving the NSX Manager host name correctly, as suggested by the error I looked at permissions as the cause of the 403 error. vCloud Director was configured to use the service.vcloud service account to connect to the previous NSX/vShield Manager and it dawned on me that I hadn’t setup user rights in the Web Client under Networking & Security. Under the Users section of the Manage Tab the service account used by vCloud Director wasn’t configured and needed to be added. After adding the user I retried the vCD job and the Edge deployed successfully.

While I was in this menu I thought I’d test what level of NSX User was required to for that service account to have in order to execute operations against vCloud Director and NSX. As shown below anything but NSX or Enterprise Administrator triggered a “VSM response error (254). User is not authorized to access object” error.

At the very least to deploy edges, you require the service account to be NSX Administrator…The Auditor and Security Administrator levels are not enough to perform the operations required. More importantly don’t forget to add the service account as configured in vCloud Director to the NSX Manager instance otherwise you won’t be able to have vCloud Director deploy edges using NSX-v.

 

 

Cross Cloud: Why The VM Shouldn’t Be The Base Unit of Measurement

I’ve been sitting on this topic since the VMworld 2016 US Keynote where VMware announced the Cross Cloud Architecture. I posted some raw thoughts the day after keynote and have been reflecting on how the Cross Cloud Platform could impact on VMware’s vCAN business. As mentioned previously I believe it’s representative of how VMware is worrying over it’s future relevance and reacting to current market fads all while ultimately worrying about how the hyper-scalers will impact their core infrastructure business.

The concept of cross cloud isn’t new and in truth a lot of vendors today are working to, or have solutions that aim to convert workloads from one platform to another. Zerto do this with their Cloud Fabric with the ability to move certain VMs from ESXi to Hyper-V, AWS and Azure and every combination in between. Veeam also have a new feature where you can restore ESXi or Hyper-V VMs to Azure…again, limited in functionality but a strong indication of what’s to come given the latest Veeam announcements.

Both Zerto and Veeam market their solutions well, however those that have been involved in V2Vs know that only under certain conditions do conversions go smoothly. There is no doubt this cross platform world is getting more reliable and more and more vendors are chasing the perfect conversion. However what Veeam and Zerto are offering is Backup and DR services that complement VM workloads either on-premises or in a cloud…the end game with these products isn’t mobility…its availability.

Focusing back on VMware it was clear to almost everyone that the Cross Cloud Platform featuring Azure and AWS workload migrations, was tech previewed to show that VMware is relevant in an enterprise multi cloud world but I am going to argue that the focus on the VM as the base unit of measurement is misguided…especially when it comes to VMware supporting it’s vCloud Air Network providers. I understand it as a necessity being able to have a class of portable applications in this new microservice and serverless world while having them transportable between multiple clouds. Again, I don’t believe the VM should be the base unit of measurement and the unit shown to be the most transportable.

Service providers need to play to their strengths, which in the vCAN world is no bill shock fixed cost IaaS workloads. This remains the base platform for a significant portion of any on-premises or cloud workload. Service providers take most of their revenue stream from compute, storage and networking that are the building blocks of instance based and resource pool offerings from which VMs can be provisioned and consumed. If you ask any service provider they would say that they would like total VM stickiness and any mechanism that aims to make VMs more portable will impact the bottom line and threatens ongoing viability.

Having customers access a VMware provided console that moves VM workloads off VMware based infrastructure and onto AWS or Azure to my mind is close to madness, and while there is an argument to suggest that cloud is the new hardware and VMware want to manage this new hardware…it still doesn’t make up for the fact that most revenue is made by having VMs staying local and not having an easy way to migrate them to platforms where smaller margins are the norm.

Going back to the point of this post around the theory that the VM shouldn’t be the base unit in a cross cloud world, I believe that for the sake of the vCAN VMware should be focusing within the VM and the applications that run within them…working towards a truly hybrid scenario whereby Platform and Feature as a Service offerings are managed, configured and operated via the Cross Cloud platform. This will help achieve a sustained revenue stream for IaaS providers that in truth, still represents the best value for money for the vast majority of critical business applications that are in existence today, all while allowing consumers the choice of going out and finding the best “As a Service” offering that specifically suits application requirements.

At the end of the day I do wonder which side of the VMware business wins out…the one that derive their revenue from Enterprise…or the one that derive their revenue from Service Providers. Unfortunately I know where the bigger revenue streams lie and that doesn’t bode well for Service Providers. It’s all about the corporate dollar after all.

CloudPhysics: Rightsizing Intelligence and Cost Calculator for Private Cloud

CloudPhysics have been a little quiet over the past twelve or so months with focus shifting from presenting data via Cards to Dashboards and also focusing on delivering on boarding solutions for managed service provider partners that has resulted in their channel business growing successfully. Before VMworld they announced the release of their Cost Calculator for Private Clouds in addition to releasing a couple more dashboards for their SaaS based landing page as well as adding a tagging feature for VMs and other objects.

CloudPhysics roots is all about data science and what can be achieved with literally billions of data points…so it’s no surprise that they are starting to put that front and center when it come to their new feature capabilities. Rightsizing at the 99th and 95th percentile usually cuts off the top 5% or 1% of metric peaks, and then presents the data at the nearest metric rate. In this way infrequent peaks are ignored, and the data is better suited to making decisions against. Now CloudPhysics rightsizing can be applied with intelligence to virtual machines and compute/storage infrastructure and capture savings by reducing workloads to match actual demands and reduce over provisioning.

The CloudPhysics Cost Calculator for Private Cloud lets you apply basic costing models to determine your actual costs per virtual machine (VM) in terms of power, compute resources, memory, storage, licensing, and more to generate a cost baseline.

As you can see below the new Card gives you the option to enter in cost points for most input items in a typical private cloud situation. They have not only included standard costs of servers hardware, memory and storage but also given you options to enter in depreciation terms, hypervisor cost details, environment costs relating to power and cooling but also additional 3rd party license costs that could be used for backup or acceleration software.

Once entered in you can filter through your platform as seen by the CloudPhysics Observer and get an understanding of what each individual VM is costing you in relation to your inputs. You also get a Cost as Configured amount that can be adjusted for the 99th and 95th percentile as well.

This view really gives you an understanding of what VMs are costing you the most and then get an idea of how to plan for any move to a public cloud where rightsizing based on more than just maximums is key. There is an option to click on the Compare Cloud Costs button which takes you to a new sister Card that displays the side by side cost of hosting your private cloud on AWS or Azure and again lets you manipulate the data with rightsizing.

In talking with the CloudPhysics team I’m hopeful that they will add to this card to include vCloud Air Network service providers running vSphere based IaaS platforms. I’m sure the 4000 odd vCAN SPs would appreciate a direct comparison for potential new customers looking to make a choice between the hyperscalers and their on platforms.

New Dashboard Items and Tags:

As mentioned in the opening paragraph CloudPhysics also added a couple new dashboards that can be configured to look at a number of different VM and Host metrics and show a trend over the last one, seven for thirty days. These new dashboard items as shown below are extremely handy for being bale to pick up problem objects in your infrastructure.

Also added is the basic ability to add Tags to VMs for easier searching from withing the CloudPhysics interface. In future these will be possibly integrated with vSphere tags which would be a welcome feature as more and more people are implementing tags for Storage Based Policy Management and Backup Management.

All in all another great set of enhancements to the CloudPhysics platform and I can tell you all that you need to keep an eye on what the team has in store for the next 6-12 months as I believe they are ready to take their offering to the next level and expand well and truly beyond anything they have done up to this point.

They have a free edition which you can tryout here: CloudPhysics Free Edition

Additional Content:

Chris Schin from goes through some of the new features during VMworld.

Resources:

https://en.wikipedia.org/wiki/Percentile

http://vmblog.com/archive/2016/08/25/cloudphysics-unveils-cost-calculator-for-private-cloud-with-public-cloud-comparison-tool.aspx#.V9au3Lh94-W

NSX Bytes: NSX-v 6.2.4 Released …Important Upgrade!

NSX-v 6.2.4 was released the week before VMworld US so might have gotten somewhat lost in the VMworld noise…For those that where fortunate enough to not upgrade to or deploy a greenfield 6.2.3 site you can now safely do so without the nasty bugs that existed in the 6.2.3 build. In a nutshell this new build delivers all the significant features and enhancements announced in 6.2.3 without the dFW or Edge Gateway bugs that forced the build being pulled from distribution a few weeks back.

In terms of how and when to upgrade from previous versions the following table gives a great overview of the pathways required to get to 6.2.4.

The take away from the table above is that if possible you need to get onto NSX-v 6.2.4 as soon as possible and with good reason:

  • VMware NSX 6.2.4 provides critical bug fixes identified in NSX 6.2.3, and 6.2.4 delivers a security patch for CVE-2016-2079 which is a critical input validation vulnerability for sites that uses NSX SSL VPN.
  • For customers who use SSL VPN, VMware strongly recommends a review of CVE-2016-2079 and an upgrade to NSX 6.2.4.
  • For customers who have installed NSX 6.2.3 or 6.2.3a, VMware recommends installing NSX 6.2.4 to address critical bug fixes.

Prior to this release if you had upgraded to NSX-v 6.1.7 you where stuck and not able to upgrade to 6.2.3. The Upgrade matrix is now reporting that you can upgrade 6.1.7 to 6.2.4 as shown below.

I was able to validate this in my lab going from 6.1.7 to 6.2.4 without any issues.

NSX-v 6.1.4 is also fully supported by vCloud Director SP 8.0.1 and 8.10

References:

http://pubs.vmware.com/Release_Notes/en/nsx/6.2.4/releasenotes_nsx_vsphere_624.html

http://www.theregister.co.uk/2016/07/22/please_dont_upgrade_nsx_just_now_says_vmware/

VMworld 2016: Top Session Pick Videos Right Here!

VMworld 2016 US is done and dusted and for those that didn’t attend or attended but missed out on sessions due to “scheduling conflicts”, VMware has been awesome in opening up the sessions catalog recordings to everyone. In previous years this was limited to attendees only and only opened up after a period of time however this year VMware have made them available here after entering in some basic form details.

http://www.vmworld.com/en/sessions/2016.html

Before VMworld 2016 kicked off I listed my Top Session Picks for this years event. I’ve gone through and embedded all the awesome sessions below for your viewing pleasure.

  • Virtual SAN – Day 2 Operations [STO7534]
  • Advanced Network Services with NSX [NET7907]
  • A Day in the Life of a VSAN I/O [STO7875]
  • vSphere 6.x Host Resource Deep Dive [INF8430]
  • The Architectural Future of Network Virtualization [NET8193R]
  • Conducting a Successful Virtual SAN 6.2 Proof of Concept [STO7535]
  • PowerNSX and PyNSXv: Using PowerShell and Python for Automation and Management of VMware NSX for vSphere [NET7514]
  • Evolving the vSphere API for the Modern Era [INF8255]
  • Multisite Networking and Security with Cross-vCenter NSX: Part 2 [NET7861R]

I was blown away with the quality of all those sessions listed, but I must make special mention to Frank Denneman and Niels Hagoort for their session [INF8430] which I’ve already watched and listened to a couple times. It’s truly one of the best deeply technical sessions you will ever come across at any VMworld and full of brilliant insight into compute, storage and networking.

NOTE: IF the Video’s don’t load and ask for a username/password you may need to hit the registration link here and then refresh this page.

Also up and available are the vBrownBag Tech Talks and while I didn’t participate this year it looks like it was a great success once again with awesome quality content delivered by the community.

Enjoy!