Tag Archives: CloudPhysics

CloudPhysics: Rightsizing Intelligence and Cost Calculator for Private Cloud

CloudPhysics have been a little quiet over the past twelve or so months with focus shifting from presenting data via Cards to Dashboards and also focusing on delivering on boarding solutions for managed service provider partners that has resulted in their channel business growing successfully. Before VMworld they announced the release of their Cost Calculator for Private Clouds in addition to releasing a couple more dashboards for their SaaS based landing page as well as adding a tagging feature for VMs and other objects.

CloudPhysics roots is all about data science and what can be achieved with literally billions of data points…so it’s no surprise that they are starting to put that front and center when it come to their new feature capabilities. Rightsizing at the 99th and 95th percentile usually cuts off the top 5% or 1% of metric peaks, and then presents the data at the nearest metric rate. In this way infrequent peaks are ignored, and the data is better suited to making decisions against. Now CloudPhysics rightsizing can be applied with intelligence to virtual machines and compute/storage infrastructure and capture savings by reducing workloads to match actual demands and reduce over provisioning.

The CloudPhysics Cost Calculator for Private Cloud lets you apply basic costing models to determine your actual costs per virtual machine (VM) in terms of power, compute resources, memory, storage, licensing, and more to generate a cost baseline.

As you can see below the new Card gives you the option to enter in cost points for most input items in a typical private cloud situation. They have not only included standard costs of servers hardware, memory and storage but also given you options to enter in depreciation terms, hypervisor cost details, environment costs relating to power and cooling but also additional 3rd party license costs that could be used for backup or acceleration software.

Once entered in you can filter through your platform as seen by the CloudPhysics Observer and get an understanding of what each individual VM is costing you in relation to your inputs. You also get a Cost as Configured amount that can be adjusted for the 99th and 95th percentile as well.

This view really gives you an understanding of what VMs are costing you the most and then get an idea of how to plan for any move to a public cloud where rightsizing based on more than just maximums is key. There is an option to click on the Compare Cloud Costs button which takes you to a new sister Card that displays the side by side cost of hosting your private cloud on AWS or Azure and again lets you manipulate the data with rightsizing.

In talking with the CloudPhysics team I’m hopeful that they will add to this card to include vCloud Air Network service providers running vSphere based IaaS platforms. I’m sure the 4000 odd vCAN SPs would appreciate a direct comparison for potential new customers looking to make a choice between the hyperscalers and their on platforms.

New Dashboard Items and Tags:

As mentioned in the opening paragraph CloudPhysics also added a couple new dashboards that can be configured to look at a number of different VM and Host metrics and show a trend over the last one, seven for thirty days. These new dashboard items as shown below are extremely handy for being bale to pick up problem objects in your infrastructure.

Also added is the basic ability to add Tags to VMs for easier searching from withing the CloudPhysics interface. In future these will be possibly integrated with vSphere tags which would be a welcome feature as more and more people are implementing tags for Storage Based Policy Management and Backup Management.

All in all another great set of enhancements to the CloudPhysics platform and I can tell you all that you need to keep an eye on what the team has in store for the next 6-12 months as I believe they are ready to take their offering to the next level and expand well and truly beyond anything they have done up to this point.

They have a free edition which you can tryout here: CloudPhysics Free Edition

Additional Content:

Chris Schin from goes through some of the new features during VMworld.

Resources:

https://en.wikipedia.org/wiki/Percentile

http://vmblog.com/archive/2016/08/25/cloudphysics-unveils-cost-calculator-for-private-cloud-with-public-cloud-comparison-tool.aspx#.V9au3Lh94-W

CloudPhysics Exploration Mode – New Host View

Late last year CloudPhysics released their VM Exploration mode feature which allowed for a detailed look into what was happening holistically to a VM with the ability to view key metrics and VM related events over an extended period of time. Last weekend CloudPhsyics extended this to also include Hosts. Extending Exploration Mode to include Hosts further improves the proactive monitoring and analysis capabilities of the CloudPhysics platform as it looks to break away from its roots of Card Views.

With Exploration Mode now encompassing both VMs and hosts, administrators can focus in on a workload performance issue and “replay” the environment to correlate events, resource utilization patterns, and environment changes in the seconds, minutes or days leading up to a problem in application performance or availability.

To view a Host with Exploration Mode, you use the new Search Virtual Machines and Hosts bar at the top of the CloudPhysics Web Console.

Once the Host has been selected you get taken to a dashboard that gives you configuration details of the Host, any changes (Power Operations, Snapshot, vMotions) that have been done against that VM in the provided date range and a performance graph that covers CPU, Memory, Network and Storage. There is also an Issues section which alerts you to any possible configuration issues or mismatch.

There is also the introduction of a Tab View which allows you to have open multiple Hosts and/or VMs to compare against…what would be nice would be the ability to overlay both Hosts and VMs to try and pinpoint events or key metrics points as they happened.

Below is a YouTube video from a recent webinar where the CloudPhysics VP Product Management Chris Schin walks through the way the platform uses Exploration Mode to identify root causes of VM Performance issues.

If you are interested in giving CloudPhysics a try, they have a free edition which you can register for and download here: CloudPhysics Free Edition

Quick Post: CloudPhysics Weekly Digest

CloudPhysics are still quietly working away in the background continuing to improve their analytics service…and apart from recently announcing record results for 2015 have started to send out a weekly digests which gives a great snapshot of whats happened in your vSphere environment over the course of the previous week.

The digest contains the following:


 

Environment Summary As of 1 Feb Weekly Change
vCenters 3 0
Observers Online 3 0 —
Observers Offline 0 0 —
VMs 4071 0
Powered on 3086 3 ▲
Powered off 979 2 ▼
Suspended 6 1 ▼
Templates 0 0 —
Hosts 169 1
Clustered 168 1 ▲
Standalone 1 0 —
Avg. VMs (on) / Host (on) 30.7 0.4 ▼
Clusters 13 0
Datastores 64 0
Total Storage 1805.09 TB < 0.01 TB ▲
Free Storage 1477.48 TB 2.02 TB ▲
Events: 25 Jan – 1 Feb
VMs with Config Changes 90
VMs Created 7
vMotions 337

Issues: As of 1 Feb

We found 5 critical issues from VMware and top datacenter vendor knowledge base articles that are highly relevant to your vSphere installation.

Visit KBA Card

1 virtual machine(s) have snapshots using more than 75% of the disk space used by all their respective virtual disks.

Snapshots Gone Wild

4 host(s) are running unsupported VMware ESXi versions.

3 host servers are not supported by VMware.

Host Inventory

26 VM(s) have experienced performance degradation due to datastore contention in last 24 hours.

VM I/O Contentions

2 datastore(s) are in inaccessible state.

Datastore Space

Best Regards,
CloudPhysics


 

They have also made public the vCenter “Deep Linking” feature which I had been beta testing and using for a while…its a great mechanism to attend to issues directly from the CloudPhyics web UI and have the vCenter Web Client launch directly to the areas that requires attention.

Click here to start a free trial that can be up and running in 15 minutes!

Firstlook: CloudPhysics Exploration Mode

During VMworld CloudPhysics released their new Dashboard Feature which saw a change of direction in the way CloudPhysics customers get presented with their data and was the first time Card Based analytics was not used to allow access to the wide array of metrics CloudPhysics stores in their data warehouses.

I’ve been working closely with the CloudPhysics team for a number of years now and they are great at listening to feedback around how to improve the platform. One of my biggest gripes (if you could call it that) over the years was that there was no way to view in detail (and historically) what was happening to a particular VM. One of the other issues was the time it took for data to show up in the CloudPhysics UI which meant that you could get access to data after about thirty minutes.

With the release of Exploration Mode there is more a case for proactive monitoring and analysis of VMs and their issues and the data refresh rate has been brought down to about 15 minutes which allows for more real time troubleshooting as well as allowing us to go back in time a number of days to try and correlate issues and try to look at patterns that might have occurred over the course of those days.

With Exploration Mode, administrators can go back in time, correlating events, issues, and changes that are associated with any selected time range in the vSphere environment, making it possible for users to see exactly what transpired in the seconds, minutes or days leading up to an application performance or availability problem.

To view a VM with Exploration Mode, you use the new Search VMs bar at the top of the CloudPhysics Web Console.

Once the VM has been selected you get taken to a dashboard that gives you configuration details of the VM, any changes (Power Operations, Snapshot, vMotions) that have been done against that VM in the provided date range and a performance graph that covers CPU, Memory, Network and Storage. There is also an Issues section which alerts you to any possible

CloudPhysics have always been a personal favorite of mine and I’m legitimately excited with what the team has got in store to further develop the platform into an extremely powerful analytics tool for VMware based platforms.

They have a free edition which you can tryout here: CloudPhysics Free Edition

Sources:

http://www.marketwired.com/press-release/cloudphysics-releases-exploration-mode-which-lets-vmware-users-identify-root-cause-2069851.htm

#VMworld: First Look – CloudPhysics New Release

Over the past year the guys at CloudPhysics have been relatively quiet compared to the proceeding 3 years since they burst onto the Scene at VMworld 2012. The reason relative radio science has been had was that they have been busily working away on a revamp of their SaaS based Analytics platform…and the results are impressive.

The new release has the following highlights.

  • Always-on diagnostics: Continuous diagnosis of infrastructure with changes continuously captured, recorded and reflected. Unique data derivations, correlations, mashups and filters reduce “noise” and identify true hazards.
    • Configurable dashboards: Rich contextual views expose hot spots and potential risks before problems form and impact operations. Trending analysis consolidates multiple objects and views, enabling multi-dimensional correlation.
    • Groundbreaking exploration capabilities: Interactive ability to analyze changes over time through easily manipulated exploration mode, using time slices with zoom in/out capabilities to evaluate correlations and causation. Users can “correlate in context” to troubleshoot application disruptions with data drawn from VM performance/resource consumption; change/event log; configuration history; and known issues associated with operational hazards and best practices.
    • 20+ new analytics for managing health and preempting hazards, available in our extensive library of “cards.”
    • Platform innovation: Time series data is uniquely handled by the CloudPhysics platform to enable a user to analyze multiple dimensions of the infrastructure around the same time axis.

With the new features, CloudPhysics delivers unique, meaningful insights, giving vSphere teams the confidence to act boldly to reduce risk and waste without compromising safety of the virtual infrastructure or the applications it supports. Building on its ease of use, intuitive user interface and deep visibility across multiple vCenters, CloudPhysics now:

  • Reduces disruption and incidents with always-on diagnostics that surface hot spots and emerging problems, enabling admins to get ahead of nascent performance problems• Improves mean-time-to-resolution (MTTR) with directed exploration, enabling admins to zero in on root cause and resolve application disruptions more quickly• Generates insights for realigning misconfigured infrastructure to prevent future performance and availability issues and improve efficiency

CloudPhysics have always been a personal favourite of mine since I first chatted to Irfan back on the Solutions Exchange floor in 2012 and I’m legitimately excited with what the team has got in store to further develop the platform into an extremely powerful analytics tool for VMware based platforms.

They have a free edition which you can tryout here: CloudPhysics Free Edition

While at VMworld, head over to Booth #2346

CloudPhysics: Heartbleed Health Checks for vCenter and ESXi

The power of a service like CloudPhysics continues to grow almost weekly as they add new features and Cards. Not only is it brilliant for analytics and metrics but we are now seeing CloudPhysics release cards that aim to help VMware Administrators keep track of possible security vulnerabilities in their platforms.

This week they started a campaign to alert CloudPhysics subscribers to the fact that approx. 57% of vCenter servers and 58% of ESXi Hosts are still not patched with the latest Heartbleed update builds. Based on their own big data analytics they have found that 40% of their client base is still unprotected.

Emails where sent out to affected users earlier this week and @virtualirfan has written a blog on the CloudPhysics site detailing the concern over those numbers quoted above. If you are a current CloudPhysics user, go to the Card Store and search for Heartbleed. Add the card to the desk and run the check.

If you are not a current subscriber follow the steps below:

Three easy steps to rid yourself of Heartbleed

Is your organization part of the 40%? There’s no reason you should be. The fix is easy and the risk is not worth taking. And CloudPhysics is making it even easier: we’ve packaged up the VMware Heartbleed analytic we ran across our global data set, and it’s now available in our community (free) edition for users to run on their own VMware environments. What you can do:

  1. If you haven’t already, get CloudPhysics up and running in your datacenter (takes just a few minutes).
  2. Select and run the “Heartbleed Check.” You’ll find it in the Card Store. It will immediately show you precisely which ESXi hosts remain unprotected in your datacenter.
  3. Apply the patch(es). Here’s the table listing build numbers for the patches we’ve discussed here.

Source: http://www.cloudphysics.com/blog/vmware-heartbleed/

CloudPhysics: Enhanced Storage Analytics Cards [Part 2] – Snapshots Gone Wild 2

Following up from Part 1 which focused on the Datastore Contention v2 Card, I’ll shift focus to what can sometimes be a Virutalization Admins worst nightmare…Snapshots. Snapshots are not backups (repeat x100) …but backup systems such as Veeam utilize VMware Snapshots to do their thing…and sometimes the process of SnapShot consolidation can fail, leaving unplanned SnapShots in play. Outside of that, the most dangerous Snapshot is the one that is manually created and forgotten.

There are plenty of ways to report on snapshots and most monitoring tools can be used to check, warn and alert on their presence. I still use a combination of reporting methods to ensure that the worst case scenario of a filled up datastore…or worse, corrupt snapshot do not occur.

Before looking at how CloudPhysics does it’s thing, traditionally I would have (and still do) use a simple PowerCli command to search all VMs and report on the age and size of any snapshots in a vCenter.

Pretty basic, but does the job.

CloudPhysics first introduced a SnapShots Gone Wild Card back during VMWorld 2012 and while it’s gone through a few changes and improvements since then it’s core look and feel has remained the same:

CloudPhysics have recently released V2 of the SnapShots Gone Wild Card and much the same as the Datastore Contention V2 Card there are quiet a few enhancements to go along with a more dynamic look and feel.

The one thing that I keep on commenting on in regards to CloudPhysics is that they present the data that matters so it’s right in your face. The new Card defaults to those SnapShots that need attention relative to SnapShot size, age and growth.

The graphics at the top break down DataStore Space Usage across the datastores with SnapShots and also provides a rough Savings Opportunity which is handy for Enterprises looking to put a $$ value on having SnapShots sticking around consuming otherwise useful datastore space. You can modify the price per GB to suit your own storage costs.

From a Service Providers point of view the value of this Card is in the quick visual representation of SnapShots in the CloudPhysics monitored environment…and while not proactive in nature (for that I would strongly suggest monitoring and reporting on the size of the SnapShot vmdk file useing Nagios, OpsView or using the vCenter SnapShot Alarm + Trigger mechanism)  it’s brilliant as a way to keep visual tabs on whats going on under the surface of your VMs.

The one thing I would like to have added to this Card is some form of automated reporting and/or alerting. I know that the guys at CloudPhysics are working on this set of features…this is one Card that would certainly benefit from that!

CloudPhysics: Enhanced Storage Analytics Cards [Part 1] – Datastore Contention

The guys at CloudPhysics have been busy behind the scenes of late working on improving an already great Analytic and Monitoring platform and recently I was able to preview a new enhanced set of Storage Analytic Cards. These cards are currently in Preview and with an official write up here by @esxtopGuru Which goes through the different cards on offer.

Coming from Service Provider land, I am always extremely interested in being able to find out how my datastores are performing and which VMs are causing or have caused trouble…I am also interested in SnapShots and if any have the potential to do harm on our platform. In this post i’ll be going through the Datastore Contention v2 Card…followed by Part 2 which will go through the Snapshots Gone Wild v2 Card.

Below is the new interface to the Datastore Contention v2 Card and you can see off the bat that there is a lot more going when compared to the v1 Cards

The initial Card View will show you Datastores across your environment that need Attention and those that are of interest. This is based on an algorithm that CloudPhysics have created that dictates acceptable levels of contention on VMs on datastores. You will get an overview of Throughput, IOPS and Latency metrics as well as total VMs and how many are potentially affected by storage contention.

While the actual metrics haven’t changed here from the v1 Card the way in which you can manipulate the data has been enhanced. For a period going back the lat 24 hours (It would be nice to go back further…something I’ve mentioned as a feature request) you can dynamically change the graph to display Bandwidth, Latency, IOPS, Outstanding IOs and choose to display Average, Read and/or Write Values.

As you click on the Active Red Zones in the Graph the list of Culprit VMs and Victim VMs changes to match the time period. [UPDATED] You now have a side by side view of Culprit VMs and Victim VMs giving quick and easy access to affected instances…By Clicking on the Blue View Details Button you can further drill down into the list and view VM specific storage metrics for that period as shown below.

You also now have the ability to export each graph in a variety of usable formats which is excellent for reporting purposes. In fact exporting has been enabled at all levels and you can export to CSV the entire Card View of data by clicking on the ALL Tab that lists all datastores in your environment. As a side note…the left hand search menu drop box sorting has been added to by a dynamic search bar which allows you to search for datastores…it even takes regular expression for those that are that way inclined!

Heading back to the top of the Card View you get an overall Aggregate picture of the currently selected datastores in the main presentation area…this dynamically adjusts based on what datastore(s) are in focus at the time.

Wrapping Part 1 up the new Datastore Contention card is brilliant…not only does it better give you access to potential problem datastores and VMs but it’s able to let you visualize and export data quickly and efficiently…The enhancements in the visual representation of data shows that the guys at CloudPhysics are looking at more dynamic style of data view which makes the overall experience and usability that much more enhanced.

http://blog.cloudphysics.com/blog/whos-minding-your-storage-zoo

http://blog.cloudphysics.com/blog/2014/4/7/noisy-neighbor-where-art-thou-performance-culprit-and-victim-analysis-using-cloudphysics-storage-analytics

 

Quick CloudPhysics Card: Windows Server 2012 and E1000 NICs

Over the past couple of months we have seen a number of support requests coming into our HelpDesk around network connection dropouts in Windows Server 2012 Guest systems. Common factor in all these requests where E1000 NICS.

https://communities.vmware.com/thread/433792

There are known issues with the E1000 NIC and Windows 2012 on ESXi 5.1 and above and the solution to the issue is to use a VMXNET3 NIC over the E1000…and before the VMware bashers start to take aim, lets not forget that the E1000 drivers are written by Intel and are provided as such. The VMXNET3 NICs are VMware written and also provide additional advantages such as better CPU efficiency and the ability to use Jumbo Frames.

Because vCloud Director’s default NIC Type is the E1000, if users are not switched on, then Guest VMs running 2012 will potentially have these issues. To get our support guys proactively advising clients of this issue I put together a quick @CloudPhysics Card using the Card Builder that will report on the Operating System Type, list the Guest NICs and their Type.

Which gives you a report as shown below:

The card should be available in the CloudPhysics Card Store shortly, but it’s easy enough to put together as per the builder blocks and filters above.

In a nutshell for Windows Server 2012…use the VMXNET3! There is really no excuse not to…when provisioning VMs in vCloud Director, be aware that the default NIC is the E1000…Maybe a feature request for the next version vCloud Prod Team? As an additional step, vCloud Users should look to template where possible and include the VMXNET3 NIC as part of the build.

For more info there is good guidance in this KB on selecting NICs based on Virtual Machine Operating System Type…

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1001805

First Look: CloudPhysics – Knowledge Base Advisor

I’ve been lucky enough to have been early access to a new Card from the guys at CloudPhyics which, at it’s core lets VMware admins quickly and precisely get an overview of issues in their environment linked back to VMware and other Vendor KB articles. CloudPhysics description is below:

So what is it? The new service is called Knowledge Base Advisor. This service matches the virtual datacenter profile of CloudPhysics users to knowledge base articles published by top virtual datacenter vendors including VMware, DELL, IBM and more. We join detailed knowledge of our customers with support content to create filtered, highly relevant and personalized support content for issues which may be present in their environments.
 
As of today the CloudPhysics Web App has had a major make over and now looks even slicker than before. The card its self is shown below, and gives you a brief overview of what’s currently happening in your environment as picked up by the CloudPhysics Probe VAs.
Clicking into the card you are presented with a list of Critical Issues relevant to your environment. It’s worth mentioning that just because an alert has been been detected it’s more of a proactive warning at this point. Obviously everybody’s environments are different and with that, these potential issues manifest themselves under varied conditions with differing trigger points.
Clicking in the alert box will expand to show you the affected hosts as it applies and by clicking on the KB Description are taken to a framed CloudPhysics page that loads up the specified KB Article. In my environment I had 200+ alerts, (16 of which where Critical, 86 High and 104 Medium), so being able to sort through and organise the alerts is crucial. The left hand menu lets you construct your own search queries base on a number of heading options.

The CloudPhysics team are great at reaching out for feedback and I’m proud to report that based on personal feedback the card has enhanced features and options that let you better deal with the initial large number of alerts that are shown. Once you have taken note of the issue you now have three options to deal with the alerts, Problem Fixed, Uninteresting and Not Relevant.

It’s great to see CloudPhysics evolve past numbers and metrics…that isn’t to say that that’s not where it’s greatest strength lies…but the more CloudPhysics can take your collated data and spit it back to you in the form of useful information, the more powerful and invaluable this platform becomes.

Once again, CloudPhysics continue to blow me away with what they have been able to achieve in a little over 12 months since announcing themselves at VMWorld 2012…I look forward to more card releases and how they will continue to assist myself and other operations teams in better understanding and managing their vSphere environments.

« Older Entries