In trying to tighten up some monitoring issues in the current environment I’m working in, I did what most people would think to do – I turned on some alarms in vCenter, specifically “Host connection and power state.” I kinda want to know when hosts just drop off the face of the planet, right? To get a feel for how this environment is running, I had that alert send me an email when it detected a host disconnect – and started getting 20-30 alerts a day. I then saw that all my hosts were fine. They weren’t dropping or disconnecting. Well, this is going to take some work, right? Read on…
I suck at scripting. I have this big vSphere PowerCLI poster sitting next to my desk that I’m only now learning to reference. But PowerShell is quickly becoming my go-to tool of choice to quickly find out how my vSphere infrastructure is doing. To that end, Quest software released an awesome get-your-feet-wet-with-PowerShell tool called PowerGUI, and then Kirk Munro released what’s called a PowerPack to allow super-quick management of VMware virtual infrastructure from within PowerGUI.
One of the best features of the latest PowerGUI / VMware Infrastructure Management release? An incredibly useful dropdown list of common vSphere gotchas to look for (click the thumbnail for a list of ‘em). As I come on board a new organization with a more diverse administration personnel model for vSphere, this list of queries has enabled me to very quickly identify some problems that could grow into larger issues – and to fix them almost as quickly.
Finding a terabyte of orphaned .vmdk files just made me a pretty valuable employee today…
Why didn’t I find this little trick sooner? On the heels of keeping your VI Client connection listing clean, you can also pre-populate your vSphere Client user name and (if you’re able to) password dialog boxes when launching the client. It’s wicked easy – get to the properties of your VI Client shortcut and modify the Target for the shortcut, adding -u username to the end without quotes. For example:
This will launch the client with the user name pre-populated with domainusername, which is super handy for me since I log into all of my vCenters using an admin account. Other switches:
-s hostname to specify a specific host or IP address
-p password to specific a password. Used with -u, ie: -u username -p password
…and with that tidbit of obviousness, a small dose of perspective from a wonderful friend. I’m lifting her post in it’s entirety – I hope you don’t mind, Kate.
I know I have devoted a lot of time to discussing the trials and struggles I have faced teaching in a low-income school. While so many of the moments I deal with at school each day are heart-breaking, I am also realizing that there are MANY moments that are heart-warming. My students face so many challenges on a day to day basis, and yet they spread so much joy to so many people. Despite their awful behavior, sometimes horrible choices and tough skins, they are just as sweet and kind as any of the other students I have taught. I think we have a love-hate relationship. I hate their crazy behavior, but I love the people that they are and the potential they hold.
The cold weather in Spokane has been an adjustment for me personally, but also another new experience in teaching at a Title 1 School. I was shocked to see children walking to school in 8 degree weather with SHORTS on and a hoodie. No gloves, no hats, broken zippers, over sized sweatshirts. It just about ripped my heart out. One boy, in particular, nearly brought me to tears. He speaks almost no English, and he was waiting in line for the bus with no coat, snow boots with no socks, and shorts on. He looked up at me with his eyes watering from the cold and bright red cheeks and gave me a huge smile. I had just spent the past few minutes complaining about how cold I was to another teacher while wearing my winter coat, gloves and scarf, and he was smiling despite the bitter cold. Yet again, I found myself confronted with a cold, harsh dose of perspective, and I will never forget it. He gave me another huge smile and wave as he got on the bus that warmed my whole heart.
This weekend I will be stocking my classroom with spare gloves, hats, and mittens to distribute to the children that come completely under-dressed for the weather. It’s not much, but it’s something. For tonight, I plan to stay cooped up in my warm house, which now means so much more to me than it ever did before. I hope that you and all of my students are somewhere warm, too.
I’m more of a fan of Google Chrome than anything else, having found that Firefox seems slow and bloated by comparison. But I might just need to keep that Firefox install running on the Macbook for one reason: Firesheep, via Eric Butler – my new Internets hero:
This is a widely known problem that has been talked about to death, yet very popular websites continue to fail at protecting their users. The only effective fix for this problem is full end-to-end encryption, known on the web as HTTPS or SSL. Facebook is constantly rolling out new “privacy” features in an endless attempt to quell the screams of unhappy users, but what’s the point when someone can just take over an account entirely? Twitter forced all third party developers to use OAuth then immediately released (and promoted) a new version of their insecure website. When it comes to user privacy, SSL is the elephant in the room.
Today at Toorcon 12 I announced the release of Firesheep, a Firefox extension designed to demonstrate just how serious this problem is.
While I don’t come down too heavily on one side or the other of the debate over who is responsible for end user privacy, I still come down heavily on end user privacy itself. Enter the Firesheep extension for Firefox (and hopefully for Chrome, soon?) and my burning desire to try this out at some coffee shop the next time I’m out. I have a feeling, though, that more end users would blame the messenger (Eric for creating it, me for trying it?) than be able to accurately pinpoint the problems with the infrastructure leading to these sorts of vulnerabilities…
A job or two ago, I worked in an organization that had a definite need for monitoring but a budget that couldn’t keep up with the expansion of the virtual infrastructure we were building out. After searching around for various options, I set up a Zenoss Core server, loaded up some needed Zenpacks, and we were off and running with a better idea of what our infrastructure was doing (or wasn’t doing when it stopped doing it). Despite a little bit of a learning curve having come from a WhatsUp shop, Zenoss fit the bill pretty much perfectly.
To help use this awesome tool, we are excited to annouce that we have created a Free Zenpack to perform ESX/ESXi server monitoring with esxtop. The Zenpack does the following:
Automatically captures and graphs performance metrics from ESX/ESXi servers
Allows the setting of thresholds and exception alerting based on performance metrics and events as they occur
Keeps historical data for in-depth root cause analysis
With Zenoss’s VMware ESX Server Zenpack, you get visibility into your ESX/ESXi server and you are also able to proactively manage it. A perfect example of this is over-provisioning of virtual CPUs; a condition that without esxtop is hard to diagnosis.
I’ve often found that monitoring becomes almost an afterthought during rapid deployment of almost any infrastructure. If you’ve found yourself in that spot, or just simply want to get a different perspective on what sort of trending and alerting you can get – for free, even! – I would highly suggest giving Zenoss a shot.
A curious thing happened on the way to finishing this post by Duncan Epping: I started the think that VMware’s High-Availability solution will actually come to be something that we can rely on. I could fill a lot of column space with the issues we’ve had in trying to protect our VMs using HA, and how – at times – we’ve simply turned the system off. In the early days, HA agent re-installation in order to even get hosts set up were common. Currently, storage failures for VMs do not trigger HA events as reliably as we’d like. Almost all of the points that Duncan mentions below speak to some sort of failure that we’ve encountered with HA:
…I wanted to share with all of you and that is around some of the HA futures.
All New Architecture, a single lightweight HA agent process
Eliminate concept of “Primaries”
Storage heartbeating as backup communication channel
Automatic resolution of network partitions
VMs still protected during partitions, no “fighting” for VM control
Greater scalability, extensible
Ability to deal with any number of simultaneous host failures
New lightweight communication model
All state required to recover from any failure is persisted
Improved isolation actions (VMs left running and restarted as needed via storage subsystem monitoring)
No dependencies on DNS
High Availability is one of the cornerstones of arguing for virtualization. It’s supposed to be the answer to the inevitable question: “But what happens if a one of your hosts goes down?” VMware’s Fault Tolerance is a step in that direction as well, but limited support (a single CPU? really?), increased network traffic, and doubling your storage requirements currently make FT a poor choice for the majority of our VMs. Like I said, I’ll be watching these developments closely…
This post from Eric Sloof is near and dear to my heart: I’ve probably migrated around 300 servers to VM in my current position, and learning how to quickly clean up those VMs once they’re converted can save you a tremendous amount of time (time usually on the weekend or after hours!). I’ve always used a set of scripts that I found via the VMware forums that Eric Sloof has now clarified to have been developed by Phi Phi Wang at IBM. But for you GUI types, there’s a new open source tool out there called Ghostbuster that will quickly list out ghosted devices in Windows and allow you to select them for removal easily and quickly. I’ve listed out some tips for physical-to-virtual migrations before, and this tool would easily replace the very manual, point-click-delete-repeat process of removing these by hand in the Windows Device Manager. Could have used this about a year ago, definitely…
Eric Sloof just posted up a fantastic resource for those in need of vmdk and general VM help: Ulli Hankeln’s Sickbay.
Welcome – looks like something is wrong with your virtual machine.
Before we go into details …
relax …
drink a coffee and follow the VM-sickbay-rules
1. DON’T PANIC
2. do not try to start the VM again
3. do not mount disks from other VMs
4. do not use vmware-mount or vdiskmanager
5. do not use vdk in read/write mode
6. make a copy of all vmware-log files
7. relax
Having found myself in a spot with a sick VM in the past, I’ve definitely done steps 2, 3, 4, and maybe even 5 and been the worse off for it. In addition, and the original point of Eric’s post, Ulli has posted up the most comprehensive collection of vmdk troubleshooting info in one spot I’ve seen yet. Definitely bookmarked…
A curious thing happened on the way to finishing 
