Reset unresponsive VM with Powershell PowerCLI
I had a recent use case to be able to check the reachability of a certain VM which had shown itself to be problematic. If the VM wasn’t responsive or had crashed, restarting the VM with a graceful restart via VMware tools wouldn’t be possible. Using Powershell and specifically PowerCLI, we should be able to check the connectivity to the VM and then if it is not responsive, we can run a reset on the VM. Obviously, the best things would be to get to the bottom of the issue with the VM bluescreening or freezing, however, this is a stop gap solution until finding the underlying cause. Sometimes, you have to work around situations with a solution that works for a specific use case. Let’s look at how to reset unresponsive VM with Powershell PowerCLI and how we can use a short little script to check and then reset a VM.
Reset unresponsive VM with Powershell PowerCLI
Let’s take a look at the script and then we can break it apart
#variables
$vcenter = 'vcenter.domain.local'
$cred = get-credential
$Log = 'c:\log\testfail.log'
$Time = get-date
$servers = 'testserver'
#connect vCenter
connect-viserver -server $vcenter -credential $cred
#Health Check
Foreach($server in $servers)
{
if(Test-Connection -Cn $server -BufferSize 16 -Count 1 -ea 0 -quiet) {
“Testing connection to $server is successful”
}
if (!(Test-Connection -Cn $server -BufferSize 16 -Count 1 -ea 0 -quiet)) {
"Connection Failed `t`t" + $Time.DateTime + "`t`t" + ($endTime – $Time).TotalSeconds + " seconds" | Out-File $Log -Append; send-mailmessage -from "<[email protected]>" -to "Some One <[email protected]>" -subject "$server not responding" -body "Test Server not responding, waiting 10 seconds and will try again" -smtpserver 1.2.3.4;
"Waiting 15 seconds"
sleep 15
if (!(Test-Connection -Cn $server -BufferSize 16 -Count 4 -ea 0 -quiet)) {
"Connection Failed `t`t" + $Time.DateTime + "`t`t" + ($endTime – $Time).TotalSeconds + " seconds" | Out-File $Log -Append ; (Get-VM -Name $server).ExtensionData.ResetVM();
send-mailmessage -from "<[email protected]>" -to "Some One <[email protected]>" -subject "Test Server crash" -body "Test Server must have crashed" -smtpserver 1.2.3.4;
sleep 30
}
ELSE {“Successful `t`t" + $Time.DateTime + "`t`t" + ($endTime – $Time).TotalSeconds + " seconds" | Out-File $Log -Append; send-mailmessage -from "<[email protected]>" -to "Some One <[email protected]>" -subject "$server is responding" -body "Test Server is responding again and was not reset" -smtpserver 1.2.3.4;}
}
}
We define the variables here in this part of the code, including our vcenter server, getting credentials, log location, and getting the current time.
#Variables $vcenter = 'vcenter.domain.local' $cred = get-credential $Log = 'c:\logs\testfail.log' $Time = get-date $servers = 'testserver'
Using the variables above, we can make the connection to our vCenter server:
#vCenter Connection connect-viserver -server $vcenter -credential $cred
Finally, we can actually make the check on the server. The “for” loop loops through a list of servers, or a single server as we have in this example. It uses the test-connection commandlet to ping the server once and verify connectivity. If the connection fails, it logs the failure time and sends out an email noting the failure. It then waits for “15 seconds” and tests to see whether or not it was just a blip in connectivity or if the connectivity outage is real. If it fails again, it logs the error/time, issues a reset on the VM in question, and also sends an email to note the failure and action.
If connectivity has been restored, the final ELSE statement will log the Successful connection and send an email back out noting the server is now reachable.
#Health Check
Foreach($server in $servers)
{
if(Test-Connection -Cn $server -BufferSize 16 -Count 1 -ea 0 -quiet) {
“Testing connection to $server is successful”
}
if (!(Test-Connection -Cn $server -BufferSize 16 -Count 1 -ea 0 -quiet)) {
"Connection Failed `t`t" + $Time.DateTime + "`t`t" + ($endTime – $Time).TotalSeconds + " seconds" | Out-File $Log -Append; send-mailmessage -from "<[email protected]>" -to "Some One <[email protected]>" -subject "$server not responding" -body "Test Server not responding, waiting 10 seconds and will try again" -smtpserver 1.2.3.4;
"Waiting 10 seconds"
sleep 10
if (!(Test-Connection -Cn $server -BufferSize 16 -Count 4 -ea 0 -quiet)) {
"Connection Failed `t`t" + $Time.DateTime + "`t`t" + ($endTime – $Time).TotalSeconds + " seconds" | Out-File $Log -Append ; (Get-VM -Name $server).ExtensionData.ResetVM();
send-mailmessage -from "<[email protected]>" -to "Some One <[email protected]>" -subject "Test Server crash" -body "Test Server must have crashed" -smtpserver 1.2.3.4;
sleep 30
}
ELSE {“Successful `t`t" + $Time.DateTime + "`t`t" + ($endTime – $Time).TotalSeconds + " seconds" | Out-File $Log -Append; send-mailmessage -from "<[email protected]>" -to "Some One <[email protected]>" -subject "$server is responding" -body "Test Server is responding again and was not reset" -smtpserver 1.2.3.4;}
}
}
Download
Download the full script on Github.
Thoughts
The script is a work in progress and is a melting pot of other scripts and things I have used recently. Let me know your thoughts and please feel free to fork the code and issue a pull request as well. Hopefully the above script will be helpful for anyone wanting to reset unresponsive VM with Powershell PowerCLI.
