A few weeks ago, for TSQL Tuesday, I wrote about how I used Powershell to manage failing over clusters. At work, we have now implemented that script in our patching process and it has reduced our patching time significantly. However, the process was still missing something. A validation step was needed to ensure that the script was doing what was expected. Without the automated validation, the on-call person still had to VPN into the environment and manually check the status of each cluster.
After discussing this with the team, we determined that all we needed was an email confirmation post process. This email would need to be sent out after the clusters are failed over. This would allow the on-call person to easily identify via email if something was incorrect.
The email needed to contain a couple of things to make it useful.
- The expected owner of each cluster
- Who currently owns the active node
We are using this script to manage several clustered instances.
The Parts
The first step is to setup the metadata about the email itself.
import-module failoverclusters $FromAddress = "from@somedomain.com" $ToAddress = "me@somedomain.com" $MessageSubject = "Before Patching - Cluster Node Owners" $SendingServer = "smtp.somedomain.com"
Next we will create a hash table. We will use this to neatly keep track of what we expect for each respective clustered instance. There might be better ways to do this, but I thought that this was sufficient for what we needed.
$expected = @{
"Cluster1" = "Owner1";
"Cluster2" = "Owner2";
"Cluster3" = "Owner3";
"Cluster4" = "Owner4"
}
Now we will define a style sheet for the HTML email message. This style sheet will be applied to the two tables that will be embedded into the message.
$style = "<style>BODY{font-family: Arial; font-size: 10pt;}"
$style = $style + "TABLE{border: 1px solid black; border-collapse: collapse;}"
$style = $style + "TH{border: 1px solid black; background: #dddddd; padding: 5px; }"
$style = $style + "TD{border: 1px solid black; padding: 5px; }"
$style = $style + "</style>"
Now we need to get information about each cluster instance. There are probably several ways to accomplish this, but I choose to use a powershell array.
#build the current status array $list = @((get-clustergroup -cluster Cluster1) , (get-clustergroup -cluster Cluster2) , (get-clustergroup -cluster Cluster3) , (get-clustergroup -cluster Cluster4))
We can now start to build out the body of the message. Let’s being with the expected outcome. One thing to note is that a hash table is a single PowerShell object. In order to do some manipulation, such as applying the style sheet, I had to use the getenumerator() method. This “unwraps” each object.
#this will get us the expected outcome into a table format $body1 = $expected.getenumerator() | sort-object -property name | name, value | convertto-html -head $style | out-string
The second part of the body for the message will be the current status of each cluster. Similar to the hash table, we have to get each object out of the array that we created. We do this by piping the list into a ForEach-Object cmdlet.
#this will get us the current status of each cluster into a table format
$body2 = $list | % {$_} | select Cluster,name, ownernode,state | convertto-html -head $style | out-string
Now that we have the two body elements, we can combine them into a single body element. Since the message will be in HTML format, I added wording around each table so that it is easier to read.
#bring it all together into a single body to email $body = "Expected: " + $body1 + " Current Status: " + $body2
Finally, we can send the message. We will be using the send-MailMessage cmdlet to accomplish this.
#send the message
send-MailMessage -SmtpServer $SendingServer -To $ToAddress -From $FromAddress -Subject $MessageSubject -Body $body -BodyAsHtml
The Whole Pie
import-module failoverclusters
$FromAddress = "from@somedomain.com"
$ToAddress = "me@somedomain.com"
$MessageSubject = "Before Patching - Cluster Node Owners"
$SendingServer = "smtp.somedomain.com"
$expected = @{
"Cluster1" = "Owner1";
"Cluster2" = "Owner2";
"Cluster3" = "Owner3";
"Cluster4" = "Owner4"
}
$style = "<style>BODY{font-family: Arial; font-size: 10pt;}"
$style = $style + "TABLE{border: 1px solid black; border-collapse: collapse;}"
$style = $style + "TH{border: 1px solid black; background: #dddddd; padding: 5px; }"
$style = $style + "TD{border: 1px solid black; padding: 5px; }"
$style = $style + "</style>"
" #build the current status array
$list = @((get-clustergroup -cluster Cluster1)
, (get-clustergroup -cluster Cluster2)
, (get-clustergroup -cluster Cluster3)
, (get-clustergroup -cluster Cluster4))
#this will get us the expected outcome into a table format
$body1 = $expected.getenumerator() | sort-object -property name | name, value | convertto-html -head $style | out-string
#this will get us the current status of each cluster into a table format
$body2 = $list | % {$_} | select Cluster,name, ownernode,state | convertto-html -head $style | out-string
#bring it all together into a single body to email
$body = "Expected: " + $body1 + " Current Status: " + $body2
#send the message
send-MailMessage -SmtpServer $SendingServer -To $ToAddress -From $FromAddress -Subject $MessageSubject -Body $body -BodyAsHtml
Summary
The result is a nicely formatted HTML message, as shown below.
You can see that the email is easy to understand. This will allow us to quickly validate that things are where they should be. Once the patching is completed, we’ll use this same email validation to confirm that things are the way that they should be.
