January 25, 2016 at 2:43 am
Hello,
I've battling with a bit of an odd problem and I'm hoping someone can help. I have a cloning process running via PowerShell that periodically gets stuck for a large number of minutes taking a database offline prior to the cloning of the files underneath and I cannot find out why it's doing it.
The basic process for the offline steps is as follows
What's particularly odd is that the checks show that there are not any sessions on the database to explain it getting stuck and the behaviour is identical whether you use the SMO SetOffline() function or the TSQL SET OFFLINE WITH ROLLBACK IMMEDIATE. It also doesn't happen every time
PowerShell code snippet for the offline function (With lots of timespans to report where the delays are coming from)
function Set-DatabaseOffline
{
param
(
$instance,
$database
)
try
{
$stepStart = Get-Date
$machineName = (Get-HostFromAlias $instance)
$stepDuration = (New-TimeSpan -Start $stepStart -End (Get-Date))
Write-Host "Instance alias gathered in $($stepDuration)" -ForegroundColor DarkMagenta -BackgroundColor White
Write-Host "Creating SQLPS objects" -ForegroundColor Yellow
$stepStart = Get-Date
$serverObject = Get-Item "SQLSERVER:\SQL\$($machineName)\DEFAULT"
$dbObject = Get-Item "SQLSERVER:\SQL\$($machineName)\DEFAULT\Databases\$($database)"
$masterObject = Get-Item "SQLSERVER:\SQL\$($machineName)\DEFAULT\Databases\master"
$stepDuration = (New-TimeSpan -Start $stepStart -End (Get-Date))
Write-Host "SQLPS Objects created in $($stepDuration)" -ForegroundColor DarkMagenta -BackgroundColor White
Write-Host "Closing all open connections to $($database) on $($instance) and setting it offline" -ForegroundColor Yellow
$stepStart = Get-Date
$serverObject.KillAllProcesses($database)
$stepDuration = (New-TimeSpan -Start $stepStart -End (Get-Date))
Write-Host "All processes killed in $($stepDuration)" -ForegroundColor DarkMagenta -BackgroundColor White
# Validation - Check if we have any open connections to this DB still
[Int]$connectionCount = $serverObject.GetActiveDBConnectionCount($database)
Write-Host "There are now $($connectionCount) connections to $($database)" -ForegroundColor Yellow
if ($connectionCount -gt 0)
{
# Report the list of current connections for that database
$serverObject.EnumProcesses() | Where Database -ieq $database | Select Spid, Login, Host, Status, Program | FT -AutoSize
}
$stepStart = Get-Date
$masterObject.ExecuteNonQuery("IF (SELECT user_access_desc FROM sys.databases WHERE name = '$($database)') != 'RESTRICTED_USER' BEGIN ALTER DATABASE [$($database)] SET RESTRICTED_USER WITH ROLLBACK IMMEDIATE END;")
$stepDuration = (New-TimeSpan -Start $stepStart -End (Get-Date))
Write-Host "Database swapped to restricted access in $($stepDuration)" -ForegroundColor DarkMagenta -BackgroundColor White
# Validation - Check if we have any open connections to this DB still
[Int]$connectionCount = $serverObject.GetActiveDBConnectionCount($database)
Write-Host "There are now $($connectionCount) connections to $($database)" -ForegroundColor Yellow
if ($connectionCount -gt 0)
{
# Report the list of current connections for that database
$serverObject.EnumProcesses() | Where Database -ieq $database | Select Spid, Login, Host, Status, Program | FT -AutoSize
}
$stepStart = Get-Date
$dbObject.SetOffline()
$stepDuration = (New-TimeSpan -Start $stepStart -End (Get-Date))
Write-Host "Database taken offline in $($stepDuration)" -ForegroundColor DarkMagenta -BackgroundColor White
Write-Host "$($database) has successfully been taken offline" -ForegroundColor Yellow
}
catch
{
throw
}
}
Output from a delayed run
Creating SQLPS objects
SQLPS Objects created in 00:00:01.9461207
Closing all open connections to somedb on SOMESERVER and setting it offline
All processes killed in 00:00:00.2320159
There are now 0 connections to somedb
Database swapped to restricted access in 00:00:00.4600213
There are now 0 connections to somedb
Database taken offline in 00:07:59.3086630
somedb has successfully been taken offline
Database offline step completed in 00:08:02.4678561
Thanks very much in advance for any help as this is driving me mad!
January 25, 2016 at 2:54 am
Does it take that long every single time? Have you tried using sp_whoisactive or similar to find out what is going on during the SET OFFLINE operation?
John
January 25, 2016 at 6:07 am
Yeah, what can you see from the SQL Server side? Is the process blocked or waiting on a resource or what? That's what I'd want to know.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
January 26, 2016 at 8:44 am
Hi both,
Firstly, thanks for taking the time to reply.
It's definitely not every time and seems very sporadic as to when it does happen. I've checked via the recorded playback in Spotlight and I honestly cannot see anything that would cause this happening at the time, although it's without drawbacks as Spotlight is on a one minute interval collection. The blocked process monitors never fire during the delayed clone operation though, so I'm not sure that it's getting blocked per say.
I'll add an extra line to the PowerShell to spit out the sp_whoisactive output just before the offline step gets processed and see if I can find anything else that might explain it and let you know what I uncover.
Thanks again
Viewing 4 posts - 1 through 3 (of 3 total)
You must be logged in to reply to this topic. Login to reply