June 4, 2012 at 7:19 am
If you stop SQL Server, can you copy the files? Alternately, did that snapshot backup work, can you restore that elsewhere and check that it's OK?
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
June 4, 2012 at 7:59 am
All of our testing has been on a snapshot backup. We're not trying anything in production.
June 4, 2012 at 8:10 am
Have you run the DBCC against the production database then? Just trying to make sure that I understand the last post correctly.
David
@SQLTentmaker“He is no fool who gives what he cannot keep to gain that which he cannot lose” - Jim Elliot
June 4, 2012 at 8:11 am
So the bad block came across with the snapshot backup? Ow... Try copying the files with SQL stopped, see if that will get rid of it,
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
June 4, 2012 at 9:48 am
We can try that, but we'll have to find a window to bring down production.
June 4, 2012 at 9:52 am
Dumb question, but have you engaged either or both of Netapp or Microsoft?
June 4, 2012 at 10:10 am
Mick Opalak (6/4/2012)
We can try that, but we'll have to find a window to bring down production.
For what? CheckDB is an online operation. Will add lots of load to the DB, but doesn't require that the DB is inaccessible.
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
June 4, 2012 at 10:58 am
You can keep trying to do this without impacting production, but you run the risk, perhaps a serious risk, of losing lots of your data. You need to spend the resources, and potentially downtime, to fix this.
As Gail said, kick this off in production. You need to know what you have there.
I'd also start restoring older backups elsewhere and running DBCC to make sure you have a clean backup to get back to if something happens. This could die anytime, and then you'd be seriously down.
June 5, 2012 at 7:24 am
We did run CHECKDB on production and it did come back clean. On the Snap copy we made CHECKDSK was run and it found the bad block and marked as so. Now the SQL backup finishes successfully on the copy. So that's good news, but the boss still wants to find a way to correct the bad block and not just mark it bad and ignore it. We have Netapp involved.
June 5, 2012 at 7:34 am
And checkDB completes successfully on the copy once the block has been marked bad?
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
June 8, 2012 at 2:00 pm
So it turns out that the solution is going to be clearing out the bad block at the storage level via a Netapp tool. We've tested the process on a Snapshot copy of the database and it works without corrupting the database.
June 8, 2012 at 2:08 pm
CheckDB comes back clean after doing so?
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
June 8, 2012 at 2:22 pm
GilaMonster (6/8/2012)
CheckDB comes back clean after doing so?
Yep.
June 8, 2012 at 3:43 pm
Then I would say take at least one more snap backup and proceed with caution (and frequent CheckDBs)
Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability
Viewing 14 posts - 16 through 28 (of 28 total)
You must be logged in to reply to this topic. Login to reply