My SQL Server Database is Corrupt - Now What?!

Question

My SQL Server Database is Corrupt - Now What?!

Paul Randal

One Orange Chip

Points: 29544
More actions
January 28, 2013 at 10:00 pm

#288467

Comments posted to this topic are about the item My SQL Server Database is Corrupt - Now What?!
Paul Randal
CEO, SQLskills.com: Check out SQLskills online training!
Blog:www.SQLskills.com/blogs/paul Twitter: @PaulRandal
SQL MVP, Microsoft RD, Contributing Editor of TechNet Magazine
Author of DBCC CHECKDB/repair (and other Storage Engine) code of SQL Server 2005
February 4, 2013 at 1:13 am

This was removed by the editor as SPAM

Viewing 13 posts - 1 through 12 (of 12 total)

You must be logged in to reply to this topic. Login to reply

Ness SSCarpal Tunnel Points: 4282 More actions · Answer 1

Hi,

Would you run the DBCC CHECKDB with the REPAIR_REBUILD option (after recovery from a backup/HA solution)? I seem to remember that this was the preferred option the MS 2005 / 2008 MCITP learning. I know that real life can be far detached from the learning, or are you attempting to fix any Ix's individually?

SQL DBA
Every day is a school day, and don't trust anyone who tells you any different.
http://sqlblogness.blogspot.co.uk

manik_anu SSCrazy Points: 2377 More actions · Answer 2

hi all,

i have deleted the tables in a database n particular sequence. after that the database name has changes as <databasename(suspect)>. What did it means? the database corrupted??? if it correpts how can i recover??? DBCC CHECKDB will useful to this.....?????

Manik
You cannot get to the top by sitting on your bottom.

Ness SSCarpal Tunnel Points: 4282 More actions · Answer 3

Try recovering from a backup first and then if that does not work try the following : http://www.sqlskills.com/blogs/paul/creating-detaching-re-attaching-and-fixing-a-suspect-database/

SQL DBA
Every day is a school day, and don't trust anyone who tells you any different.
http://sqlblogness.blogspot.co.uk

SALIM ALI SSC Eights! Points: 859 More actions · Answer 4

Failover, Restore and Repair

I think one has been forgotten, and it's one that's not to be taken likely because you have to know your data. This method is Reconstruction.

Sometimes, in the event of only having an old backup, it is possible to reconstruct the data in certain circumstances. I had a table corruption s few years back where the only backup I could use for recovery was 3 weeks old. {edit} This also had the corrption so I had to go back to an earlier tape copy {end edit} You can usually guarantee that if something is going to go wrong, it will go wrong when you're on leave and no-one will pick it up, leaving it for you to deal with upon return.

The corruption was on the clustered index. I was unable to drop it as I kept getting errors. The clustered index ordered the data sequentially, so all the historic stuff was at the head of the table with the new data at the tail end. The corruption was somewhere in the middle (dated prior to the last good backup, good!)..

DBCC CHECKDB told me where the corruption was.

DBCC IND ('trapezedata', {tablename},-1)

provided me with a list of the PageIds for the corrupted table

I used these to fuel the next DBCC command...

DBCC TRACEon(3604)

DBCC PAGE ('{datanasename}',1,{PageID},3)

GO

provided me with the data on that page.

This allowed me to output all of the tail end data after the corrupted pages into a text file. The data output was in the format similar to :

Slot 54 Offset 0x10d4

---------------------

Record Type = PRIMARY_RECORD

Record Attributes = NULL_BITMAP VARIABLE_COLUMNS

484730D4: 003e0030 000a5d59 00000001 00000000 0.>.Y]..........

484730E4: 00000000 00000000 00000000 00000000 ................

484730F4: 0000002b 00000000 00000000 00000000 +...............

48473104: 00000000 00000000 00000000 00120001 ................

48473114: 02000000 4e004c00 494c4300 5453 .....L.N.CLIST

LegId = 679257

PassType = CLI

SpaceType = ST

NumSpacesPu = 1

NumSpacesDo = 0

FareTypeId = 0

FareAmount = 0

FundingSource1Amount = 0

FundingSource2Amount = 0

FundingSource3Amount = 0

FundingSourceId1 = 43

FundingSourceId2 = 0

FundingSourceId3 = 0

FundingSource4Amount = 0

FundingSource5Amount = 0

FundingSourceId4 = 0

FundingSourceId5 = 0

FareCalcType = 1

I ran the DBCC PAGE for the newer pages/data that did not appear on the restored database , which I had restored to a copy table. I loaded the resulting (and large) text file back into a temporary table, cleaned up the data to leave the column names and values and then transposed these back into columns with a little bit of TSQL scripting.

I was then able to insert the latest data back into the restored table, drop the corrupted one and rename the restored one to replace it.

No data loss, but there was some loss of functionality to the users system during the fix.

It's a horrible position to be in and not an elegant or easy fix, but it can be done this way if you run out of ideas.

Paul Randal One Orange Chip Points: 29544 More actions · Answer 5

Ness (1/29/2013)
Hi,
Would you run the DBCC CHECKDB with the REPAIR_REBUILD option (after recovery from a backup/HA solution)? I seem to remember that this was the preferred option the MS 2005 / 2008 MCITP learning. I know that real life can be far detached from the learning, or are you attempting to fix any Ix's individually?

I'd only run REPAIR_REBUILD if they were just nonclustered index errors that I couldn't fix by doing a manual rebuild or a drop/create inside a transaction. The advice in the MCITP learning is incorrect. Never put the database offline (i.e. from going into SINGLE_USER) if you can avoid it.

Paul Randal
CEO, SQLskills.com: Check out SQLskills online training!
Blog:www.SQLskills.com/blogs/paul Twitter: @PaulRandal
SQL MVP, Microsoft RD, Contributing Editor of TechNet Magazine
Author of DBCC CHECKDB/repair (and other Storage Engine) code of SQL Server 2005

Paul Randal One Orange Chip Points: 29544 More actions · Answer 6

SALIM ALI (1/29/2013)
Failover, Restore and Repair
I think one has been forgotten, and it's one that's not to be taken likely because you have to know your data. This method is Reconstruction.

Nope - not forgotten, just not useful to the vast majority of DBAs out there so not worth mentioning in a short, 1-200-level article such as this one.

Paul Randal
CEO, SQLskills.com: Check out SQLskills online training!
Blog:www.SQLskills.com/blogs/paul Twitter: @PaulRandal
SQL MVP, Microsoft RD, Contributing Editor of TechNet Magazine
Author of DBCC CHECKDB/repair (and other Storage Engine) code of SQL Server 2005

@SQLFRNDZ SSCrazy Eights Points: 9095 More actions · Answer 7

Hi Paul,

I have similar issue yesterday occurred after upgrading to the new version sql2008r2 from 2005 version. We haven't changed the compitible mode after the upgarde.

I have followed all steps in your blogpost

Initially, I have tried regular recover option like below

ALTER DATABASE [MYDB] SET EMERGENCY;

GO

ALTER DATABASE [MYDB] SET SINGLE_USER;

GO

DBCC CHECKDB (N'MYDB', REPAIR_ALLOW_DATA_LOSS) WITH NO_INFOMSGS, ALL_ERRORMSGS;

GO

But received Error:

The database 'MYDB ' is in 611. The database needs to be updated to the latest version.

Then I followed your blog post below:

http://www.sqlskills.com/blogs/paul/creating-detaching-re-attaching-and-fixing-a-suspect-database/

As it is reporting database, we are not concerned about the data but need the reporting store procedure from the corrupted dB.

Issue: DBCC checkdb cannot be run , also I checked other post of you on fatal errors where dbcc checkdb can not be run, In this case Is there anyway we could get the stored procedures out of the corrupted database.

[font="Tahoma"]
--SQLFRNDZ[/url]
[/font]

Paul Randal One Orange Chip Points: 29544 More actions · Answer 8

Nope - something stopped the upgrade from completing. You'll need to go back to the copy you still have on 2005 and figure out what the corruption is there.

Paul Randal
CEO, SQLskills.com: Check out SQLskills online training!
Blog:www.SQLskills.com/blogs/paul Twitter: @PaulRandal
SQL MVP, Microsoft RD, Contributing Editor of TechNet Magazine
Author of DBCC CHECKDB/repair (and other Storage Engine) code of SQL Server 2005

Jeff Moden SSC Guru Points: 1004323 More actions · Answer 9

I'm one of "those" people that kept saying that there were a million articles on how to determine if corruption had occurred and very few on what to do about it. Thanks for rising to the occasion, Paul, and well done!

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

Stuart Davies SSCoach Points: 19505 More actions · Answer 10

Good article Paul - thanks.

Things will and do go wrong - no one should try to kid themselves otherwise.

The main thing we can do is our best damage limitation exercise.

-------------------------------Posting Data Etiquette - Jeff Moden [/url]Smart way to ask a question
There are naive questions, tedious questions, ill-phrased questions, questions put after inadequate self-criticism. But every question is a cry to understand (the world). There is no such thing as a dumb question. ― Carl Sagan
I would never join a club that would allow me as a member - Groucho Marx

quackhandle1975 SSChampion Points: 11055 More actions · Answer 11

I would echo all the other comments, it's not "if" it is "when". I always tell my clients, no one ever thought they would slam jumbo jets into skyscrapers but it happened.

My current client have no SQL DR whatsoever so about to create their first DR plan.

Test & prepare, great article Paul.

qh

[font="Tahoma"]Who looks outside, dreams; who looks inside, awakes. – Carl Jung.[/font]