Backup and restore solution for 800TB of data

Question

Backup and restore solution for 800TB of data

Viewing 15 posts - 16 through 30 (of 59 total)

You must be logged in to reply to this topic. Login to reply

Dugi SSCoach Points: 17998 More actions · Answer 1

If the situation is true, it would be very interesting about DB architecture info ( data files & log files architecture ) :w00t::hehe::w00t:

============================================================
SELECT YOUR PROBLEM FROM SSC.com WHERE PROBLEM DESCRIPTION =
http://www.sqlservercentral.com/articles/Best+Practices/61537/[/url]

Gail Shaw SSC Guru Points: 1004504 More actions · Answer 2

GregoryF (4/29/2010)
Are you really sure it's 800TB and not 800GB?
SQL 2008 only support 512TB Databases
http://msdn.microsoft.com/en-us/library/ms143432.aspx
So does SQL 2005
http://msdn.microsoft.com/en-us/library/ms143432(SQL.90).aspx

No. Not 512 TB. 524,272 TB. 524 PB (petabytes)

Gail Shaw
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)
SQL In The Wild: Discussions on DB performance with occasional diversions into recoverability

We walk in the dark places no others will enter
We stand on the bridge and no one may pass

Dugi SSCoach Points: 17998 More actions · Answer 3

GregoryF (4/29/2010)
Are you really sure it's 800TB and not 800GB?
SQL 2008 only support 512TB Databases
http://msdn.microsoft.com/en-us/library/ms143432.aspx
So does SQL 2005
http://msdn.microsoft.com/en-us/library/ms143432(SQL.90).aspx

No SQL Server supports over 512 TB, take a look more closely, the MAX spec. is 524272 TB (that's means in Peta Bytes)! So if the situation is true, it can be ! :w00t:

============================================================
SELECT YOUR PROBLEM FROM SSC.com WHERE PROBLEM DESCRIPTION =
http://www.sqlservercentral.com/articles/Best+Practices/61537/[/url]

GregoryF SSCertifiable Points: 6296 More actions · Answer 4

Dugi (4/29/2010)
GregoryF (4/29/2010)
Are you really sure it's 800TB and not 800GB?
SQL 2008 only support 512TB Databases
http://msdn.microsoft.com/en-us/library/ms143432.aspx
So does SQL 2005
http://msdn.microsoft.com/en-us/library/ms143432(SQL.90).aspx
No SQL Server supports over 512 TB, take a look more closely, the MAX spec. is 524272 TB (that's means in Peta Bytes)! So if the situation is true, it can be ! :w00t:

Thanks, I did not notice the factor of 100, I'm not used to files of such size...

But with a 16TB limit on datafiles, you would still need 50 datafiles. With a limit of 2TB on logs, 50 logs. That's an unrealistic number of drives with corresponding HBA cards. And both of those numbers assume that all files are at 100%.

The server that could process all of this, and the SAN that could store it, and the networking that can transfer it, would out of reach for all but the richest of companies.

If it's true, I'd love to play with it

/* ----------------------------- */
Tochter aus Elysium, Wir betreten feuertrunken, Himmlische, dein Heiligtum!

Bex SSCrazy Points: 2437 More actions · Answer 5

Steve Jones - Editor (4/29/2010)
Also, I'm curious how you are backing up the log every 10 sec. Are you using some external program. AFAIK, jobs to schedule can be done every minute, not more granular than that.

2008 can indeed go down to seconds, but I can't say I had noticed it before.

Michael Valentine Jones SSC Guru Points: 64818 More actions · Answer 6

hor_netuk (4/29/2010)
Steve Jones - Editor (4/29/2010)
Also, I'm curious how you are backing up the log every 10 sec. Are you using some external program. AFAIK, jobs to schedule can be done every minute, not more granular than that.
2008 can indeed go down to seconds, but I can't say I had noticed it before.

If you really need, you can schedule a job to run multiple times per minute even in SQL 2000.

To schedule a job to run every 15 seconds, create four schedules that run once per minute, with schedule 1 starting at 12:00:00 am, schedule 2 starting at 12:00:15 am, schedule 3 starting at 12:00:30 am, and schedule 4 starting at 12:00:45 am.

I have a script I wrote that can schedule a job to run every arbitrary number of seconds by creating the necessary schedules.

WayneS SSC Guru Points: 95460 More actions · Answer 7

D.Oc (4/29/2010)
My bet is on GB's not TB.

Especially singe "G" is right under "T" on the keyboard...

Edit: but then again, he did write out "Terabytes" in the first post...

Wayne
Microsoft Certified Master: SQL Server 2008
Author - SQL Server T-SQL Recipes

If you can't explain to another person how the code that you're copying from the internet works, then DON'T USE IT on a production system! After all, you will be the one supporting it!

Links:
For better assistance in answering your questions
Performance Problems
Common date/time routines
Understanding and Using APPLY Part 1 & Part 2

Dugi SSCoach Points: 17998 More actions · Answer 8

WayneS (4/29/2010)
D.Oc (4/29/2010)
My bet is on GB's not TB.
Especially singe "G" is right under "T" on the keyboard...

I think we choose the situation, now what any solution scenario for this situation!?

============================================================
SELECT YOUR PROBLEM FROM SSC.com WHERE PROBLEM DESCRIPTION =
http://www.sqlservercentral.com/articles/Best+Practices/61537/[/url]

Grant Fritchey SSC Guru Points: 398909 More actions · Answer 9

Dugi (4/29/2010)
WayneS (4/29/2010)
D.Oc (4/29/2010)
My bet is on GB's not TB.
Especially singe "G" is right under "T" on the keyboard...
I think we choose the situation, now what any solution scenario for this situation!?

I haven't a clue. This is a whole different problem set and there are only a very few people on the planet that have had to deal with it.

"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt

Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning

Dwayne Dibley SSCrazy Points: 2350 More actions · Answer 10

I hope the OP returns to clarify the situation. Perhaps he will when he has finished lugging the backup tapes onto the truck...

Michael Valentine Jones SSC Guru Points: 64818 More actions · Answer 11

I really doubt there is an 800 TB database being managed by someone who has to ask questions here about how to do backups.

SQLRNNR SSC Guru Points: 281334 More actions · Answer 12

Steve Jones - Editor (4/29/2010)
It can be 800TB, but that is huge. Are you sure you're not mistaking the size by a factor of 10?
Also, I'm curious how you are backing up the log every 10 sec. Are you using some external program. AFAIK, jobs to schedule can be done every minute, not more granular than that.
Maybe you can post the output from
select * from sys.database_files

SQL 2008 now allows jobs to 10 sec.

Jason...AKA CirqueDeSQLeil
_______________________________________________
I have given a name to my pain...MCM SQL Server, MVP
SQL RNNR
Posting Performance Based Questions - Gail Shaw[/url]
Learn Extended Events

SQLRNNR SSC Guru Points: 281334 More actions · Answer 13

GilaMonster (4/29/2010)
Get a professional in. There are going to be very few people with experience in databases over a few TB and an 800TB has to be one of the biggest SQL Server databases around.
I'd suggest approaching Microsoft and seeing if you can get the CAT (customer advisory team) people involved. Or contact Paul Randal and see if he'd be available for some consulting
With a database this big, this active and as important as you indicate, you should be looking at getting the absolute top people in to do a professional and comprehensive job. Relying on free help for something like this is recklessness at best.

Absolutely. It is well worth it to pay for somebody to help with these issues.

Jason...AKA CirqueDeSQLeil
_______________________________________________
I have given a name to my pain...MCM SQL Server, MVP
SQL RNNR
Posting Performance Based Questions - Gail Shaw[/url]
Learn Extended Events

SQLRNNR SSC Guru Points: 281334 More actions · Answer 14

GregoryF (4/29/2010)
The server that could process all of this, and the SAN that could store it, and the networking that can transfer it, would out of reach for all but the richest of companies.
If it's true, I'd love to play with it

Me too!

Jason...AKA CirqueDeSQLeil
_______________________________________________
I have given a name to my pain...MCM SQL Server, MVP
SQL RNNR
Posting Performance Based Questions - Gail Shaw[/url]
Learn Extended Events

SQLRNNR SSC Guru Points: 281334 More actions · Answer 15

Michael Valentine Jones (4/29/2010)
I really doubt there is an 800 TB database being managed by someone who has to ask questions here about how to do backups.

Good point.

Jason...AKA CirqueDeSQLeil
_______________________________________________
I have given a name to my pain...MCM SQL Server, MVP
SQL RNNR
Posting Performance Based Questions - Gail Shaw[/url]
Learn Extended Events