January 13, 2011 at 9:16 pm
Comments posted to this topic are about the item Scaling Up
January 14, 2011 at 2:03 am
Tb level databases? I have enough problems with the few Gb databases that are all I ever need to deal with! 🙂
January 14, 2011 at 2:13 am
Ok, not large scale but I developed and maintain a 550 gig datawarehouse database, got 1 table with close to 900 million records, and anothers with 600 mil, 400 million and 300 million records.
This gets updated on a daily basis (at night) and during the day it is used for reporting.
2 years into the project and I cant complain, although I've learned a lot during the time about bulk loading and tuning the database for faster access.
But over all I would not standback for a database in the terabytes, not a problem, worked with them before.
January 14, 2011 at 2:43 am
I'd be happy as long as the business were realistic about the hardware requirements - my first job was in a business with a 250mb Oracle database on a 20mhz VMS machine with 20 interactive users and they refused to accept that the box wasn't up to the job even when quarter end routines, that's every 3 months to save you thinking about it ;-), took 4 months to run.
January 14, 2011 at 6:05 am
I have one 1.2 TB database currently on line and about 6 more to follow within the next 1 to 2 years. It is a scary experience learning how to deal with it from a maintenance and performance stand-point but it helps you to charpen all skills especially performance tuning. Managing VLDB(s) can certainly be a challenge but if management is supportive and you have the drive to meet the challenge it can be exciting at the same time.
January 14, 2011 at 6:21 am
Technical details are one thing which don't matter really to me. It's a great opportunity to learn.
What's the basic, and I believe it applies to any jobs, is how do you want to manage time between job and family / personal life. From the article you forecast long working hours.
Do you agree working the requirement minimum and get the maximum with your family and your life? Then you need to accept lower salary and advantages or in the contrary. working more for advancement but at the expense of your personal and family development?
There's a lot of factors involved and it's up to everyone to decide which path they want to take but in the end you need to ask yourself, "What am I the most proud of? Career, Family, Friends, personal development?" then you know where to spend your time.
But time is like any resources, you don't get enough to do all what you would want.
Oops: forgot adding: Yes I'm up to the challenge, but not at the expense of what count the most for me.
January 14, 2011 at 6:31 am
No way. I never really enjoyed the Admin side, and haven't been involved with it in over a decade. Love SQL development though. Back in the 90's I managed many client DBs in the 100s of GB range, and oh what a PAIN! Got to have too much patience and time for that stuff. Not for me.
January 14, 2011 at 6:37 am
We have numerous TB sized datamarts on the same server. The largest table is 13 billion rows , 1,250 GB of data. The main focus is building and testing the environments to handle the workload before the first byte of data arrives.
January 14, 2011 at 6:40 am
I have young kids and value my time with them over any increase in compensation or the thrill of the additional challenges. No amount of money can replace the time I spend with them now after work and on weekends.
I prefer development to admin anyway but even if not, same answer.
January 14, 2011 at 6:54 am
I'm working on a development which may end up as having the largest SQL Server database in Europe, its not quite 800 GB at the moment but the project has only just started rolling out. It's a new challenge but worth getting stuck in just to see what can be done, especially with replicating same to different environments. I hadn't realised you could replicate from a snapshots.
January 14, 2011 at 6:56 am
I am not at a stage of life where long hours are to much of a hinderance however I prefer to only have them occasionally. The db's I typically get pulled into are between 500GB and 1.2 TB of used space and OLTP. I think that the larger size is wonderful as long as you have the controls and knowhow to put them on adequate equipment for your needs. I would dread running some of my databases on systems from even 5 years ago. Now that we have our customers understanding what a powerful server\SAN can do for them though it is no where near as stressfull and time consuming.
It goes back to the age old argument that SQL will never outperform Oracle and my simple response of Put SQL Server 2005\2008 on a server with equal horsepower and equal coddling and you will see that it does just fine.
I have noticed though that many of my customers' db teams are extremely leary about even looking at our db's because of their size and the unknowns. They are scary beasts until you get to know them.
January 14, 2011 at 7:11 am
"My thought is that at this stage of my life, I don't need the long hours required to deal with some operations on these systems. Simple checks, test restores, and all sorts of maintenance become larger issues when you are dealing with TBs of data. Unless we get substantially faster hardware in the future (in terms of IOPS), I can't imagine how we will handle PB sized databases."
"Would you want to manage a large scale SQL implementation?"
This answer depends on a few things though, it's not just a simple answer for everyone.
1. Where are you in your career?
2. How old are you?
3. Are you presently in good health?
4. Are you married?
5. Do you have children?
6. Are you used to dealing with complicated problems and solutions?
7. Do you have a social life?
8. Do you handle stress well?
All, or even any of these things above drive the answer to this question. Also, I would like to hear Paul S. Randal's viewpoint on this as well. 😀
"Technology is a weird thing. It brings you great gifts with one hand, and it stabs you in the back with the other. ...:-D"
January 14, 2011 at 7:44 am
TravisDBA (1/14/2011)
All, or even any of these things above drive the answer to this question. Also, I would like to hear Paul S. Randal's viewpoint on this as well. 😀
I think Paul would be fine with it, especially since he travels a lot each year. He's used to long hours.
I might get there at some point, but the questions you list discourage me at this time. My youngest is 9, so I can see myself taking a job that could require some long hours and on-call at some point in the future.
January 14, 2011 at 8:30 am
Hundreds of TBs? No thanks. Not interested. Mostly because of management's unrealistic expectations. Tech life was *much* easier before everyone used Google, FB, Amazon, iPhones, etc. and then expected the same "magic" in their corporations -- without allocating the needed resources. Biggest DB I manage is less than 30G, so recovery times never require a cot at work.
January 14, 2011 at 8:46 am
andycao (1/14/2011)
...recovery times never require a cot at work.
An excellent metric to use.
Viewing 15 posts - 1 through 15 (of 26 total)
You must be logged in to reply to this topic. Login to reply