April 30, 2008 at 9:52 pm
Comments posted to this topic are about the item And You Thought You Had a Big Database
May 1, 2008 at 6:17 am
Why would you let a database get to 1TB in SQL Server?
Wouldn't it be easier for backups and restores to seperate each large table to it's own database, then have a single database that's just views? Backups could be faster cause they could be ran in parallel and if someone wiped out a single table, then only that table could be restored.
Maybe when they are saying 1TB, they mean 1TB worth of databases?
On my main server I have about 30 databases that are max 30gb data and 10gb log. If the log is clear then the zipped file of the backup will still fit on a DVD-R and transfer over our screaming 10mip network in a reasonable time. A couple of the databases only house 1 or 2 tables.
Yes, we still operate on a 10mip network and yes I do work for a MAJOR telecommunications company that just changed their name recently to a company they bought.
May 1, 2008 at 7:13 am
design is a huge factor in size. I work mainly on check imaging databases for research purposes. One system i supported stored images in the db. The largest db for this version was approx 800GB. Not bad seeing as it was 'designed' for small sites typically < 2-300GB. The other system i mainly support stores images in file shares and stores pointer information. The largest we have currently is 1.2TB of data (1.7TB on disk). If we include the image space we would be talking in excess of 12TB. I couldn't imagine attempting to support a db inclusive of 12+TB of images. Especially as this is not the end of growth. The customer will mainly retain < 7years of data and thus will probably max out the db size around 2TB for the main data.
It does really put it into perspective though when talking with other dba's about their sizes (insert snickering here).
Fortunatley for me only a few are in the TB arena. The majority fit between 100 and 700GB. i would prefer less databases overall as if we were to do the 'total db size supported' as you mentioned i would be in the neighborhood of 100TB plus approx 1000TB of image storage. Fortunatley they are spread around at all different locations and customer sites.
May 1, 2008 at 7:17 am
There are plenty of databases that are in the TB range because they have that much data. Going to a different architecture, especially with views, doesn't buy you anything. You still have to backup everything together.
When I was looking for a job in 2001, there were two companies on SS2K that interviewed me. One had a 13TB SQL Server database up in the mountains (too much driving) and one had a 2TB database for credit card processing.
They exist and they bring their own challenges.
May 1, 2008 at 7:33 am
I think some of the more important reasons to maintain all the data in one database are transactional integrity and referential integrity. Doing a restore to a common, consistent transactional state across multiple databases is really almost impossible if you have a busy database. DRI is not supported across databases, so you would lose that. Those are awfully important items to give up to make it easier to do backups.
I think mantaining 1 TB databases are not that big a deal, but you have to make sure that you have all the storage and backup capacity needed to properly manage them. There are many good ways to get backups in a reasonable amount of time: LiteSpeed or other utilities, backups to multiple output files in parallel, filegroup or file backups, differential backups, etc.
May 1, 2008 at 7:41 am
The Oracle vs SQL Server debate is a good one when it comes to size. I do not have much knowledge of DB2 but I will ignorantly put it in the Oracle group.
What do you install SQL Server on - Any size machine
What do you install Oracle\DB2 on - Enterprise class machines
I would put good money on the fact that if you architect a MSSQL system to the degree that an equally scaled Oracle\DB2 system is, you would see very similar results.
May 1, 2008 at 9:20 am
I guess the whole 'restore to point in time' would kill the multiple-databases-with-one-view-database model. I am limited on stoage and network speed, that is why I have developed the system I use now.
I can't transfer a single file over about 5gb off my server due to network constraints. I know it sucks, but I have learned to deal with it. Can you believe it - a MAJOR communications company's headquarters with 5,000+ employees is running at 10mp????
I just wish some manager would do a study on the soft dollars lost each month with 5,000+ emps in my building alone operating on at 10mp network compared to a 100mp network. Considering all it probably would be is to replace a couple of routers.
Manager's don't see soft dollars though, cause it doesn't make their bottom line look bad. That's another thread all together though.
May 1, 2008 at 9:58 am
Robert Hermsen (5/1/2008)
The Oracle vs SQL Server debate is a good one when it comes to size. I do not have much knowledge of DB2 but I will ignorantly put it in the Oracle group.What do you install SQL Server on - Any size machine
What do you install Oracle\DB2 on - Enterprise class machines
I would put good money on the fact that if you architect a MSSQL system to the degree that an equally scaled Oracle\DB2 system is, you would see very similar results.
I'm, right now, going through enhancements to a production system. I backed up their database and restored to my laptop. Queries that take me minutes run in a fraction of a second on the real (multi-core, fast RAID, boatload of memory) hardware.
ATBCharles Kincaid
May 1, 2008 at 10:09 am
Will Summers (5/1/2008)
Can you believe it - a MAJOR communications company's headquarters with 5,000+ employees is running at 10mp????I just wish some manager would do a study on the soft dollars lost each month with 5,000+ emps in my building alone operating on at 10mp network compared to a 100mp network. Considering all it probably would be is to replace a couple of routers.
Manager's don't see soft dollars though, cause it doesn't make their bottom line look bad. That's another thread all together though.
As part of an upgrade plan one former customer wanted performance enhancement recommendations. I showed how their building network using cascaded hubs was a bottle neck. The brute force traffic study showed that the Network Neighborhood browser keep alive packets were accounting for about a 10% load with NOT A SINGLE USER at a desk. :w00t:
They replaced the contents of the network closet with a 256 port HUB. :crazy: Actual improvement on traffic - Zero. The laugh I enjoyed on the drive home - Priceless.
ATBCharles Kincaid
May 1, 2008 at 10:30 am
I showed how their building network using cascaded hubs was a bottle neck. The brute force traffic study showed that the Network Neighborhood browser keep alive packets were accounting for about a 10% load with NOT A SINGLE USER at a desk. :w00t:
They replaced the contents of the network closet with a 256 port HUB. :crazy: Actual improvement on traffic - Zero. The laugh I enjoyed on the drive home - Priceless.
By reading your post, it seems that you were recommending them to goto less HUBs. When they did and no performance was gained, you lauged? I'm sure I'm missing something vital here. What was your recommendation for them to do? I'm assuming it really wasn't to goto the 256 port HUB like it seems in your post.
On my network at home I have a this product:
I have my cable modem hooked up to my wireless router that is connected to the above switch to it to give internet access to my 10 PCs at home. I have about 22 wired connections in my basement, so every port is filled in the switch.
Two questions :
1. What software did you use to do the brute force study?
2. I have a 5mp internet connection coming into my house. What would you suggest to improve my network speeds?
May 1, 2008 at 10:57 am
Will Summers (5/1/2008)
I showed how their building network using cascaded hubs was a bottle neck. The brute force traffic study showed that the Network Neighborhood browser keep alive packets were accounting for about a 10% load with NOT A SINGLE USER at a desk. :w00t:
They replaced the contents of the network closet with a 256 port HUB. :crazy: Actual improvement on traffic - Zero. The laugh I enjoyed on the drive home - Priceless.
By reading your post, it seems that you were recommending them to goto less HUBs. When they did and no performance was gained, you lauged? I'm sure I'm missing something vital here. What was your recommendation for them to do? I'm assuming it really wasn't to goto the 256 port HUB like it seems in your post...
Oh course I can't speak for Charles, but I am sure he would have recommended going from hubs to switches at the very least, and maybe to use routers to limit the traffic.
Hubs send all the traffic to all the nodes, so the chatter that each node hears can be enormous. Switches only send the traffic where it needs to go.
May 1, 2008 at 11:42 am
I kinda figured that his recommendation was to goto switches/routers, but I'm not a network guy, so I wasn't sure. I set my network up in my basement about 2 years ago and have just forgot about it. I couldn't remember which was bad - the hub or switch. I remember now that a switch is basically an intelligent hub.
May 1, 2008 at 11:57 am
Will,
The problem with Hubs is that every packet that come in goes out all the other ports. They should have put in an N+1 switch. Even better would have been to put in a tree of smaller switches. This was a government building that housed several agencies. Every PC, and server, on that floor saw every bit of traffic from everybody else. Some manager tries to print a picture of his kid on the new IP color printer next to him blocks network traffic for everybody else. Thankfully this was in the days before steaming audio and video.
I could not speak about your home network. Shop around for a local consultant to give you a visit.
ATBCharles Kincaid
May 1, 2008 at 12:02 pm
So how do they even sell hubs anymore? I think I got a good setup at home. Nice dodge on the network recommendation. jk. lol.
May 1, 2008 at 12:12 pm
It used to be that Hubs were cheaper, so there was a price point advantage. High-end switches are still quite a bit more expensive.
On the low end - the price difference is so small these days I'm not sure why you wouldn't pay for a switch.
----------------------------------------------------------------------------------
Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?
Viewing 15 posts - 1 through 15 (of 16 total)
You must be logged in to reply to this topic. Login to reply