February 9, 2009 at 12:59 am
Check this wonderful video which every DBA should watch to understand how to plan scalablity. For me, I had never seen 150 Database Servers for Single Application :hehe:
But MySpace.com is using those many DB Servers because they have 130 million users... Find more
http://wtv.watchtechvideos.com/topic70.html
Throw some comments about your experiences like this.... It will help upcoming DBA's
Regards,
Sakthi
My Blog -> http://www.sqlserverdba.co.cc
February 9, 2009 at 6:54 am
Some Technical Details about MySpace.com Gathered from this Video
Technical Details about MySpace.com as on Nov 2006:
130 million users
50 million Unique Users
5 Billion Ads Per days
260 new users/day
3 billion total images
3 million images added daily
3.4 million concurrent users at peak
5 Datacenters
5,000 Web Servers
160 Cache Servers 16gb RAM
650 Ad Servers
150 Database Servers
6,000 Disk in SAN Architecture
45,000 mb/s bandwidth
50,000 mb/s on CDN
Server Specifications of MySpace.com As fof De 2007:
Server Brand: Hewlett-Packard (HP) 585 servers
Processors/CPUs: 4 AMD Opteron dual-core, 64-bit processors
Memory: 64 Gigabytes of memory (recently upgraded from 32)
Operating System (OS): Windows 2003, Service Pack 1
Database software (DBMS): Microsoft SQL Server 2005, Service Pack (64-bit)
Network Communication: 10-gigabit-per-second Ethernet network card
SAN Communication: 2 host bus adapters for storage area network (SAN) communications.
The infrastructure for the core user profiles application includes 65 of these database servers with a total capacity of more than 2 terabytes of memory, 520 processors and 130 gigabytes of network throughput.
Regards,
Sakthi
My Blog -> http://www.sqlserverdba.co.cc
February 9, 2009 at 7:03 am
And yet there are those who still say SQL Server doesn't "scale out"....
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
February 9, 2009 at 8:16 am
You are right! Only by experimenting one can proove capacity. People are under estimating SQL because it is not experimented in this kind'a large scale implementations.
May be Microsoft need to sponsor this kind of large scale projects which will give lot of feedback to enhance their product!
I Love SQL!
Regards,
Sakthi
My Blog -> http://www.sqlserverdba.co.cc
February 9, 2009 at 8:26 am
That's a nice one shakthi 🙂
February 9, 2009 at 8:33 am
MySpace has been a great SQL Server story, but more importantly, they show where SQL Server has been weak and how they've had to re-architect things a few times to handle the loads.
One very interesting fact I heard was that they have 30,000+ disks, mostly 15% full to handle the need to IO.
February 9, 2009 at 8:42 am
The 260 new users/day number seems way too low. It would take over 1,300 years to get to 130 million users at that rate.
February 9, 2009 at 9:19 am
So Steve is tracking MySpace Closely? Welcome back after holiday!
Michael,
260 new users/day is current registration scenario. Any site will have bulk registrations only when site is released newly..... You should be knowing PLC (Product Life cycle) which says anything will go through the cycle of Introduction --> Growth --> Maturity --> Decline.
Regards,
Sakthi
My Blog -> http://www.sqlserverdba.co.cc
February 9, 2009 at 9:36 am
The slide said 260,000 users per day, even though the speaker said 260.
February 9, 2009 at 9:43 am
Watching the video today. I've seen a bunch of the early architectures in other presentations/articles, but this one is more interactive and interesting.
One big thing to note is that architecture matters. SQL Server isn't enough (nor Oracle, nor more hardware, etc.) Think scale out early, even if you don't do it, consider what is needed. Having multiple databases is one way to plan this if you think you'll outgrow your current server.
I did like that they went with 4 way boxes, 64 bit. A good, commodity server to scale with.
February 9, 2009 at 9:46 am
Sakthivel Chidambaram (2/9/2009)
So Steve is tracking MySpace Closely? Welcome back after holiday!Michael,
260 new users/day is current registration scenario. Any site will have bulk registrations only when site is released newly..... You should be knowing PLC (Product Life cycle) which says anything will go through the cycle of Introduction --> Growth --> Maturity --> Decline.
I understand Product Life cycles, but 260 per day was still orders of magnitude too low to be believable. Even at 100 times that rate, 26,000 per day, it would take almost 14 years to grow to that level.
Matt Stockham said in his last post that the slide had the real number, 260,000. That number seems a little high to me, but it's at least in the believable range.
February 9, 2009 at 7:31 pm
One big thing to note is that architecture matters. SQL Server isn't enough (nor Oracle, nor more hardware, etc.) Think scale out early, even if you don't do it, consider what is needed. Having multiple databases is one way to plan this if you think you'll outgrow your current server.
Do you think MySpace Founder's would have imagined such a tremendous growth? Please explain how to consider scale out early... I'm not getting the point!
For Michael Valentine Jones:
http://en.wikipedia.org/wiki/MySpace says 230,000 new users per day! As said by you "Believable!" compared to 260. May be this 230,000 was the highest registration happened in a day!
Regards,
Sakthi
My Blog -> http://www.sqlserverdba.co.cc
February 9, 2009 at 11:48 pm
Any body having similar kind of videos which will gives an idea about Scale-out can share here or that forum... It will be useful.
Regards,
Sakthi
My Blog -> http://www.sqlserverdba.co.cc
February 10, 2009 at 8:43 am
Scale Out is hard, and it's mostly architectural.
You want to spend a few minutes on it early, thinking about where you could separate out pieces of data to another server (separate db, instance, etc.), but not too much time. 99.99% of us never grow like MySpace.
As you start to perceive issues, the administration overhead of scaling out has to exceed what you spend now on performance issues. Partitioning helps, replication can move read only data, but the application people have to be thinking as well about not tying themselves too tightly to one particular databases. Having web services or SOA stuff in general will help with being able to distribute data.
February 10, 2009 at 8:44 am
Friends,
I was curious in getting the world's larget SQL Server implementations and after googling there is detail about only one...
Implemented by : Microsoft & Unisys
Instance Name : Interpay – 21 TB SQL instance
Reference : http://download.microsoft.com/download/c/4/0/c40eec0a-c937-448d-a430-9643824fa713/1123_T2_S2.ppt
Regards,
Sakthi
My Blog -> http://www.sqlserverdba.co.cc
Viewing 15 posts - 1 through 15 (of 24 total)
You must be logged in to reply to this topic. Login to reply