Must Watch Video for all SQL DBA's - How MySpace.com is working with SQL Server on 3.4 million concurrent users

  • Check this wonderful video which every DBA should watch to understand how to plan scalablity. For me, I had never seen 150 Database Servers for Single Application :hehe:

    But MySpace.com is using those many DB Servers because they have 130 million users... Find more

    http://wtv.watchtechvideos.com/topic70.html

    Throw some comments about your experiences like this.... It will help upcoming DBA's

    Regards,
    Sakthi
    My Blog -> http://www.sqlserverdba.co.cc

  • Some Technical Details about MySpace.com Gathered from this Video

    Technical Details about MySpace.com as on Nov 2006:

    130 million users

    50 million Unique Users

    5 Billion Ads Per days

    260 new users/day

    3 billion total images

    3 million images added daily

    3.4 million concurrent users at peak

    5 Datacenters

    5,000 Web Servers

    160 Cache Servers 16gb RAM

    650 Ad Servers

    150 Database Servers

    6,000 Disk in SAN Architecture

    45,000 mb/s bandwidth

    50,000 mb/s on CDN

    Server Specifications of MySpace.com As fof De 2007:

    Server Brand: Hewlett-Packard (HP) 585 servers

    Processors/CPUs: 4 AMD Opteron dual-core, 64-bit processors

    Memory: 64 Gigabytes of memory (recently upgraded from 32)

    Operating System (OS): Windows 2003, Service Pack 1

    Database software (DBMS): Microsoft SQL Server 2005, Service Pack (64-bit)

    Network Communication: 10-gigabit-per-second Ethernet network card

    SAN Communication: 2 host bus adapters for storage area network (SAN) communications.

    The infrastructure for the core user profiles application includes 65 of these database servers with a total capacity of more than 2 terabytes of memory, 520 processors and 130 gigabytes of network throughput.

    Regards,
    Sakthi
    My Blog -> http://www.sqlserverdba.co.cc

  • And yet there are those who still say SQL Server doesn't "scale out"....

    - Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
    Property of The Thread

    "Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon

  • You are right! Only by experimenting one can proove capacity. People are under estimating SQL because it is not experimented in this kind'a large scale implementations.

    May be Microsoft need to sponsor this kind of large scale projects which will give lot of feedback to enhance their product!

    I Love SQL!

    Regards,
    Sakthi
    My Blog -> http://www.sqlserverdba.co.cc

  • That's a nice one shakthi 🙂

  • MySpace has been a great SQL Server story, but more importantly, they show where SQL Server has been weak and how they've had to re-architect things a few times to handle the loads.

    One very interesting fact I heard was that they have 30,000+ disks, mostly 15% full to handle the need to IO.

  • The 260 new users/day number seems way too low. It would take over 1,300 years to get to 130 million users at that rate.

  • So Steve is tracking MySpace Closely? Welcome back after holiday!

    Michael,

    260 new users/day is current registration scenario. Any site will have bulk registrations only when site is released newly..... You should be knowing PLC (Product Life cycle) which says anything will go through the cycle of Introduction --> Growth --> Maturity --> Decline.

    Regards,
    Sakthi
    My Blog -> http://www.sqlserverdba.co.cc

  • The slide said 260,000 users per day, even though the speaker said 260.

  • Watching the video today. I've seen a bunch of the early architectures in other presentations/articles, but this one is more interactive and interesting.

    One big thing to note is that architecture matters. SQL Server isn't enough (nor Oracle, nor more hardware, etc.) Think scale out early, even if you don't do it, consider what is needed. Having multiple databases is one way to plan this if you think you'll outgrow your current server.

    I did like that they went with 4 way boxes, 64 bit. A good, commodity server to scale with.

  • Sakthivel Chidambaram (2/9/2009)


    So Steve is tracking MySpace Closely? Welcome back after holiday!

    Michael,

    260 new users/day is current registration scenario. Any site will have bulk registrations only when site is released newly..... You should be knowing PLC (Product Life cycle) which says anything will go through the cycle of Introduction --> Growth --> Maturity --> Decline.

    I understand Product Life cycles, but 260 per day was still orders of magnitude too low to be believable. Even at 100 times that rate, 26,000 per day, it would take almost 14 years to grow to that level.

    Matt Stockham said in his last post that the slide had the real number, 260,000. That number seems a little high to me, but it's at least in the believable range.

  • One big thing to note is that architecture matters. SQL Server isn't enough (nor Oracle, nor more hardware, etc.) Think scale out early, even if you don't do it, consider what is needed. Having multiple databases is one way to plan this if you think you'll outgrow your current server.

    Do you think MySpace Founder's would have imagined such a tremendous growth? Please explain how to consider scale out early... I'm not getting the point!

    For Michael Valentine Jones:

    http://en.wikipedia.org/wiki/MySpace says 230,000 new users per day! As said by you "Believable!" compared to 260. May be this 230,000 was the highest registration happened in a day!

    Regards,
    Sakthi
    My Blog -> http://www.sqlserverdba.co.cc

  • Any body having similar kind of videos which will gives an idea about Scale-out can share here or that forum... It will be useful.

    Regards,
    Sakthi
    My Blog -> http://www.sqlserverdba.co.cc

  • Scale Out is hard, and it's mostly architectural.

    You want to spend a few minutes on it early, thinking about where you could separate out pieces of data to another server (separate db, instance, etc.), but not too much time. 99.99% of us never grow like MySpace.

    As you start to perceive issues, the administration overhead of scaling out has to exceed what you spend now on performance issues. Partitioning helps, replication can move read only data, but the application people have to be thinking as well about not tying themselves too tightly to one particular databases. Having web services or SOA stuff in general will help with being able to distribute data.

  • Friends,

    I was curious in getting the world's larget SQL Server implementations and after googling there is detail about only one...

    Implemented by : Microsoft & Unisys

    Instance Name : Interpay – 21 TB SQL instance

    Reference : http://download.microsoft.com/download/c/4/0/c40eec0a-c937-448d-a430-9643824fa713/1123_T2_S2.ppt

    Regards,
    Sakthi
    My Blog -> http://www.sqlserverdba.co.cc

Viewing 15 posts - 1 through 15 (of 24 total)

You must be logged in to reply to this topic. Login to reply