Post reply

Will this design scale?

  • February 21, 2008 at 8:54 am

    #780700

    Well I can't imagine Microsoft teaching or advocating anything that would incline one towards switching away from SQL Server, so it must be some pointy-headed professors in CS classes. If nothing else these Agile articles make very interesting reading. If developers are going out of their way to avoid or go around data professionals, obviously something is broken. I don't think any of this is off-topic because it all bears heavily on which development projects will perform or scale.

    http://www.agiledata.org/essays/vision.html

    http://www.agiledata.org/essays/culturalImpedanceMismatch.html

  • February 21, 2008 at 9:19 am

    #780718

    Yeah, I've heard all those complaints. Then, we get a query that's running for 48 hours that somone finally brings over to us. We spend 3-4 hours on it and now it runs in under 1 second. We've been able to document time, after time again, that we add value. I had a manager back when I was a consultant who use to say, "You can pay me and listen to me, or you can not listen to me but keep on paying me. Your choice."

    I'm honestly not sure why there's such a vast chasm between developers & DBA's. I've done both jobs. But it's there and the DBA's, because they're the gatekeepers, are seen as the bad guys. It's a serious problem when the developers work around us rather than work with us. Your code for example, won't scale. As long as the querying methodology they use is flawed, they're just going to dig a hole. No amount of CPU or memory is going to satisify it as it grows. You have to code correctly.

    Seriously though, the one thing that confuses me about this is that it's true of VB & C# code too. You can write horrifying spaghetti code that runs for crap and is impossible to maintain or you can learn to do your job appropriately and write clean code that scales. Why should the database be any different? But apparently to lots of developers, it is. I just don't understand it.

    "The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
    - Theodore Roosevelt

    Author of:
    SQL Server Execution Plans
    SQL Server Query Performance Tuning

  • February 21, 2008 at 10:09 am

    #780746

    The big issue with the "make everything generic so that the data provider can change" is that pretty much invariably - your performance will go right to the sewer.

    Too much abstraction is a horrible problem. Making a data-oriented work the same way whether is is based on data from an excel spreadsheet vs Access vs XML vs flat files vs an RDBMS might be a neat trick, but it's going to make everything run at the lowest common denominator (meaning - badly). It also makes your support harder, since more abstraction ultimately leads to more possible places for a disconnect. Use only as much abstraction as is absolutely required (IMO).

    Microsoft isn't even trying to deny it - I was at an "official" presentation re: LINQ where the presenter was PROUD to say that (with a series of steps that make your head spin), you could make "most of your LINQ code run at 80% of the performance of native SQL queries". Notice that "most" means it will make some bad choices, and probably tear a rather large hole in the the space-time continuum (or at least in your tempDB), and that your "run of the mill" queries are still going to be crippled. Where's the advantage in that? No - really - who actually benefited there?

    Oh - by the way - there IS no LINQ provider for anything other than SQL Server at this point, so you will be in for a rude surprise if they want to "switch to Oracle tomorrow".

    Now - the communications barriers between teams are a HUGE problem (admittedly). But coding around the DBA is just like tossing a hand grenade into a room full of people: messy and hurts a LOT of people....

    Just my .00001 cents.

    ----------------------------------------------------------------------------------
    Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?

  • February 25, 2008 at 7:30 pm

    #781836

    Hi,

    Its hard to say whther your design supports large volumes of data or not!. The way you should see your database design is by having set of Non-Functional requirements and trying to validat aginst to this. This is may include who is accessing the data? Is there any daily data load from other different ssystems? Too much of normalized causes the perfomrnace issues interms of data load and data access. At the same time you need to rebuild indexes and regulary update the stats.

    Thanks -- Vj

    http://dotnetvj.blogspot.com

    Thanks -- Vijaya Kadiyala
    www.dotnetvj.com
    SQL Server Articles For Beginers


  • February 25, 2008 at 9:32 pm

    #781843

    The chart a few posts ago just begs the question of what it the typical ratio of DBA's to developers in your organization. In many organizations the ratio of DBA's to developers has got to approach 1 to 100, and a lot of the time the DBA's are not dedicated to development - thus production issues trump working with the developers on a day to day basis.

    How many of us are development vs. production DBAs?

Viewing 5 posts - 31 through 34 (of 34 total)

You must be logged in to reply to this topic. Login to reply