December 4, 2010 at 6:10 pm
Heh... <soap box mode on>
I'll agree that some bloatware has caused problems and that data is growing but I think the real problem is that some really sloppy code has been written in just about every application all over the world. A lot of developers don't really know it's sloppy because machines are so bloody fast now that they cover up a lot of performance problems during development especially in development databases where rowcounts are typically very low. Different story when it finally gets to the customer, though.
How many times have we seen on these very forums where someone justifies some form of RBAR by saying "It'll never get bigger than {insert some low number here} rows"? I've also seen a similar attitude towards memory usage on the "application" side of the house as well as the RBAR and performance problems caused by ORM's that no one writing the application seems to ever concern themselves with.
Then there's the ol' "Never optimize prematurely" thing that so many people believe in. The problem is, they rarely go back to find out if an increase in scale will cause their code problems. They usually wait until it becomes a problem and then they patch it almost as badly as the original write. It's pretty easy to write code that's been optimized by 90 or 95% without killing a schedule. Why fix something that's not broke? Actually, it is broken... you just haven't seen it, yet. 😉
Then there's the famous, "We need to deliver it this week" problem where people take all sorts of shortcuts to deliver against a terrible schedule. They even justify the use of RBAR to meet deadlines which is a bit ironic because it's actually easier and faster to write set-based code. Very few people go back to fix any of those shortcuts until it becomes a problem and, by then, it's too late. The company has a black eye in the face of the public where, unless they have a corner on the market like MS does, it could cause loss of future sales in a competitive market.
Let's go one step further. Many of the gurus on this forum have taken posted code and made it hundreds and even thousands of times faster than the original code. It's taken almost 10 years for computer clock rates just to triple. Where is anyone going to find a machine that's "thousands" of times faster? If you have DBCC TIMEWARP installed, you might be able to get that today but, otherwise, it's just not going to happen anytime soon. In that vein, I've personally experienced where the company went from a 1.4 GHz 4 CPU server with 3 gig of Ram to a 3GHz 8 CPU 16GB machine with dual network cards and a brand new high performance SAN with virtually no change in performance of the code (both on the database side and on the GUI side) because of the way it was written. Even if you can only make the code 2 or 3 times faster, paying customers are going to notice and drift your way.
Then, there's the selection of the right tool for the right job. It's all too comfortable a thing to use an ORM or embed some SQL in the GUI. It's all too comfortable to move all the data you want to work with to the client and have client code do something like a dupe-check. Yeah... I've been there, as well. To make a very long story much shorter, we had 3rd party client code that was supposed to check daily created DB's (yeah... that's another story) for 3 months back. 93 DB's, 1 table of 4 mega-rows each. A "mere" 372,000,000 rows of data to dupe check in groups of 3 databases at a time (1 for each month for the same day). They couldn't do it in a single day. The best they could do was 62 DB's and it would usually fail multiple times during the day. We rewrote that bit of computational heaven so it would all be done in the database... it only took 45 minutes to do all 93 DB's.
Yeah... let's do DB stuff in the GUI. 😀
Speed is right where I've always said it's at... it's in the code. You just have to write the right stuff using the right tool... and there doesn't need to be a decrease in functionality or flair to do it... and you don't need to miss a schedule because you wrote good code. 😛
<soap box mode off>
😉
--Jeff Moden
Change is inevitable... Change for the better is not.
December 5, 2010 at 1:10 am
I know there's all the rendering bits and whatnot to make sites pretty, but I know what 2400 baud looks like. I tried playing graphical MUSH's on them. I almost cried on my first 9600 when I installed it.
That's some interesting data in the article Steve linked. I didn't dig deep enough into the controls on the experiment but a simple 100 ms increase in response time on the site killing 20% of revenue seems... outrageous. Unless it was 100 ms per query call and you had 30 going on the same page... okay, I could see that. (Meh, site's slow today, I'll come back tomorrow.)
To go to along with Jeff's commentary above, I know as a techie myself I love my quad monitor 4billion RAM 16 core desktop with local server installed. She flies baby, FLIES!
She's also absolutely *useless* when I'm trying to see what a customer response might look like. I may bring this up at our next tech meeting. My current environment has a number of cross-trained folks (app/sql) using localhost as the primary dev environment. They never really see what the server does. I have heard this statement literally: "Yeah, it's 8 seconds... but that's on my piece of crap." I'm trying to drill into their heads: "And your piece of crap has NO competition for the data or resources."
Dev boxes need to be older hardware. Yes, I said it. It needs to be on a computer with enough processor to try to parallel, not enough RAM to hold everything on god's green earth in memory, and with a single NIC connector to the world. There's not enough pressure on Dev constantly to really see an impact, so it needs to be a piece of crap to MAKE you code it well.
Speed matters. It will always matter. It's, really, the only damned thing that matters, from the data side. Do you honestly care if you spin 4000 more reads off a query if you cut the time in half? Yes, it's an indicator you can do better, but really... do you optimizer for reads except in special case scenario? No, you optimize to time. The almighty millisecond.
(Off my soapbox now too. I apologize if this rant went long or semi-incoherent... I haven't slept since ...um... 42 hours ago or so... and if you really want to give me advice or ask why send a PM.)
Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.
For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA
December 5, 2010 at 5:57 pm
Craig Farrell (12/5/2010)
Dev boxes need to be older hardware. Yes, I said it. It needs to be on a computer with enough processor to try to parallel, not enough RAM to hold everything on god's green earth in memory, and with a single NIC connector to the world. There's not enough pressure on Dev constantly to really see an impact, so it needs to be a piece of crap to MAKE you code it well.
Well said and I agree. You also need a good set of data that mimics the sizes you'll face in production. Actually, if you want to build the application really well, increase your test sizes by 20% over production and make those queries run quick
December 5, 2010 at 10:17 pm
At the risk of sounding like a nit-picking pedant, in international English one says 'more quickly'.
I.e. 'my query runs more quickly', not 'my query runs quicker'
The days of grammatically correct journalism are numbered anyway, I'm sure.
December 6, 2010 at 6:27 am
ddezwaan (12/5/2010)
At the risk of sounding like a nit-picking pedant, in international English one says 'more quickly'.I.e. 'my query runs more quickly', not 'my query runs quicker'
The days of grammatically correct journalism are numbered anyway, I'm sure.
Interesting point. The Cambridge Dictionary doesn't list "quick" as an adverb, so going by that, you are right. The American Heritage Dictionary does, however, and so the construction "my query runs quicker" would be absolutely correct. Do you have a source for your international English grammar at all?
http://dictionary.cambridge.org/dictionary/british/quick_1
http://www.yourdictionary.com/quick
John
December 6, 2010 at 6:43 am
Jeff
Heh... <soap box mode on>
I went looking for your desk, thought that you were sitting next to me or at least close by:hehe:
I standardize on a minimum of 2 years of data where I am, regardless of what I am told will be the length of time we maintain the data. I have had too many occaisions where I created a table/database with the scope telling me that we are keeping 2 weeks to have that increase to 30 days to 1 year to 18 months. (See a pattern?)
My current project, delete 3 years of data from a database which was supposed to have 2 years of data in it. In some tables that is over 1B rows of data - and BTW, don't impact day to day operation.
Steve Jimmo
Sr DBA
“If we ever forget that we are One Nation Under God, then we will be a Nation gone under." - Ronald Reagan
December 6, 2010 at 9:20 am
In my experience, performance limitations are more often caused by a design decision than by sloppy code, such a decision to go with cubes. They are easy to design, they accommodate sparse data easily, they are convenient for business analysts, but their performance sucks and you cannot even cache them actively.
On most of my projects over the past three or four years the number one requirement was that the code must be easy to read and maintain. Performance came way after that.
December 6, 2010 at 3:53 pm
sjimmo (12/6/2010)
Jeff
Heh... <soap box mode on>
I went looking for your desk, thought that you were sitting next to me or at least close by:hehe:
I standardize on a minimum of 2 years of data where I am, regardless of what I am told will be the length of time we maintain the data. I have had too many occaisions where I created a table/database with the scope telling me that we are keeping 2 weeks to have that increase to 30 days to 1 year to 18 months. (See a pattern?)
My current project, delete 3 years of data from a database which was supposed to have 2 years of data in it. In some tables that is over 1B rows of data - and BTW, don't impact day to day operation.
I'm definitely the same way... almost everything I build has to survive (at least) a million row test for performance.
Thanks for the feedback... nice to see there are kindred-spirits out there. 🙂
--Jeff Moden
Change is inevitable... Change for the better is not.
December 7, 2010 at 11:38 am
Speed's nice, but I prefer to think of all the oomf in modern computers as power, not speed. A V8 engine is only "fast" if it's pulling a light car. Try having a V8 pull a dual-trailer cargo rig (think "18-wheeler with two trailers", c. 150+ tons), and see how "fast" it is. Well, modern computers are pulling more and more load. More RPMs will only take you so far.
Jeff's bit about fast code is true, but really it's about the power of the code. How many rows can it tow at one time, at a speed that's not going to get read-ended by a glacier?
Most devs seem to focus on nitrous injection on a unicycle with a V8, test it on a closed track, and don't realize the production version needs to haul a couple of hundred tons of cargo on a dirt road. If they worry about anything, it's stability (understandable on a unicycle with a V8), crash-prevention (understandable again), and maybe on whether the ignition key is secure enough being stored in a pants pocket.
That analogy seems to work.
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
Viewing 10 posts - 1 through 9 (of 9 total)
You must be logged in to reply to this topic. Login to reply