November 3, 2016 at 10:53 am
xsevensinzx (11/3/2016)
That's my life in a nutshell. How to support super large reads on a single machine without breaking the bank.
I think you're a bit of an edge case.
Also, sometimes you can reframe the question so queries run more efficiently. Sometimes you can't, and then it's a business decision. Pay in time or pay in $$.
November 3, 2016 at 10:54 am
Eric M Russell (11/3/2016)
So instead of the traditional Scope Triangle of Time / Cost / Quality, we may need to now add Coffee.
Ah, coffee is a given. No need to debate this.
November 3, 2016 at 11:34 am
The trouble I find on some projects is that they have done impact assessments (or just guesswork) in terms of costs and some organisations are less that happy to reallocate project budgets e.g. if you could use some of the pool of money allocated to software development staffing costs and try to reallocate it to buying hardware then you may come across issues within the financial management of one's project. This is often petty but enforced.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
November 3, 2016 at 11:45 am
It always surprises me that companies will have no issues putting up 6-7 figures on deployments in licensing/man hours but then put up a huge fight over an extra 5-10 thousand on the hardware. There's a finite amount that good coding can overcome but for some reason there's just some inherent distrust of even small requests for better hardware.
November 3, 2016 at 12:35 pm
ZZartin (11/3/2016)
It always surprises me that companies will have no issues putting up 6-7 figures on deployments in licensing/man hours but then put up a huge fight over an extra 5-10 thousand on the hardware. There's a finite amount that good coding can overcome but for some reason there's just some inherent distrust of even small requests for better hardware.
Crazy, huh?
November 3, 2016 at 1:05 pm
I enjoyed the expensify article.
There seem to be quite a few articles emerging along similar lines. Some of the articles I've seen advocating a slimmed down architecture sized for your company are coming from companies who were advocating complex architectures built for "web scale".
I've watched the NOSQL evolution with a mix of frustration and amusement.
First, they discovered why we like ACID. BASE really didn't work out for bitcoin exvhanges.
Next they discovered that SQL is a useful abstraction language.
Now they are discovering that joining data together opens up a whole world of opportunities.
Makes me wonder what they'll disvover next.
I've been working in data fields my entire career and the two problems I run into the most are the data quality and the data model. Rarely do I find a technology problem.
REDIS I like, that does solve problems we used to try to solve in RDBMS's such as session state.
Quite a few try to solve a problem that doesn't really exist while inventing very real technology problems.
Many sacrifice the needs of the many for the convenience of the few.
November 6, 2016 at 11:50 pm
Eric M Russell (11/3/2016)
So instead of the traditional Scope Triangle of Time / Cost / Quality, we may need to now add Coffee.
Well Eric, coffee is important. It's like petrol to the car.
Manie Verster
Developer
Johannesburg
South Africa
I am happy because I choose to be happy.
I just love my job!!!
November 7, 2016 at 7:40 pm
Eric M Russell (11/3/2016)
So instead of the traditional Scope Triangle of Time / Cost / Quality, we may need to now add Coffee.
Sounds good to me. 😀
December 20, 2019 at 4:31 pm
"..dual core 2.8Ghz CPU with 500GB of storage.."
Maybe it was all a misunderstanding; I'm pretty sure the above specifications were just a request for upgrading the DBA's laptop.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
December 23, 2019 at 2:42 pm
On the subject of the tired adage of "Good/Fast/Cheap... Pick 2.", I end up saying to people that tell me that, "Ummmm... Ok... I thought you were a professional". 😉
There's only one answer to that question and that's "Good". And, speaking of "good", if they're good, fast and cheap will happen naturally.
On the subject of hardware, I absolutely agree that good hardware is necessary. I can do a compressed backup of one of my 1TB databases on my Prod box in about 26 minutes. Doing the same thing on my Dev box, even when pointed to the same destination for the backups, takes more than 2 hours. While that happens overnight and doesn't matter to most folks, that's when a whole bunch of other stuff also happens on both boxes. Hardware makes a huge difference there.
I've also seen it where some code is so bad that hardware doesn't matter. There has been more than one instance at the companies I've worked for in the past where people did upgrades to hardware and changed from the Standard to the Enterprise editions, doubled the memory, and tripled (16 to 48) the core threads by tripling the number of core (8 to 24) and expected all code to run 10 or more times faster and then couldn't understand why almost none of the code saw an improvement. The best improvement was 2X on a couple (and I do mean just a couple) large batch jobs (which was great) and some of the code actually ran slower.
Hardware is good and having it enables you to run things faster but... performance is in the code... or not. Lousy code will bring even super servers to their knees.
One of my "favorite" such code problems was a piece of code that ran 10's of thousands of times each day. It executed in "only" 100 milliseconds (which I thought was terrible and fixed it to execute in only 2ms) but that wasn't the big issue. The issue was that it was written in such a way where it would do a recompile EVERY time it executed. While even that may not sound bad to a lot of people, it was taking between 2 and 22 seconds to recompile EVERY time it executed until we fixed it.
Bad code is the great equalizer. Without "coffee", the hardware may not matter. 😀
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 10 posts - 16 through 24 (of 24 total)
You must be logged in to reply to this topic. Login to reply