How Often Does Production Drift?

Question

How Often Does Production Drift?

Steve Jones - SSC Editor

SSC Guru

Points: 736212
More actions
November 22, 2014 at 11:33 am

#292385

Comments posted to this topic are about the item How Often Does Production Drift?

Viewing 12 posts - 1 through 11 (of 11 total)

You must be logged in to reply to this topic. Login to reply

Gary Varga SSC Guru Points: 82166 More actions · Answer 1

This is almost as important to developers as it is to DBAs as this can lead to developers creating issues as what they are developing against isn't what they will be running against. Assumptions will be made and, therefore, errors made. This can be better or worse depending on what the responsibilities of the developers and the DBAs are i.e. who does what.

Gaz

-- Stop your grinnin' and drop your linen...they're everywhere!!!

Yet Another DBA SSCarpal Tunnel Points: 4299 More actions · Answer 2

Yet Another DBA

SSCarpal Tunnel

Points: 4299

November 24, 2014 at 7:18 am

#1760506

Drift?? More like herding cats!!!

akljfhnlaflkj SSC Guru Points: 76202 More actions · Answer 3

I'm a little confused. If you are talking about changes to production outside of a normal release, then this certainly is high risk, unless it goes through the same testing that a normal release would have. There will always be those emergencies where production updates can't wait until next month. But those off-the-cuff updates still have to be tested. The reality is that they are never tested enough.

Andrew..Peterson SSCertifiable Points: 6838 More actions · Answer 4

Change management is always an issue. And many of us, if we have had the DBA hat on, have had times that required a quick fix, one that could not wait for even a bit of testing.

So, how does this tool, SQL Lighthouse, differ from RedGate's tool that does a comparison between databases? It is really just a comparison tool, right. Or does it look read thru the system tables looking for changes based on date?

As for third party systems, that do not even allow index changes under the EULA. I really hate that limitation. The organization has to live with either breaking the EULA, or living with really slow systems. Or, trying to get a EULA exception, if they can.

The more you are prepared, the less you need it.

Steve Jones - SSC Editor SSC Guru Points: 736212 More actions · Answer 5

Iwas Bornready (11/24/2014)
I'm a little confused. If you are talking about changes to production outside of a normal release, then this certainly is high risk, unless it goes through the same testing that a normal release would have. There will always be those emergencies where production updates can't wait until next month. But those off-the-cuff updates still have to be tested. The reality is that they are never tested enough.

Yes, ad hoc changes that haven't gone through a testing cycle.

Steve Jones - SSC Editor SSC Guru Points: 736212 More actions · Answer 6

Andrew..Peterson (11/24/2014)
So, how does this tool, SQL Lighthouse, differ from RedGate's tool that does a comparison between databases? It is really just a comparison tool, right. Or does it look read thru the system tables looking for changes based on date?

It actually works based on a DDL trigger, so that it can track who and when a change was made, but it also recognizes versions of schemas.

Matt Miller (4) SSC Guru Points: 124210 More actions · Answer 7

Steve Jones - SSC Editor (11/24/2014)
Iwas Bornready (11/24/2014)
I'm a little confused. If you are talking about changes to production outside of a normal release, then this certainly is high risk, unless it goes through the same testing that a normal release would have. There will always be those emergencies where production updates can't wait until next month. But those off-the-cuff updates still have to be tested. The reality is that they are never tested enough.
Yes, ad hoc changes that haven't gone through a testing cycle.

We have an abbreviated testing cycle for hot-fixes and/or emergency updates. Rather than going through the typical lifecycle, they are built up and deplyoed to MO (our pre-production environment, where we perform UAT and perf testing). This is one environment that is always at the same code level as production and hasn't been "polluted" with later builds that are still underway. An abbreviated regression, supplemented with specific tests to check the effectiveness of the patch against whatever is "burning", and an abbreviated stress test occurs before it heads to prod.

The challenge is then to merge those changes back into QA and dev once it goes up to prod. One lesson we've learned is to make sure you mergge BEFORE anything else deploys (so that the next "conventional" build doesn't reintroduce the production issue you're trying to fix); not so hard to do, as long as you remember the communication to the "main branch" team.

(edited for typos)

----------------------------------------------------------------------------------
Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?

Evil Kraig F SSC Guru Points: 100851 More actions · Answer 8

Most places I've worked at avoid drift by doing a development refresh from production (with necessary scrambling of identifiers/CC#s/etc, if necessary). At worst, a sprint is only a few changes off from the production version, which is corrected by the next refresh.

I've rarely, if ever, worked at a place where production didn't come down to dev at some point to stabilize drift. Anywhere that was a concern, drift was outright avoided by enforcing the chain of deployment.

- Craig Farrell

Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.

For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA

Gary Varga SSC Guru Points: 82166 More actions · Answer 9

Evil Kraig F (11/24/2014)
Most places I've worked at avoid drift by doing a development refresh from production (with necessary scrambling of identifiers/CC#s/etc, if necessary). At worst, a sprint is only a few changes off from the production version, which is corrected by the next refresh.
I've rarely, if ever, worked at a place where production didn't come down to dev at some point to stabilize drift. Anywhere that was a concern, drift was outright avoided by enforcing the chain of deployment.

Please tell us that CC #s are not held as plain text!!!

Gaz

-- Stop your grinnin' and drop your linen...they're everywhere!!!

Jeff Moden SSC Guru Points: 1004355 More actions · Answer 10

Yet Another DBA (11/24/2014)
Drift?? More like herding cats!!!

BWAAA_HAAAA!!!! (Sorry... did that come out loud?) 😀

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

Eric M Russell SSC Guru Points: 125561 More actions · Answer 11

Yes, DDL modification events are definately one of the alerts you want sent to your phone.

"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho