The Time to Patch

Question

The Time to Patch

Steve Jones - SSC Editor

SSC Guru

Points: 736105
More actions
January 3, 2020 at 12:00 am

#3707183

Comments posted to this topic are about the item The Time to Patch

Viewing 10 posts - 1 through 9 (of 9 total)

You must be logged in to reply to this topic. Login to reply

Jeff Moden SSC Guru Points: 1004299 More actions · Answer 1

Considering the absolute idiocy that occurred in SQL Server 2012 until the finally fixed {almost} everything they broke in the SPs and CUs prior to SP3 and the SSIS nightmares that some poor people had to go through with 2014 SP1, and going all the way back to SQL Server 2000 SP3, I can't blame anyone for seriously delaying such updates, never mind going "evergreen". It's another one of those things that scares the hell out of me about the cloud.

Our general schedule is that we download and make ready the Windows and SQL Server updates (we "only" install the SQL Server updates if they have security patches in them... which is a lot, unless there's a particular repair that I determine that we need, which is also a lot {Major face-palm}). We deploy those changes to the Dev boxes the following Thursday. We deploy to the staging boxes the following Thursday if nothing went wrong on Dev (we have had to rollback one the past that broke 3rd party software). We then wait two Sundays to deploy to prod if nothing went wrong in Staging.

By the time we get the patches ready to deploy to prod, a lot of the rest of the world or MS itself has done a final shakedown and either issued a patch of the patch or the original has been proven good.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

David.Poole SSC Guru Points: 75996 More actions · Answer 2

Whatever your patching strategy I would hope people are moving to

Scripted deployments for server installs, service packs and patches
A large battery of mechanical tests to verify both their database code and their applications function as expected.
A catalogue of servers with version numbers harvested from the servers themselves.

A number of the apps I work with are built by Grade. There is a section in the properties file that lists all the plugins and their versions. This can be harvested to tell us what version of every library, framework and app is running across the enterprise. If you design your install and patch scripts carefully you can design explicitly to allow the patch levels to be harvested.

LinkedIn Profile

Jo Pattyn SSC-Dedicated Points: 32342 More actions · Answer 3

We try to implement them every 2 months. First a testing period of 1 month in dev and acceptance. So far the updates didn't break anything

Summer90 SSC-Dedicated Points: 32891 More actions · Answer 4

Cannot until the software vendor certifies the CUs. They are horrible about this. If they say you apply it and we won't support you we are stuck.

I have seen several CUs that 2-3 weeks after they release them that they withdraw or release a newer revised version of it. So, staying current is probably not a good thing.

Steve Jones - SSC Editor SSC Guru Points: 736105 More actions · Answer 5

Vendors are a huge problem in patching. I'd like to see contracts explicitly state they'll certify within xx months, with xx being < 3.

Jeff Moden SSC Guru Points: 1004299 More actions · Answer 6

Steve Jones - SSC Editor wrote:

Vendors are a huge problem in patching. I'd like to see contracts explicitly state they'll certify within xx months, with xx being < 3.

That's not easy to do. Look at ANY Microsoft EULA. They take zero responsibility for anything they do even if it takes one of your servers down (2014, initial release of SP1) or destroys your data (2012, Online Rebuild of Clustered Indexes can corrupt the index) just to put a label on a couple of the many events.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

Steve Jones - SSC Editor SSC Guru Points: 736105 More actions · Answer 7

Except, Microsoft provides support for the CUs. A vendor should be able to test that a CU will, or will not, cause issues in their test suite. It's not perfect, but they do certify for a version of SQL . Recertifying ought to be something they can do quickly, precisely because of security concerns.

Summer90 SSC-Dedicated Points: 32891 More actions · Answer 8

The one problem with that mindset is they probably support 3-10 versions of their software due to clients not upgrading. Are they supposed to test with every single combination that could be out there? One vendor we worked with would release 3-10 minor versions of their software per year. That included bug fixes and slight product enhancements. You sometimes have companies that are 2-3 years behind so how far back must the vendor test a CU?

Steve Jones - SSC Editor SSC Guru Points: 736105 More actions · Answer 9

Every one they support. If you support it, you should be supporting it on the platforms that are out there. Most minor changes shouldn't be affected, but limiting the ability of customers to take security updates is a big deal.

Every vendor ought to have a test suite they can run. That's the price of doing business.