December 16, 2008 at 5:37 pm
Comments posted to this topic are about the item Versions of Disaster
December 17, 2008 at 4:52 am
If you are talking about software versions, do not forget about the Windows patch level, the AV control file level, etc, etc ad almost infinitum. Then we get into hardware versions - it is actually possible to obtain a working 10-year old server that has the same hardware build as the environment we are supposed to reproduce.
My understanding is that the law does not mandate us to record the exact details of the software used on our servers on any given day. Legal process may ask us to show to the best of our ability how a particular piece of software worked at a given point in time, but it does not ask us to do the impossible.
If a reconstruction of a crime, etc, is done, effort is made to use clothing, actors, etc that match as closely as known to the actual event. But very seldom does the legal process care if the the weather is a bit warmer,colder, cloudier, etc than the actual event. The same precedents have to apply in computing.
A legal adversary may try to get penalties applied if the requested environment cannot be reproduced en every detail. It is the job of your own lawers to show that reasonable care has been taken in reproducing the environment, and it is as close to the original environment as it is reasonable to be. If the opponent is not asking for every detail (including the hardware) they seriously weaken their case that you are negligent in what you have been able to do. Any judge worth their office will accept that people cannot do the impossible. If you have complied with standard practice in your industry then you have a strong defense.
You just need to convince your own lawers about all of this...
Original author: https://github.com/SQL-FineBuild/Common/wiki/ 1-click install and best practice configuration of SQL Server 2019, 2017 2016, 2014, 2012, 2008 R2, 2008 and 2005.
When I give food to the poor they call me a saint. When I ask why they are poor they call me a communist - Archbishop Hélder Câmara
December 17, 2008 at 5:03 am
Of course the version of SQL is recorded in the errorlog at startup, and in the event of an instance crash these logs will likely still be there.
Just in case as part of my DR processes I script out the server configuration, including the latest errorlog plus explicitly the SQL version using:
select convert(nvarchar(10),serverproperty('productlevel'))
select @@version
This is then offsited to another server and tape.
SQLdiag is a good little tool to run once in a while to give a good picture of the OS as well, but I do rely on the server admins to maintain doco on server builds.
---------------------------------------------------------------------
December 17, 2008 at 7:39 am
If you accept that disk space is very cheap relative to the cost and hassle of recreating your server environment, you can snap a copy as a virtual machine.
We run all our servers as virtual machines and can back them up while they're running. Copies of the backups can be handed to developers to test changes on what was, until just a day or two ago, the current production system.
Even if you don't run your server as a virtual machine, you can still make a copy of it as a VM.
However, I'm not kidding about requiring a liberal attitude about disk space. The image files are in the tens+ Gb range and sometimes much higher. Keeping track of dozens of copies requires some advance planning.
-- Craig Yellick
December 17, 2008 at 7:42 am
VM ware or some other Virtual PC. is the Key to Backware Capablity.
Make a Virtual PC Copy of the computer at Install with just App. -No Data-
Kepp Virtual PC Patched and Update the Same as the Rest fof the Servers.
Back up after Each Patch and Update Cycle.
Then when the problems come you can roll back the Virutal PC to the Time Needed.
December 17, 2008 at 9:05 am
I really don't have a great recommendation on how to handle this other than build some automated system that tracks the current build number on a daily basis, perhaps even putting it in each database.
Put it in the database? :laugh: You mean the one we CAN'T restore?
How about a central database to store all your version data and history? If you want to make it independent of SQL Server version issues try MySQL. Or :shudder: Access. Excel does not have any version problems. :Whistling: Maybe a Text file. Wait, there's all that UTF-8 and Unicode business. I know! PUNCH CARDS! Where are those 15 dozen boxes of blanks? Ahh, back of the car for weight in the snow. Everything is good for something.
Side bar: Why did they think the stand up comedian committed suicide? He killed himself.
Seriously though, reading legacy formats is, and likely always will be, a challenge. Just the other day a legacy application that has been running flawlessly for years went belly up. It threw errors talking about "linked tables". The guy working the issue asked me, "What the heck are linked tables?" The client had to dig to find the Access CD and actually install it on the machine in question. Access pointed to two different MySQL databases and some tables on the AS-400. One of the MySQL tables was corrupt and needed a table repair.
ATBCharles Kincaid
December 17, 2008 at 9:58 am
A little operational maturity goes a long way. If you're following MOF or ITIL, you're already keeping track of all these "little" details and should be able to reconstruct your environment from the logs to line up with those backups.
Having years and years of backups is a start, but...
That said, I don't think I've ever seen an operational shop (even in large enterprises) above a maturity level of ~2.5.
December 17, 2008 at 12:32 pm
This is just one reason why change control is SO important. If you have a method of testing and documenting all changes made in your enviornment then it becomes pretty simple to look at the date of the backup file and match that with the state of the server at that point in time.
/*****************
If most people are not willing to see the difficulty, this is mainly because, consciously or unconsciously, they assume that it will be they who will settle these questions for the others, and because they are convinced of their own capacity to do this. -Friedrich August von Hayek
*****************/
December 18, 2008 at 11:04 am
We have implemented an SQL Server instance inventory systems which consists of an SSIS polling mechanism and a centralized database repository. Among the many things we track on a daily basis is the SQL version and build.
Benjamin Lotter
http://BenjaminLotter.info/[/url]
http://www.LinkedIn.com/in/BenjaminLotter
Delight thyself also in the LORD and He shall give thee the desires of thine heart.
~Psalm 37:4
December 18, 2008 at 5:23 pm
Typically the version differences don't prevent the restore of a db, however they can affect the functionality.
I used to track this daily, mainly to make sure devs and others didn't change boxes without permission
January 13, 2009 at 11:52 am
Isn't this what change management requests are for. To keep track of what and when things are being done. Yes the data might be on the system that crashed that is why a hard copy is kept is a safe location. Keeping up with documentation is the worst part of the job. However, it is also one of the major parts of the job.
January 13, 2009 at 12:01 pm
I read a story about a shop that kept point-in-time restore data for all applications on all servers, throughout their history. They could restore any retired server, into a virtual environment, exactly as it had been on the particular day they needed.
One day, this became crucial for legal reasons, and they restored an old server into a virtual environment, with the databases and so on exactly as they had been on the critical day. And nobody could remember what the admin passwords had been, so the data might as well have not existed.
True or not, it was an amusing read. If true, I'm sure it was not so amusing to the people involved.
There are easy ways around that issue in SQL Server, of course.
As for keeping server version data, that's easy enough to do, but it isn't quite automatic. Good point, since it could very well matter when doing a restore.
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
September 15, 2013 at 6:45 am
Getting too paranoid about installing updates in case you have to reconstruct some time in the future is not a good idea; it will completely destroy any development and release method that involes incremental release and/or incremental deployment and/or incremental maintenance. As someone else alrady pointed out, courts will usually be sympathetic if you can show that have followed acknowleged (industry standard) best practice and done your best if something needs to be reconstructed for legal reasons and you can't do it successfully. Using ITIL or something similar will ensure that you have records of what version was where when, because you need them for more than just legal reasons. Those records should include things like passwords or a copy of your password safe or equivalent (old passphrases for that safe should be kept in a secure physical safe - if you discard them you are probably dead in the water), encryption keys (only those held in password-protected format outside the database), platform patch state, and application patch state as well as stuff about the hardware modification state; the software components installed (platform, middleware, and application) should be kept in there installed forms along with all updates. That will mean that you can always get back to a past point in time, provided you can produce or simulate the required hardware.
Very few shops indeed actually do this. None that I've ever worked in did it properly, but in many cases QA of software changes included checks that behaviour of existing functionality with old data was no different betwen old and new versions. Of course such checks will rarely be 100% comprehensive, but they can in almost all cases be thorough enough to provide a 99.999% certainty of no change. Then there are changes part of whose purpose is to change the behaviour of existing functionality with old data (as opposed to adding separate new functionality) and there version records are indispensible. But again they aren't always kept for more than a year or two, so in real life most people are not prepared for a sudden requirement to reproduce some ancient history. I suspect that while the editorail will make people think about this, and make some people who are already trying to have the reversion capability think hard enough that they improve their methods, it will have little or no lasting effect on the majority of people who just can't be bothered with a requirement to recover anything from more than a couple of months back (obviously people in some industries can't legally be part of that majority).
Tom
Viewing 13 posts - 1 through 12 (of 12 total)
You must be logged in to reply to this topic. Login to reply