September 29, 2015 at 10:59 am
I have a rogue developer I need to stop. Bottom line. I work for a small company with several metric tons of valuable, revenue generating data. We have a development environment, choc full of db's that are (mostly) carbon copies of the Production environment. Prior to my arrival a couple years back, there was zero development structure. Zippo. The individual I'm referring to is a holdover from that environment. Our other Achilles Heal is that because it grew out of only 2 "developers", everyone, I mean everyone, knows the admin password so there are no profiles, policies, restrictions, etc.
We (us more formally trained and experienced people) have said repeatedly not to develop in the production environment. Today, we get an email that a new Stored Proc was "released" to the production environment. All testing was done by the "developer", no one else. When asked about it, the reply was "it's just a stored proc that executes a bunch of Select Statements". It has variables, and an Update statement that was tested in the Development world, but regardless.
My beef is that a Stored Proc was created in the PRoduction environment, regardless of whether it had one line of code, or 1000, it's a database object that has no testing basis or roots in the development environment. There are TONS of blogs out there... but I can't seem to locate anything Microsoft, Per Se, about best practices.
Also, the "developer" hasn't done the documentation yet. Nor was there any "ticket" (we use JIRA), but the developer says "This is only 1 part of the process, when we actually start putting some documentation in Confluence, I’d be happy to demo/answer any questions "
Am I missing something here? Everyone else around here seems to understand that what's happening isn't good, nor correct. Any input with links to some "official" stuff would be greatly appreciate. I'm trying to appear impartial; however, I'm getting really tired of dealing with this.
September 29, 2015 at 11:38 am
I'd start yanking that developer's access to anything and everything I could keep them out of. Then, when they go to management, you go in with a set of documented breaches. The thing is, I feel your pain. I worked for an organization that made everyone 'sa'. It was extremely painful and I only lasted about 9 months before I quit.
Microsoft doesn't really have documentation on the proper ways to do database development. It's the kind of guidance I would have loved to have received from them for years and years. It just doesn't happen. So, you're limited to other resources. You can't get away from that fact. One set of documentation I can point you to, other than blogs, is all the information that Redgate has online in support of our tools and training offerings. Here's our DLM documentation[/url]. I'm not sure we have a single document that says, outright, what a stupid practice it is to develop in production.
What about legal compliance? Are you guys dealing with medical data or are you publicly traded? In either case, you're unlikely to survive an audit with uncontrolled access to production.
Let me know what else I can do to help out. This is one of my personal crusades.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
September 29, 2015 at 12:01 pm
Until you have an issue, you can't argue anything. If the dev does things and they work, then ignore it. Track things.
There isn't anything that I've seen other than blogs that says "don't develop in production." There are things like this: https://msdn.microsoft.com/en-us/library/cc296714%28v=bts.10%29.aspx
I've dealt with this, and the only way I change things is to get a manager to buy in, and that only happens when there are pain points. Once that's done, I immediately change all prod passwords, including app passwords that I don't deploy to developers. If the pwd is in the code, I need to know where, which config file, and then I change it, setting the file folder to allow only Ops people access (and services).
From here, you start to require some testing or other docs, but don't be vindictive. This isn't about making the process harder, but about preventing issues. You still want to maintain velocity from developers as much as possible.
October 2, 2015 at 9:25 am
an update to this:
The exec staff got involved in this. Yet another proc showed up that "just did xyz" and they decided they'd had enough. The person in question was given a directive to NEVER push anything to production again unless specifically told it could be done by someone that has QC'd the work first.
October 2, 2015 at 9:41 am
CptCrusty1 (10/2/2015)
an update to this:The exec staff got involved in this. Yet another proc showed up that "just did xyz" and they decided they'd had enough. The person in question was given a directive to NEVER push anything to production again unless specifically told it could be done by someone that has QC'd the work first.
It seems that this would be a good time to explore locking down the production environment. Ideally, developers have no need to access production. They might need periodic access to read data, but that should only be granted when investigating an issue, and then it should be promptly removed when no longer needed.
Wayne
Microsoft Certified Master: SQL Server 2008
Author - SQL Server T-SQL Recipes
October 2, 2015 at 12:49 pm
WayneS (10/2/2015)
It seems that this would be a good time to explore locking down the production environment. Ideally, developers have no need to access production. They might need periodic access to read data, but that should only be granted when investigating an issue, and then it should be promptly removed when no longer needed.
+1
Viewing 6 posts - 1 through 5 (of 5 total)
You must be logged in to reply to this topic. Login to reply