May 17, 2011 at 1:30 pm
RyderD (5/17/2011)
I'm not actually disagreeing that developers who are supporting production enviornments need access to a version of production data. Of course they do - there, in fact, is really no other way to help diagnose and fix some problems.
Usually your sprint/cycle/whatever developers are also the ones doing production support, so you end up in the mixed bag.
I just think the data ONLY should be refreshed - not an entire database restore - ESPECIALLY if there are security concerns and if there are separate servers involved.
Yes and no. A data only refresh in the middle of a development cycle can actually throw you off during root cause location.
Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.
For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA
May 17, 2011 at 1:44 pm
And a full database restore in the middle of a development cycle wouldn't throw things off? I can't tell you how many times I've lost users, roles & permissions, breaking unit or UAT due to that method of "refresh".
Or are we just talking about different things now, meaning that developers and production support people should be in completely different sandboxes/playgrounds?
I can see where that would be beneficial to some degree.
Though then development developers don't ever get to grow from the fun that can only be brought upon during production support development.
Have you ever seen a guy design something that sounded good on "paper" and seemed like a good idea at the time, the cycle moves along, and so the design person moves on to the next project before rollout and implementation of the original, designing the same flaw over and over again that other people have to live with for years?
Nah. Never happens.
May 17, 2011 at 2:17 pm
RyderD (5/17/2011)
And a full database restore in the middle of a development cycle wouldn't throw things off? I can't tell you how many times I've lost users, roles & permissions, breaking unit or UAT due to that method of "refresh".Or are we just talking about different things now, meaning that developers and production support people should be in completely different sandboxes/playgrounds?
I can see where that would be beneficial to some degree.
Though then development developers don't ever get to grow from the fun that can only be brought upon during production support development.
Have you ever seen a guy design something that sounded good on "paper" and seemed like a good idea at the time, the cycle moves along, and so the design person moves on to the next project before rollout and implementation of the original, designing the same flaw over and over again that other people have to live with for years?
Nah. Never happens.
Heheh, oh, I don't disagree, and re-rolling the DB changes for that version of the cycle is a pita too. In most cases, if they have the drive space, you usually end up with two DBs. One's a development version, another is a prod copy. Yes, roles, permissions, and the like can be annoying. You can also add some real confusion and match different login's sids on the servers. IE: A dev login and a prod login ending up with the same sid that's used at the DB level so they act the same but you control the login levels... things like that.
But yes, there's pros and cons to it, thus why I said Yes and No. 🙂 I'm currently in a continuous integration environment (which is a whole 'nother set of complaints on my side) so they don't hurt as badly. However, you're right, data refreshing is usually the preferred method. It depends on how much dev work is going on in any dev level db at one time though. Losing your changes is annoying, at best.
Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.
For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA
May 17, 2011 at 2:24 pm
Eeep. I just saw how old this thread is. Zombie thread! YEAH!
Sorry 'bout that. And thank you for having the conversation.
May 17, 2011 at 2:53 pm
RyderD (5/17/2011)
Eeep. I just saw how old this thread is. Zombie thread! YEAH!Sorry 'bout that. And thank you for having the conversation.
Bah, no worries about that. I'm not the anti-necro police. Usually only an issue if you're intending to try to carry the conversation on with the OP or someone else that responded a while back. The topic is still as viable and as debateable today as it was 2 years ago.
Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.
For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA
May 19, 2011 at 4:58 pm
Steve Jones - SSC Editor (5/13/2009)
:), whoops.not implying developers are not people...
I've worked with some developers that I am pretty certain were not people. :alien:
:hehe:
LC
May 19, 2011 at 5:35 pm
After having read every post in this thread, I have to say that I pretty much agree and maybe even somewhat disagree with all of you. As a software developer for almost 25 years, before I exclusively became a DBA, I worked for a lot of companies and I saw most of the problems you've described at one time or another.
As for the best way to develop software, (as Jeff Moen (spelling?) often says), "It depends." The best way to develop software depends on many variables. There is no one best way, no one best group of people, no one best methodology. Everything has to be tailored to the environment, circumstances, people involved, corporate culture, nature of business, and the project requirements.
These statements include the test data and the methodologies used for testing.
There are known best practices and they are the starting point. But, honestly, how many of us have ever seen a company adhere to every known best practice in the development processes we've lived through? Companies cut corners to save costs/time. It's reality. So, knowing that, how can they ensure that they deliver a quality result, on time, and on budget?
The answer is finding BIG problems early in the development cycle. From my experience, and this is strictly my experience, I'd say maybe half the people in software development are lacking essential skills. I've seen many developers who could write an algorithm to meet a spec, have it pass functional unit tests, have it pass integration tests, and fail miserably in performance tests. They simply could not, would not, had no conception of, writing efficient code.
The cure for this problem is not to wait until the end of the development cycle to see if it all works efficiently. As jparra pointed out, this approach is really expensive. I saw this end-of-the-process approach cost a Fortune 70 company a year and a million dollars to rectify.
In MY experience, the only data that is truly effective for testing, after unit tests are performed, is production data. I say this for 2 reasons:
1. SIZE. A lot of software performs well when data sizes are small, but it's lousy when data sizes are very large. These problems need to be discovered early in the development cycle.
2. DIRTY DATA. In most of my career, the data I've had to work with was dirty, very dirty. The exceptions caught in software by dirty data need to be caught early in the development process. I have found that it is just not possible to dream up every possible dirty data condition for test data.
Should the data be scrubbed? Definitely.
Should there be a set of data for regression testing? Definitely.
But ultimately, production data is the real test for software and the people that develop it.
Anyway, my 2 cents.
LC
May 20, 2011 at 8:13 am
Lee Crain (5/19/2011)
After having read every post in this thread, I have to say that I pretty much agree and maybe even somewhat disagree with all of you. As a software developer for almost 25 years, before I exclusively became a DBA, I worked for a lot of companies and I saw most of the problems you've described at one time or another.As for the best way to develop software, (as Jeff Moen (spelling?) often says), "It depends." The best way to develop software depends on many variables. There is no one best way, no one best group of people, no one best methodology. Everything has to be tailored to the environment, circumstances, people involved, corporate culture, nature of business, and the project requirements.
These statements include the test data and the methodologies used for testing.
There are known best practices and they are the starting point. But, honestly, how many of us have ever seen a company adhere to every known best practice in the development processes we've lived through? Companies cut corners to save costs/time. It's reality. So, knowing that, how can they ensure that they deliver a quality result, on time, and on budget?
The answer is finding BIG problems early in the development cycle. From my experience, and this is strictly my experience, I'd say maybe half the people in software development are lacking essential skills. I've seen many developers who could write an algorithm to meet a spec, have it pass functional unit tests, have it pass integration tests, and fail miserably in performance tests. They simply could not, would not, had no conception of, writing efficient code.
The cure for this problem is not to wait until the end of the development cycle to see if it all works efficiently. As jparra pointed out, this approach is really expensive. I saw this end-of-the-process approach cost a Fortune 70 company a year and a million dollars to rectify.
In MY experience, the only data that is truly effective for testing, after unit tests are performed, is production data. I say this for 2 reasons:
1. SIZE. A lot of software performs well when data sizes are small, but it's lousy when data sizes are very large. These problems need to be discovered early in the development cycle.
2. DIRTY DATA. In most of my career, the data I've had to work with was dirty, very dirty. The exceptions caught in software by dirty data need to be caught early in the development process. I have found that it is just not possible to dream up every possible dirty data condition for test data.
Should the data be scrubbed? Definitely.
Should there be a set of data for regression testing? Definitely.
But ultimately, production data is the real test for software and the people that develop it.
Anyway, my 2 cents.
LC
Lee, I agree 100% with your post and have had the same experience in my 20-ish years in this world - beginning as a data entry clerk and moving through various software/development positions as well as DBA duties.
I think that sometimes DBA's hear "we need production data for testing and development" as "give us access to the production database server" and balk - which I can relate to.
You are so right that each and every environment is a little bit different.
Your post brought up some other thoughts for me:
If the team has the luxury of using a development database server that is secure, there is little need for "scrubbing" or modification of any kind when talking about security or sensitive data.
More issues come in when we try to rig a develpment environment on the developers' PC (for example) or some such. Developers are then set up to fail, as they are working locally and do not understand the additional complications when the time comes to deploy to the server. Even just a different database name on the same server leads to unforseen issues for the new guy coming in.
Other random thoughts:
I'm always astonished when new folks come in who either can't type (HOW do you write code for a living and not be able to type?) or know to set up their tools to save scripts as they go.
When the time comes to deliver to either testing or production, to "put it all together" for deployment, life is much simpler if you've already saved all of your changes as SQL scripts (assuming you used the GUI during development) and can then go through and edit those scripts in to one (or however many) deployment output.
I have a colleague who remarks often at how lazy I'm allowed to be - in his day, you had to write the code correctly the first time every time, or an entire day was lost while waiting for compilation and deployment to the mainframe. I love him. He keeps the punch cards around to remind us.
🙂
Viewing 8 posts - 31 through 37 (of 37 total)
You must be logged in to reply to this topic. Login to reply