June 25, 2009 at 9:13 am
I am still a newbie in terms of moving to agile development. I sort of get it but I'm not 100% won over yet.
Some of the practises jar with my previous experience and what seems to be good agile practise has all the hallmarks of an accident waiting to happen.
I would like to know what other people's experience has been in moving to agile methodologies.
What was good?
What was bad?
What caused problems?
How were those problems addressed?
Was there a culture shock?
How did DBAs react to agile?
June 25, 2009 at 10:06 am
We have been using Scrum for a couple of years now to manage our Agile development
We were all a bit reluctant to change the way that we were working, but after a lot of fine-tuning we now have an effective way of working that solves a lot of problems that we had in the past, the main issue we had was that Project managers were used to giving projects directly to developer whenever they felt like it, this caused problems as the developer found it hard to plan their personal work schedule and their day was at the whim of the PM.
Now with SCRUM we can tell the PM to talk to our Technical owner who will put the task on the backlog for us. While these sort of things could have been solved without agile it is nice to have a structure that everyone has to work by, not just developers but also Testers, DBAs and most importantly PMs.
There were a lot of teething problems and we have modified the SCRUM method a little to our way of working and it is going well, it is more the fact that we now have A methodology, rather than work thrown at us, that has helped the most.
June 25, 2009 at 11:25 am
It's been a very mixed bag around here. Some of the general principals have survived for a couple of years, daily stand-up, short term planning, precise definition for those short term plans... But a lot of the overall approach has ended up tossed. Unfortunately the willingness to embrace change basically ended up causing complete rewrites of the entire code base once every six weeks or so as someone found a better way to do things or the business changed their mind about a requirement. Most of the teams that were doing full-on, whole hog agile methodologies have largely abandoned them now.
It was especially hard on database development. We just couldn't completely redesign databases every time a developer decided he had found the new road to Nirvan. Ultimately this was the root cause for at least one of our teams adopting nHibernate and another couple of switching over to Microsoft Dynamics. They really want to change their minds, on the fly, without having to deal with databases and DBA's. They have yet to go to production, so I'm not sure what they're going to do when they realize it wasn't us that was the root of the problems, but the data, that has a tendency to stick around, no matter how great an idea one of the developers had about redesigning the entire structure.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
June 26, 2009 at 4:35 am
We have used agile methods for creating a prototype to obtain user feedback when faced with a vague specification. (Some basic OO analysis was also done - use cases, base classes, a few state/sequence diagrams etc). It worked well for teasing out the requirements but we then chickened out, having similar reservations to David, and reverted to a more traditional approach.
Grant makes a good point about keeping up with database changes. If we did another agile prototype I would be inclined to use some sort of object persistence layer like Hibernate. Given that data integrity is usually important, I would be loathed to use Hibernate in a production system as a well designed schema, with plenty of constraints and a stored procedure interface, acts as a reality check on what the program is doing.
June 26, 2009 at 2:57 pm
We have an experienced architect who has worked with agile development for a number of years.
He has some pearls of wisdom
1. It takes 18-24 months for the agile methodologies to bed down.
2. If the business want to change everything then they have to measure this requirement by looking at the cost vs benefit. If they decide they want to go ahead then they have to accept the costs. We all know it takes longer and costs more to change something but if the business judge this to be worthwhile there is job security.
3. It is the TEAM that deliver software, not just the DBAs. There has to be communication and compromise on both sides and sometimes the DBAs have to lower the quality bar. At the moment this sounds absolutely horrible as the whole point of a DBA is to raise the bar and be ruthless in pursuit of performance and quality.
4. Adopt all the principles of agile or none of them. Drop one principle and there is a domino effect that takes the whole lot down.
5. Read Scott Ambler's book from cover to cover at least twice.
6. Communication is all and it has to go both ways.
7. There is an optimum size for a team which is somewhere between 6 and 10 members.
8. DBAs become developers with a DB specialism. Personally I like to have a broad range of skills but at present I feel that what a DBA does is underappreciated by developers.
9. Code that is of production quality that fulfils the business need and delivers the none functional requirements is good enough. You aren't looking to produce the perfect software you are looking to deliver utility to the business.
June 26, 2009 at 3:08 pm
Unless you're very careful, it becomes nothing more than an excuse for releasing bad or slow product with no documentation. Contrary to what many believe, it does take some planning and it should never be used as an excuse to not test or not document.
--Jeff Moden
Change is inevitable... Change for the better is not.
June 27, 2009 at 2:13 pm
David.Poole (6/26/2009)
We have an experienced architect who has worked with agile development for a number of years.He has some pearls of wisdom
Uh huh. I want to be clear that what follows is not directed at you David, but at any experienced architect that would say things like this..
2. If the business want to change everything then they have to measure this requirement by looking at the cost vs benefit. If they decide they want to go ahead then they have to accept the costs. We all know it takes longer and costs more to change something but if the business judge this to be worthwhile there is job security.
There is no wisdom in this that we have not had for over forty years (see Fred Brooks, Mythical Man-Month). What the new project methodologies (of which Agile is the darling for the moment) bring to the table is an almost total capitulation by the developers and project management of their role in this dialog by failing to take responsibilty for estimates that are more than 3-6 weeks out. And without accurate and responsible estimates telling them that their decision will add 18-24 months, the business cannot make accurate business decisions about their own requests and eventually will refuse to take responsibilty by first rejecting the project and ultimately the whole methodology.
3. It is the TEAM that deliver software, not just the DBAs. There has to be communication and compromise on both sides and sometimes the DBAs have to lower the quality bar. At the moment this sounds absolutely horrible as the whole point of a DBA is to raise the bar and be ruthless in pursuit of performance and quality.
You are right David, this DOES sound horrible. This is hardly the first time I have heard someone say "We have to communicate and compromise and sometimes you will have to concede your standards.". You might notice that there are a few things missing from that statement and after my second or third time sitting in a large meeting where some fair-hared wog lectured us with these almost identical words I came to the conclusion that the omissions were not an accident. Because by "communicate" they meant that they would tell us what we should do, not that they were obligated to listen to us. And by "compromise" they meant us, not them. And by "concede" or "lower", they again meant us, not them. And I mean this irrespective of what our relative roles were, not just Development to DBAs.
5. Read Scott Ambler's book from cover to cover at least twice.
OR.. they could spend some time listening to the DBA that they claim is part of their "TEAM". At least once.
6. Communication is all and it has to go both ways.
Agreed, it should. It does not usually, but it should.
8. DBAs become developers with a DB specialism. Personally I like to have a broad range of skills but at present I feel that what a DBA does is underappreciated by developers.
This conflates the role of the DBA with the Database Developer. Although the cross-over between them can be high, the job description and business responsibilities are very different. What's even more disturbing is the implication that an experienced architect has so little understanding of the DBA's role.
9. Code that is of production quality that fulfils the business need and delivers the none functional requirements is good enough. You aren't looking to produce the perfect software you are looking to deliver utility to the business.
The word "none" is a typo, right? Otherwise this is just too easy. 🙂
[font="Times New Roman"]-- RBarryYoung[/font], [font="Times New Roman"] (302)375-0451[/font] blog: MovingSQL.com, Twitter: @RBarryYoung[font="Arial Black"]
Proactive Performance Solutions, Inc. [/font][font="Verdana"] "Performance is our middle name."[/font]
June 27, 2009 at 3:32 pm
Thanks, you have just confirmed my worst nightmares and I have to say I'm not altogether surprised.
By none functional requirements I meant things like "
a. must support x queries per second
b. must be scaleable
c. must be secure
...etc
June 28, 2009 at 5:39 am
David.Poole (6/26/2009)
We have an experienced architect who has worked with agile development for a number of years.He has some pearls of wisdom
1. It takes 18-24 months for the agile methodologies to bed down.
2. If the business want to change everything then they have to measure this requirement by looking at the cost vs benefit. If they decide they want to go ahead then they have to accept the costs. We all know it takes longer and costs more to change something but if the business judge this to be worthwhile there is job security.
Pretty much agree with all this, but it really is incumbent on the leaders of development to accurately communicate the cost to the business. Far too little of that is done in my opinion.
3. It is the TEAM that deliver software, not just the DBAs. There has to be communication and compromise on both sides and sometimes the DBAs have to lower the quality bar. At the moment this sounds absolutely horrible as the whole point of a DBA is to raise the bar and be ruthless in pursuit of performance and quality.
Absolutely. A team. But... I'd like to rephrase it just slightly. It is the TEAM that delivers software, not just the developers. I realize those guys write the code. Heck, I used to write the code. But if you're storing and retrieving data from a relational database system, you might want to work with and listen to the guys who are putting their efforts into understanding how that system works. I could care less if you call it persistance in an effort to make it so the concept of a database is eliminatd. You're still storing the data on an RDBMS. Wether you call it persistance layer or database, the strengths and weaknesses of that RDBMS haven't changed.
Maybe, just maybe, getting a better concept of what the term team really means should be the very first lesson of agile development, but it isn't really there.
4. Adopt all the principles of agile or none of them. Drop one principle and there is a domino effect that takes the whole lot down.
5. Read Scott Ambler's book from cover to cover at least twice.
Yep. Did that. Actually read a couple of Scott Ambler's books. It just doesn't seem to change the fact that data persistance really gets in the way of rearchitecting a system every six weeks.
6. Communication is all and it has to go both ways.
That would be really, really nice.
7. There is an optimum size for a team which is somewhere between 6 and 10 members.
8. DBAs become developers with a DB specialism. Personally I like to have a broad range of skills but at present I feel that what a DBA does is underappreciated by developers.
Yeah, I have a bit of a hard time buying this one. I worked with a developer that keeps telling me, for the last four years, that next year, my job is going away. I just don't see it. I agree that you need a DBA that can develop. I just don't see it as a developer with database specialty. There's just too much to learn on the database side of things to place the focus in that way.
9. Code that is of production quality that fulfils the business need and delivers the none functional requirements is good enough. You aren't looking to produce the perfect software you are looking to deliver utility to the business.
By and large, I do agree with the approaches around agile development. I just have yet to see them done in a way that produces quality product. By quality, I don't mean perfect, I mean functional. It does what the business wants & needs in a timely enough manner, without hurting the business.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
June 28, 2009 at 12:55 pm
Grant Fritchey (6/28/2009)
7. There is an optimum size for a team which is somewhere between 6 and 10 members.
8. DBAs become developers with a DB specialism. Personally I like to have a broad range of skills but at present I feel that what a DBA does is underappreciated by developers.
Yeah, I have a bit of a hard time buying this one. I worked with a developer that keeps telling me, for the last four years, that next year, my job is going away. I just don't see it. I agree that you need a DBA that can develop. I just don't see it as a developer with database specialty. There's just too much to learn on the database side of things to place the focus in that way.
"We don't/won't need DBAs." Wow, have I ever heard that a lot in the last 30 years. But everyone who said it has been wrong so far. And the reason is simple, not only does it require a specialized set of skills, as Grant said, but corporations are also too dependent on their data to take risks with it.
And unlike software, databases (indeed all dynamic data storage) require maintenance to continue safe operation. Sure software needs maintenance too, but in truth only to do new things, things that it did not previously do, but if you want you can always choose to use it without any maintenance (new features, new supported configurations, new bugfixes, etc.). This is NOT true of databases, because, unlike software, the very act of using it, changes it, and that means that it must be maintained and managed.
I have known many cases of a critical SW application that was used for years after the source code was lost or destroyed, the vendor disappeared, the compilers were lost, etc., and all ability to modify and maintain the software was long gone. But I have never know a case where an actively used, writable database stopped needing maintenance at some point. So trust me, programmers are going to disappear long before DBAs.
[font="Times New Roman"]-- RBarryYoung[/font], [font="Times New Roman"] (302)375-0451[/font] blog: MovingSQL.com, Twitter: @RBarryYoung[font="Arial Black"]
Proactive Performance Solutions, Inc. [/font][font="Verdana"] "Performance is our middle name."[/font]
June 28, 2009 at 7:36 pm
I have a hard time understanding how agile development would work within the biggest trend in software development, outsourcing development to offshore teams that have little direct involvement with the business.
When your developer is half a world away, doesn’t work the same hours as the business, has limited ability to interact with other team and business users, and does not have much experience, then I think you have the exact opposite of agile development.
Although we can see the obvious disadvantages of doing development this way, the highest level of the business may only care that the hourly rate for offshore developers is one-third the in-house rate.
June 28, 2009 at 8:47 pm
Michael Valentine Jones (6/28/2009)
I have a hard time understanding how agile development would work within the biggest trend in software development, outsourcing development to offshore teams that have little direct involvement with the business.When your developer is half a world away, doesn’t work the same hours as the business, has limited ability to interact with other team and business users, and does not have much experience, then I think you have the exact opposite of agile development.
Although we can see the obvious disadvantages of doing development this way, the highest level of the business may only care that the hourly rate for offshore developers is one-third the in-house rate.
What is even harder to understand is that they don't realize it is actually costing them more when the development time is 2 to 3 times longer plus the additional work that still has to be done locally before the app goes live or is ready for market.
June 29, 2009 at 4:20 pm
The Agile approach does seem to cause some discussion and it seems very clear having tried it on a number of occasions that yes, there is a difference in approach between application and data development. Personally (like a lot of DBA's), I've done most - software developer, lead developer, applications manager and software company owner in my spare time. I now find myself as a full time DBA working with a number of developers who are more than happy with the Agile approach. Firstly, it seems a good thing to open up communication between the development side and the business, but Agile seems a tad quick to pass the buck back to the business for poor choices about what they actually want (the most obvious problem in development is finding out what they want in the first place). It can easily become a stick to beat them with if they decide on poor features. They don't know what's good or bad, just are concerned with other aspects etc...
Agile offers a continued stream of work - OK, that's a good thing. Better communication, yep - happy with that. Better software ? I've not seen enough yet to be happy with this if I'm honest. What I have seen could be a lot better.
Inception = waste of time
Elaboration = waste of time
Construction = the opportunity to rush headlong missing the bigger picture if Agile is approached badly.
It can work when done right though.
Opening up the DB development side to those less used to large scale systems is a spectacular way of making a mess. With Agile, it won't be as apparent as each story will have been answered as needed. When you look back....Oops, that'll be a bad system design. OK, we should get the opportunity to 're-factor', but I seriously doubt it will be enough to make it right.
The biggest change to make is the perception of what a good or exceptional DBA does as well as a perceived bottleneck and the assumption that DBA's want to model everything first. Not really, I just want to put in what I've spent 15 years learning to do right (and what you are paying me a lot for).
Generally, developers tend to assume that DBA's don't do too much other than complain and get too pedantic about the detail. When we talk of highly concurrent solutions, the detail is an absolute must. They assume DBA's haven't developed software (as we assume similar about them 🙂 ), so there becomes a split. To get an Agile (read iterative) approach working, we must break down the perception of what we do. If you see us, something is broken and likely to be fixed quickly. There is a likeness to airline pilots in some respects.
The DBA needs to be sympathetic to the developers and work with them to get the best out of this. Unique to agile ? Not really.
Somethings just need re-iterating however we approach the amount of work to be done in a given time.
Viewing 13 posts - 1 through 12 (of 12 total)
You must be logged in to reply to this topic. Login to reply