September 5, 2013 at 8:23 pm
Comments posted to this topic are about the item Developer Pressure
September 6, 2013 at 1:26 am
Continuous integration is a no-brainer. JFDI.
Continuous deployment makes me nervous from a DBA perspective.
If a deployment means a change to a big table and replication is involved then it gives me the screaming willies.
Where there are small changes to the DB I'm much less worried. It all depends on what the change might be and therein lies the rub.
There is also consideration for the data rollback strategy.
How does a team of developers decide where on the spectrum of worry the deployment will put the DBA?
Ideally I'd like to be able to support continuous delivery with the caveats mentioned earlier.
I don't think a Continuous Delivery for non-data deployments is conducive to a good developer/DBA relationship. No matter how legitemate DBAs will be seen as blockers not guardians.
In terms of non-DB developers continous delivery eliminates a layer of bureaucracy. Frankly the traditional release process is stressful. I've seen a project manager in tears when facing a remedial release. Sometimes project managers need a hug:ermm:
September 6, 2013 at 3:52 am
While I understand the other potential problems, in general I'm all for CI/CD. I find it so annoying to have to re-visit and re-familiarise with something that was 'finished' 2 or 3 months ago because of the delays due to build/test/release schedules.
September 6, 2013 at 5:58 am
It depends on how intermingled the changes are. If the applications are siloed, you can go fast.
If they depend on or update each other, it needs more than simple automated testing.
A developer makes a simple change that works for itself, but some other application that has not been changed, gets broken. Or worse 2 developers make changes that go in opposite directions.
That said, having long cycles that leave trivial updates waiting for months is not acceptable.
"FigThing Irish" on the web site should not wait for next annual refresh.
September 6, 2013 at 6:00 am
We also use Deployment Manger for our software...
Ha! Couldn't help it. You have Development Manager in the email. Might want to fix the typo on the web.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
September 6, 2013 at 6:44 am
I'm all for CI. We're trying to crack that nut in a complex system with many moving parts. One thing you mentioned is that customers may expect new features sooner or faster. A concern I have, that isn't necessarily from CI, is if people start expecting things to happen faster then these new features aren't as well thought out and planned. CI doesn't preclude the need for business planning and development planning. So, I like the tool for the benefits it can give while keeping an eye on the actually solutions to the business needs.
September 6, 2013 at 7:38 am
Kind of a mixed situation. Probably helpful for in house applications.
For published apps, though, it can be a support nightmare. People are concerned about Android's fragmentation, but with each custiomer potentially having a different 'version' depending on their last install/update....
...
-- FORTRAN manual for Xerox Computers --
September 6, 2013 at 7:46 am
Grant Fritchey (9/6/2013)
We also use Deployment Manger for our software...Ha! Couldn't help it. You have Development Manager in the email. Might want to fix the typo on the web.
Arrgghhh, fixed. I think I mixed up the terms in the recording as well.
September 6, 2013 at 8:24 am
I'm all for CI/CD because they build development departments that have a functional and well understood deployment process--and a functional and well understood rollback process for the occasional oops. This, in the end, reduces stress because you have good testing processes for releases and everyone has 'been there, done that' regularly so any given release is not such a big deal.
That said, getting any group to that point is stressful. Building the necessary testing and rollback process as well as getting the wider community of users, testers, SMEs, etc. all folded into the process properly takes time and inevitably involves a few painful experiences. It is best accomplished at a modest pace and with a good sense of humor. The benefit of traversing this painful phase can be huge, though.
Case in point, I'm on a contract where my work from almost a year ago has not been implemented--largely because everyone here is terrified of touching production, aka the production house of cards. Of course, by now it is very hard to remember exactly what the changes are or how best to implement them due to the impact of any other changes that may have transpired since I wrote the migration scripts. There will need to be a big 're-learning' curve before this work can migrate beyond Dev. In the end, this re-scripting and re-testing is wasted work and a useless delay.
September 6, 2013 at 2:32 pm
Does it require more from the developer? Yes! Is there more stress? Yes. Does the developer get paid more for more work? No! Does the company have to hire more skilled people to develop? Yes. Is that really going to happen and this is going to work? I'll leave that up to you to answer.
We have to remember the conversation about You can have things fast good or cheap but you have to pick just two. You do not get all three. That isn't going to happen. So this rapid development and deployment technique requires the code to be developed fast, and the user requires that it be good or excellent.
So Fast + Good/Excellent = Expensive.
Now management and the company shareholders want things cheap. This helps build a better bottom line. So their approach is to make it Fast and Cheap. And Fast + Cheap = Inferior.
So where does that leave us. Hard question on a Friday!
Not all gray hairs are Dinosaurs!
September 6, 2013 at 3:30 pm
We're mostly CI and being pushed strongly towards CD, and I am just starting to think through some of the challenges. DB changes are seen as a major block to CD, and it does create some friction with some developers. Unfortunately databases are pretty much the anti-pattern for CD, they are shared across multiple teams, require specialized knowledge to code for properly, etc. A couple of issues I'm thinking about in particular are...
Performance: Databases in most systems are single point bottlenecks, so a single poorly performing query can have system wide effect in a way that most app code can't. Performance tests in general are a tricky part of CD, and database performance testing is trickier than most. This isn't just about queries. Adding a not null column to a table that has 10,000 rows in a developers environment probably won't even flag as an issue, but will turn into an unpleasant evening when the same script is deployed to the production system with 100 million rows.
Rollback: Rolling back code is easy. Rolling back data changes is really not. I was just reading a very well known book on the subject where they suggested "always make sure to backup your database" as a way to handle rollbacks. I almost threw the book down because of how fundamentally wrong headed that is. If I have to take a downtime long enough to restore one of our big db's from backup, I'm going to be unemployed. Rollback strategies for destructive data changes require careful planning and often keep both versions available for some period of time, using triggers, hiding through views, etc.
Not to suggest that there aren't others. I think my first bite at the agile apple will probably be to instrument db deployments in a performance environment in order to capture slow scripts.
September 6, 2013 at 8:09 pm
cdesmarais 49673 (9/6/2013)
...Rollback: Rolling back code is easy. Rolling back data changes is really not. I was just reading a very well known book on the subject where they suggested "always make sure to backup your database" as a way to handle rollbacks. I almost threw the book down because of how fundamentally wrong headed that is. If I have to take a downtime long enough to restore one of our big db's from backup, I'm going to be unemployed. Rollback strategies for destructive data changes require careful planning and often keep both versions available for some period of time, using triggers, hiding through views, etc.
A lot of our app changes were government drop-dead changes for HIPAA/Medicare changes. So the developers were directed to get the government changes in and correct. The subtle/user-ease fixes were dumped on the ground.
I was able to totally change the QA process when they moved the DEV team from Washington state to Ohio. Part of that was I build a DEV/QA silo that was based off three month behind the production DB. When they started doing second stage testing on that and it operated like crap, we started getting better code efficiency.
I never had to restore the DB for production systems. But I had a clue from the test environment. The developers knew better and we basically always worked out the door.
----------------
Jim P.
A little bit of this and a little byte of that can cause bloatware.
September 6, 2013 at 9:02 pm
The car industry tried this.
"Put it together as quickly as we can and let the consumers find the problems for us".
Then the Japanese moved in and the rest is history!
September 7, 2013 at 4:28 pm
Continuous development, continuous unit test, continuous integration, continuous system test, continuous QA, continuous release, and continuous deployment are all great concepts, and they work extremely well, as long as everyone remembers that it's no good shoving something out the door if it doesn't work - and that means that you've got to ensure that salesmen, marketeers (and managers in development, integration, QA, or operations chasing a bonus for quick work) can't hijack the process to achieve their personal aims. That's the only difficult part of continuous development, release and deployment - and it's really difficult, unless you can get the CEO onside.
And of course it isn't really continuous: it isn't even high frequency discontinuous: you are not going to be deploying a new system every 5 microseconds; it's incremental - and the people who pioneered it way back when called it incremental integration, validation, and release. And how often increments are deployed will vary - several increments in development and unit testing may turn into one larger increment in integration and QA, and several QA increments may turn into one release (I tended to try to avoid that happening when I was in charge), and not all releases will be deployed for every customer (ok, that could cause issues for customer support if you didn't keep good records of what went where and did let crud get out the door).
I haven't seen it put developers under extra pressure; in fact when properly managed it causes much less developer pressure than outmoded operating models like the waterfall.
Tom
September 11, 2013 at 4:09 am
CI is essential in modern software development. It is completely independent of CD in that you can get the benefits of CI without CD. Also CD doesn't necessarily mean that you release all successful builds deployed. CD can be to an internal/preview server. The key thing is that CI ensures that what you develop builds (and passes the tests you have created) whereas CD ensures that your successful builds deploy successfully. Deploy does not necessarily mean release.
As for devs? It is good for us. Only those that try and hide behind distant milestones, where it can be difficult to see who has done what, will be under significantly more pressure. Good. Everyone else can get satisfaction from progress and near immediate feedback.
The one caveat I will mention is that "it depends". This time it is how the project is managed.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
Viewing 15 posts - 1 through 14 (of 14 total)
You must be logged in to reply to this topic. Login to reply