Today we have a guest editorial from Andy Warren as Steve is traveling. This editorial was originally published on Mar 4, 2014.

Recently I was part of a team that was doing a medium size/complexity migration. We came up with a plan, met to review it, and then after the appropriate communication to all involved, went forward with the implementation. It was only at the end that we realized that although we had executed the plan well, we had missed something. It as disappointing and disheartening to be sure. We spent an excited five minutes discussing ways to fix the problem and then decided that while we thought we could fix it, the right thing to do was rollback. It was not production, we had the luxury of time, and we needed a fully vetted process before we did the same in production.

The rollback went fine, as expected and planned. But it didn’t feel good. In part because we weren’t “done” and in larger part because we had missed something that in the glaring light of hindsight should not have been missed. Making it work would have felt better. It would have been a chance to be heroic.  Maybe it would have been cathartic too, working off some guilt by doing some frantic work to make it all go. Everyone was a bit down the rest of the day, me included.

Rollbacks aren’t failures (though I’ll grant a failed rollback is a whole different thing). That’s why we plan for things to go wrong, why we keep the rollback plan as simple as possible, and why we double check that we can execute it if needed – no shortcuts. I k now this, yet I still felt bad about it.

It’s a great topic to talk about before it happens. It will reinforce the need for planning for success and failure, and it will help the team key their eyes focused on the real win – winding up in a sustainable place.

How would you feel? How would your manager (and their manager) react to a rollback? Assuming it’s not an all the time thing is a rollback an acceptable outcome, or is anything less than getting it done treated as a failure?