Yesterday I talked about the importance of having a tested business recovery plan. If you didn't read yesterday's post, the gist of the event that initiated it was having dinner at a chain restaurant, one of those sit down types. Their computer systems were down and they were almost completely paralyzed by it. In listening to the conversations that the IT folks were having about the point of sale systems, it sounded like they had just performed an upgrade. And apparently not everything was going smoothly. Here's the thing: it was Friday night.So when I'm talking about planning, I mean planning with regards to determining the appropriate time to do the upgrade, not the steps itself, although that is very important, too.

This isn't just a sit down restaurant, it also has a bar, like most of these restaurants do. Friday and Saturday nights are usually the busiest. So it rather surprised me that the upgrade was performed before the Friday dinner service. This is when you're going to have the most customers for the week and the least tolerance for issues. So why do you do something to impact Friday night? I don't have an answer to that. If I had a choice, I certainly wouldn't.

Another example that comes to mind is a ministry partner I deal with where you have to purchase most of your supplies from them, because they are given out. Handbooks, shirts, etc. It's a children's ministry, so it is typically busiest at the times the academic year is the busiest: right at the beginning to order supplies that will be consumed during the year and and at the end when awards are ordered. This ministry decided to upgrade its sales system at the same time as schools started back, which also happened to be their busiest time of year order wise. And the upgrade didn't go well. Orders were lost. Orders weren't complete. It was a nightmare that so backlogged their system and their people that even trying to get a person on the phone was usually an exercise in futility. I asked the question of someone in the organization, "Why did you choose this time of year to do the upgrade?" The response I got back was, "The IT folks thought it would be okay." Oh, really? Shame on them.

Both cases strike me as examples of not planning your upgrade well. In the first example, there might have been a prevailing reason to do the upgrade right before Friday night dinner service. This is a chain and this might have been considered one of the least profitable sites and therefore you bit the bullet with respect to this particular location. If that was the line of thinking, and if the upgrades needed to happen quickly, then there is a legitimate business decision which drove it. In the second case, the folks in the "business" side of the ministry should have said, "No, not in August. If something goes wrong, too many folks are affected." Actually, they shouldn't have had to, because the IT folks should have thought, "Let's play the what if game. What if the upgrade goes bad? Systems are down and orders don't happen. We're a sole source for these supplies. When would be the best time to try and do this upgrade?" And they would have concluded August was not that time.

As IT folks we have to be smart. We have to consider the implications of an upgrade that doesn't go well. What is impacted? How does that affect the organization? Is there a better time to do this upgrade than any other? What prevents us from executing on that time? Is there a time when there's a great deal of risk? Do we have backup plans and workarounds that can mitigate the risk? Have we explained the risk to business so they can make the call? Those are the types of questions smart IT types ask. We want IT to enable people to work better/more efficiently. We don't want to be a roadblock to success. A failed or failing upgrade scheduled at the wrong time is just such one roadblock.