About a year ago, fellow blogger Thomas LaRock recommended the book The Phoenix Project, claiming that “every business leader and IT professional should read this book”. Since I’m an IT professional (no jokes, please), and I’m supposed to be leading a team, that means me. He also uses the phrase “sinking ship”, which, frankly, describes how a good portion of 2013 felt for me. Tom’s advice is generally good, so I needed to read this book.
Turns out, I’m not the only one who took his advice. There have been several reviews of the book posted on SQL blogs in the past month or so. I’ve made it a point not to read any of them, because I didn’t want them to influence my own review that I knew I’d be writing. Obviously the book hits close to home for a lot of people, particularly the DBA people. After reading it, I understand why.
This is not a dry, boring self-help book. It’s a fictional account of a poor sap who finds himself being promoted to lead the IT operations for a fairly large company. Notice that I didn’t mention the industry? That’s because it doesn’t matter – the same story could be applied to virtually any company in any industry who, whether they know it or not, rely on IT just to keep the lights on. In companies like this, the IT guys are only noticed when something isn’t working. If email is down, it must be IT’s fault (vs the 99% of the time that email is working perfectly). Internet’s not working? Blame IT. Applications are slow or crashing? Blame development, another form of IT. A job in Information Technology can be a truly thankless job, very stressful, the kind of job where somebody’s always mad at somebody.
In The Phoenix Project, this is made evident from the very beginning. IT is being blamed for holding up a critical project. They’re caught between marketing and sales people who habitually over-commit and over-promise, senior management who doesn’t understand how the magic pixie dust that IT uses actually works, and end-users who think the Internet is that little blue spinning icon on the desktop. The former IT leadership has been fired, and our hero has just been forced into a promotion that he didn’t want.
Without going into too much detail and spoiling the book, I’ll simply say that as a recently promoted manager/supervisor, I can relate to the poor guy. I have the technical chops to deal with just about any “computer problem”, but when faced with the new “people problems” that my promotion brought along, I’ve been clueless. It’s been painful. It’s one thing to try to control your own behavior when fingers are being pointed at you, but trying to influence a whole team of people, and listen to their complaints (which are often the same as your own) is another thing entirely. It can be incredibly hard and frustrating when you’re not prepared for it.
I’m not going to tell you what happens in the book – go read it, you won’t be sorry. I’m going to tell you what I took away from the book, and how I’ve applied it to my own work. I’ll just say that some of the strategies employed by the guy in the book are so simple, and so commonsense, you’ll be kicking yourself for not figuring them out for yourself. That’s how I felt anyway.
So, what did I learn from reading The Phoenix Project?
Planned work vs unplanned work
This is huge, and it’s been right there in front of my face for years, but I never saw it. The easiest way to derail anything – a project, a work day, a weekend, a train, a recipe, is to throw something unplanned into the mix. Unless you’re really, really good at dealing with surprises, something you didn’t plan for is going to prevent you from doing what you had planned to do.
Since taking over leadership of the DBA team, I’ve tried to make us function as one generic organization, instead of 7 unique individuals. Oracle or SQL – it didn’t matter, we’re one team (in my mind anyway). We established a shared team mailbox. If you’re a developer and you need a DBA, you send an email to the mailbox, and whoever is available will respond. We established an on-call rotation, with each person taking the pager for a week at a time, and being responsible for monitoring that shared mailbox. Seemed like a good idea at the time, but soon project managers began complaining when the DBA that they’d been working with for two weeks is suddenly unavailable because of on-call, or because there’s a production issue. Less-experienced staff often had to escalate to the senior staff when paged for something – this led to essentially having two people on-call instead of just one.
The “team as a whole” concept wasn’t working, in spite of it seeming like such a good idea. After reading this book, and thinking about this planned/unplanned thing, I’ve restructured the team. We now have two DBAs dedicated to working on development projects – they’re no longer in the on-call rotation, or on the hook for production issues. Oracle DBAs are Oracle DBAs, they’re no longer part of the on-call rotation, they only deal with Oracle which isn’t quite as mission-critical as the SQL stuff. The two senior SQL guys are sharing on-call duties, and dealing with the day-to-day firefighting. In other words, we’ve separated the planned work (development) from the unplanned work (firefighting). It’s too early to judge, but so far, optimism is high, and the development managers are loving the idea.
Know your WIP
Certain developers and managers have a favorite DBA that they always go to when they need something. The nice lady in Finance knows nothing about DBAs or databases, doesn’t know that we have a DBA team, but she knows that nice boy named Brent who helped her with that report last year, so she always goes to Brent when she needs help. This is all well and good, but if Brent is busy working on a dozen other things, there are two likely outcomes – he stops working on those things to help the nice Finance lady (delaying planned work), or he tells her he’s busy and the nice Finance lady goes away to complain about Brent not helping her. As the manager, it’s my job to make sure the nice Finance lady is getting the help she needs and isn’t walking away to complain about Brent. Unless Brent somehow makes me aware of her request, I’m unable to do my job correctly.
Until you know what the demands are that are being placed on your team, there’s no way that you can try to manage them. It’s like trying to bail water from that sinking ship that Tom referred to, without knowing where the leaks are or how big they are. This was a huge problem for us. We were always busy, but nobody could readily say what we were working on, or what wasn’t getting done. We didn’t know our WIP, our work-in-progress.
We needed a way to make our work visible, to ourselves and to the rest of the team. I needed to be able to see what each person has on their plate, and what the team as a whole has on its plate. Email just doesn’t work for this, period. After experimenting with a couple of different tools, we’ve settled on one that seems like it offers what we need – Trello.
If you’re not familiar with Trello, it’s basically a web application based on the concept of a Kanban board. Kanban boards are key to solving the dilemma in The Phoenix Project. I’d never heard of Kanban before reading this book – I immediately saw its merits, and I’m convinced that it’s the answer to our workflow and visibility problems.
In short, work comes in, we create a card for it in Trello. That card sits on a “To-Do” list until it’s assigned to (or picked up by) someone, at which point it moves to a “Doing” list. It stays there until the work is finished (and moved to the “Done” list), or we hit a roadblock and have to wait on someone or something (moving the card to the “Blocked” list). I, as well as anybody else on the team, can see at a glance what we’re working on, what we’re waiting on, the status of a particular task, all in one place. This will help us identify bottlenecks inside or outside our team, and help me to understand which of my team members is Brent (read the book, you’ll understand).
Collaboration
In the end, The Phoenix Project is a book about DevOps. It’s about developers, QA people, IT people, and management learning to work together. In other words, it’s about collaboration.
I was already comfortably seated on the collaboration train after reading Winning From Within, but the lesson is repeated in The Phoenix Project. We’re all on the same team, we all have a vested interest in getting the work done and helping the company succeed. It’s hard to remember that some days, and some of my “management” techniques were making it harder. Hiding behind a generic team mailbox for example – by doing that, I turned the DBA team into a faceless, anonymous “thing” that nobody wanted to interact with. Kind of like calling the cable company because your Internet service is down – nobody wants to do that.
To help foster collaboration, and repair the damage done by past mistakes, I’ve committed one morning each week to physically sitting in the development section of the building. My two DBAs who are dedicated to development projects split the remaining four days of the week doing the same thing. We (the operations DBAs) sit on a different floor in the building, which doesn’t help with the “faceless entity” problem. By placing a DBA in their midst every day of the week, I’m giving us a face, making us “real” and approachable by developers.
We still have problems. There’s still a lot of animosity between the operations team as a whole and development. I can’t fix that by myself, I can only influence what the DBA team does. What we’re doing has been noticed, and we’ve gotten a lot of positive feedback, so I’m hoping others will get on board and try some of what we’re doing. If not, that’s fine too – I can still go home at the end of the day not exhausted from fighting all day long.
Get the book
Seriously. Go to Amazon right now, spend ten bucks and buy the book. You can easily read it in a week, likely much quicker than that. After you’ve read it, post a comment and let me know what you thought about it. It helped me revive myself in a job that I thought I was done with. I no longer feel burned out, I feel like the team is in better shape, and I’m repairing bridges that I’ve burned. You might say I’m creating something new from their ashes, like a Phoenix rising. Oh man, that was bad. I’m sorry…