Have you ever been the sole DBA at a company from the ground up? Me either. Full control of installations, processes and monitoring is a pipe dream for many. Understanding every aspect of your environment and knowing how to react to specific issues that arise, is the penultimate achievement. Unfortunately, many a DBA will never see this utopia, and must deal with many obstacles while working on a team. How does a team of DBAs work best together to achieve a well-rounded work experience while providing the company a valuable return on its investment?

Most DBAs have differing levels of expertise and are more proficient in certain aspects of their job. Maybe one of the following skills rings close to home: Installation and Configuration, Performance Tuning, BI, ETL, Development, High Availability, Disaster Recovery, Cloud based platforms, etc. I dare say none of the DBAs I've worked with over the years would be considered experts in all these areas. While I'm sure there is no perfect answer that works for everyone, I wanted to offer a solution that has worked well in my environment. Let's look at one possible way to blend all these together to build a proficient DBA team.

Introducing DBA Roles - While I've seen similar approaches before, this is the first time I've seen it done well. Obviously, every company has differing rules on how to treat each environment (Prod vs. Test vs. Dev), so adjustments may be needed to fit yours. Development tasks like BI and ETL are handled by another team in my organization, so are not included in this write-up.  Our team consists of 4 roles: Monitoring, Troubleshooting, Operations, and Projects. Let me provide a short description of each one.

Monitoring:

This is the primary support role. This role is responsible for all incoming requests to the DBA team, whether they be issues or simple service requests. The DBA monitors the incoming ticket queue, and emails. Here are few of the items the Monitoring role will handle...

1. After hours on-call support.
2. Restore requests (Prod to Test backups\restores, etc.)
3. Script execution requests - in non-Prod environments.
4. Responding to alerts from our monitoring system. These include failed jobs, performance alerts, storage alerts, etc.
5. Initial troubleshooting for any incidents reported to the team.
6. Develop Standard Operating Procedures

Troubleshooting:

This role is the backup to Monitoring. The Troubleshooting DBA will take over in-depth troubleshooting from the Monitoring DBA as needed. This frees up the Monitoring DBA for their primary responsibilities. Some of the Troubleshooting DBA tasks are...

1. Comprehensive troubleshooting as required to relieve Monitoring DBA.
2. Security access requests - User\Security management.
3. Patching.
4. Upgrades.
5. Migrations.
6. Growth planning.
7. DR planning and validation.
8. Preventative maintenance.

Operations:

Handles all meetings with integrations and development teams. Additional responsibilities include...

1. Execute scheduled deployments to production.
2. Planning decommission activities.
3. Maintaining inventory.
4. Licensing audits.
5. Usage reports and trending.

Projects:

Work on assigned projects, without fear of interruptions.

1. Infrastructure projects.
2. Special projects.

So, a typical day looks like this. The monitoring DBA handles the brunt of support issues. Troubleshooting takes any overload from Monitoring and coordinates communication efforts as needed. Operations handles deployment requests and external team meetings. The Projects DBA has the most available time to dedicate to ongoing project work. The deeper into the role hierarchy you go, the more time you have for specific tasks. It goes without saying that anyone can volunteer to assist other roles if they become overwhelmed.

Are you following so far? Sounds solid right? Time to mashup. Let's add a weekly rotation. Each DBA rotates between all 4 roles, thus exposing them to all aspects of the job. Obviously, communication is key to making this all work. How does each DBA know what was done in the prior rotation? We use Microsoft Teams, with a team created for each role. A weekly log is stored in each team for reference and reviewed in our weekly collaboration meetings. More detailed notes are available in each team, and all activity is linked to our ticketing system for easy searching and time keeping.

By taking this approach, each DBA becomes familiar with all the activities our team is engaged in. We are sometimes forced out of our comfort zone to learn something we weren't expecting. Everyone gets exposure to new technologies and experiences. The company benefits by having a defined, documented workflow, and a team of DBAs that have shared knowledge. No one DBA is that bastion of information whose absence can negatively impact daily operations.

What do you think? Have you implemented something similar? I would love to hear about it.