In my first article, Tips for New DBAs, I introduced some of the fundamental principals of Database Administration and highlighted two areas of common malaise in our profession. In this second article, we will continue with tips for new DBAs and also discuss some customer service best practices. While some of my positions may raise criticisms, I do understand that every DBA and every organization is different and I encourage everyone to share their perspectives in the forum discussion for this article.
Know Your Limits
If there is something a seasoned IT professional knows very well it is the boundaries and limits within which they operate. Senior staff and management act as a sort of "border patrol" for these limits and also establish new limits. It is not uncommon for many of these limits to be specific to the organization, but also "common sense" practices that are anything but! To bridge the gap between seasoned veteran and shiny new rookie, I offer the main ingredient of "common sense", which is knowing how to spot your limits, or places where limits might be.
A common mistake Junior DBA's make is that they focus on how they should do something and ignore whether or not they should do something. Here is a made up example of what I'm talking about: Let's just say a developer calls and asks for a linked server to be created that points to a database on another SQL Server. The developer provides a login to use and the Junior DBA proceeds with enthusiasm and sets up the linked server. The problem with this scenario (apart from the issues associated with supporting linked servers) is that the DBA did not verify with the database owner that creating a linked server that pointed to her database was OK. While the developer might have had a valid login, the database owner needs to know and approve of any different type of access to their database.
What are the odds of this happening in real life? Let's just say this is not a "made up" example like I originally stated... In large companies it's not uncommon for one project to not know about the other. Part of a DBA's job is to ensure good development practices using the Confidentiality, Integrity and Availability concepts I mentioned in the first article. Verifying the authorization of all requests is part of that. Good change control processes are another big part. I'd say the DBA from our example managed to place both of these top priorities behind technical ability and "good customer service". Don't get me wrong, good customer service is something to strive for, just remember all of your customers when providing good customer service.
Another example of knowing your limits is cross-team problems and knowing when a problem should be addressed by another group. An example I can think of is a customer experiencing network problems connecting to SQL Server. While it might be tempting to help this customer until the bitter end, after you've confirmed that SQL server is running and you (and other customers) can connect, it's probably time to pass the buck to the network team, or at least get them involved. A second example is when a customer calls to complain about an out-of-window outage caused by the OS team missing the patch window, you absolutely should not apologize for the OS team, and instead communicate on your customer's behalf to the OS team.
Accountability and service expectations between teams are just as critical as knowing when to hand-off a support issue. In this case, the OS team needs to hear about their mistakes if they are ever to learn from them. In the best case, mistakes are met with process improvement and the entire organization benefits from it.
In essence, knowing your limits as a DBA revolves around two things:
- Is the action you are about to take fundamentally sound? Keep in mind the relationship between a DBA, the data, and the stakeholders.
- Is there another person or group in the organization that would be more suitable for the issue at hand?
The art of saying: NO!
Good customer service (for me) is a very rewarding part of the Database Administrator's job. Good customer service, however, does not mean doing everything the customer wants. A mature organization has an established framework of defined behaviors that govern our actions and guide our actions toward what is best for said organization. When something is clearly defined as a service you provide, you need to provide it. If it is clearly defined as something you do not do, you absolutely need to say "no", because that is what has been determined as best for the organization. It is tempting to gratify your need to be a good friend, or provide super-excellent "above and beyond" service, but this is not a good thing for the organization. Allow me to elaborate with 2 reasons why you shouldn't do everything the customer wants:
Reason number 1: You create an unrealistic service level expectation
When you raise the bar, you better make sure you can jump it every day; you also need to make sure everyone else on your team can jump it too. We define our levels of service at an organizational level to ensure that everyone can clear the bar every time. It is consistency and teamwork that create winning organizations, not fantastic individual efforts. An anecdote I can share is one that many of us are familiar with.
There was once a developer on a short timeline, he phoned the SQL help desk in a panic and was quickly moved by the on-call DBA to top priority in front of other customers that have been waiting longer. As his project progressed, the developer learned that he could short-cycle the helpdesk process by emailing the unlucky DBA directly. Soon enough, this developer's project was consuming several hours per week of the unwitting Jr. DBA's time. I think most of us have fallen into this trap at one time or another or seen it played out. Besides being a personal problem for the unlucky DBA, the organization started to feel the heat when the involved DBA left for vacation and the angry developer called management because he had built a short SQL Team turnaround time into his development timeline and nobody on the SQL team was willing to fast-track his requests like the vacationing DBA was. This situation clearly got out-of-control and it was mostly due to one DBA providing "too much" customer service. The customer began to depend on "above and beyond" customer service, which can only end badly.
Reason number 2: If you provide "unprovided" services, they become de facto service offering.
It's a fact: customers talk to each other. As they do so, the following conversations DO happen in meetings: "The SQL Server team has a 3 day guaranteed turnaround, but if you email So-and-so, they'll do it faster." Or: "I never fill out my change control paperwork. The SQL team will still do it in a pinch". These types of situations can very quickly spiral out-of-control. To best serve the organization, you need to stick with the documented processes, or get your actual processes documented (if they're different) and stick to the written word.
When you come upon a situation that isn't defined by your service offering, you need to make a decision and add it to your existing service offering so there is no grey area. It would be ideal to have an agile team that can quickly decide what the new service offering will be so you don't end up doing someone a favor that ends up being unsupported (but sort of supported) later. I'm not suggesting being rigid or unreasonably biased toward the status quo. Service offerings are organic and need to change with the technologies and the ability of the organization to support them. My point is that whatever the decision is, it should be written down and it needs to be crystal clear what is supported and what is not supported. Most importantly, everyone needs to stick with the service offering and not be a hero.
So...how do you say "No" to people?
I am amazed at how hard it is for people to say no. I don't want to get into the psychological causes of this behavior, but rather provide everyone with the tools and justification needed to say no when it needs to be said, which turns out is quite often.
I find it best to use the framework of organizational processes as the foundation for any refusal. Remember you are not refusing the request personally; you are instead doing what the organization wants you to do. In no way should saying no be confrontational, personal, or difficult, but rather a professional necessity.
"Our team doesn't provide that service", or "we have processes in place to best serve the organization", are two handy phrases I find myself using when called upon to settle these issues. I'm sure there are other great phrases that are more tailored for your organization, but the key is in the firm, yet professional demeanor with which you communicate with your customer. Never get personal, never get angry, and never be rude. Always remember that your refusal is part of what is best for the company, and ignore any criticisms that might be hurled your way. If you are acting rightly and in accordance with your written policy, management will stand behind you, especially if you are firm and professional. If you slip into unprofessional behavior, management's support of your refusal might slip (necessarily) and you might find yourself providing "unprovided" services to make up for the PR losses you create by being unprofessional.
Script your way to into the heart of SQL Server
The last bit of advice in today's article is one that is near and dear to my heart. If I could offer one technical piece of advice on SQL Server administration, it would be to learn T-SQL, and especially the system stored procedures, views, and tables. Once you know where you can find everything in SQL Server, your ability to automate processes becomes a driving factor in your work. When you know how to script, you'll find that when doing something tedious a few times, you begin to drift into thoughts of writing a script instead of spending time on a boring manual process. Good scripters realize how much time can be saved in tedious administrative tasks by automating the process. Not only that, but by scripting out a manual process, you reduce the chances of manual error! For example, a script never forgets to clean up the orphaned users after moving a database to a different server... I know for a fact that DBAs sometimes do. Scripts can be modified over time for continuous improvement, too!
My recommended path of T-SQL scripting progression would be to work your way up from T-SQL batches to stored procedures, functions, and eventually mastering return codes and error handling. When learning to write T-SQL batches, never underestimate the ability of a cursor. I know cursors are out-of-fashion in some circles, but looping through a selected set of databases, users, tables, indexes, etc. and performing an action on them is the bread and butter of a scripting DBA. Dynamic SQL, INSERT...EXEC(), and stored procedures or functions with return values are also valuable weapons in the scripting DBA's arsenal.
Once you master these elements of T-SQL scripting, the possibilities open up and you find yourself able to accomplish quite a lot. The best part is that you can save all your scripts and re-use portions of them making it possible to develop new scripts quickly and easily. You don't need to commit every parameter of every system stored procedure to memory, but rather use the books online or search engines to help you remember the syntax. I also recommend filing your scripts in a logical way so you can reference them quickly when you need to re-use a piece of them.
Lastly, use the vast script library on SQL Server Central as a reference. It's very unlikely that you're the first person to have encountered the need to write a particular script. By modifying other people's scripts or taking portions of them, you can learn better ways of doing things, or even improve the scripts to be more "automatic", intelligent, and tailored to your needs.
That's it for this article. Tune in next time when I talk about "Zen and the 80/20 principal", Experience vs. Certification, and the capability maturity model.
