Don't Build a Monitoring System

  • I'm on board with Marcus... cost is a major, major factor. It's hard enough to sell a few hundred bucks for a monitoring package - but then multiply that by the number of servers in your environment, and if you're in a medium-to-large shop, it just becomes prohibitive. In most cases, we're really not all that interested in the deeper stats... we just need something that gives us a heads-up on four or five key items.

    I'd love to find a relatively inexpensive SQL monitor (or a general-purpose monitor that we could write our own stats-post into) that we can buy once, and tie in to any number of servers. Or one that has a minimal cost variance depending on tiers of servers... 1-10, 11-50, 50-200, and 200+ - something like that. I could probably 'sell' $2000 for the 11-50 tier... but could never sell $30,000 just to get basic monitoring for 20 SQL Server Express boxes that we paid... $0 for. We could just as easily hire someone, and have a multi-purpose entry-level DBA trainee.

    I find myself in a loop, though - I need better monitoring than I have today, I could write something basic if I wanted to take time away from other projects, but the price tag on the out-of-the-box ones that are worth the expense just keeps us away. So I just keep dragging along with mediocre monitoring and reactive corrections, and start the loop all over...

  • I find it depends on the enviroment and how large/varied it is on if i would want to use 3rd party tools. I have tested numerous tools out where i work and have yet to find one that is better then rolling my own solution. I think if i was in a smaller enviroment it would be easier to rely on a 3rd party solution but we have a large number of SQL servers on many different versions so it makes it hard to gather all relavant info across 1000+ servers. Also I have found that every 3rd party tool has its pro's and con's and i don't think there could be an overall winner unless it is for a specific task. Maybe something that compared these different tools against each other on a task by task basis and also included a couple ways to do it with default MS tools would be the way to go.

  • I would agree on the buy decision and I've often wondered why anyone would build their own monitoring system. And if I couldn't buy a solution I would look at several free or low cost monitoring tools before rolling my own.

    I've seen Ops Mgr (SCOM) 2007 presentations at the SQL PASS Summit even though Ops Mgr isn't "free". So, perhaps as long as its a Microsoft monitoring products its OK 🙂

    My local SQL user group has a no vendor product demo policy. Some of our user group attendees are from small shops that lack any tool funding and feel, rightly so, the product demos are of little value to them. I on the other hand, find the product demos useful, but I guess that's what vendor exhibit halls are for.

  • IMO the single biggest reason why enterprises do not go down the 3rd party SQL monitoring software route is cost. Some of these performance monitoring tools are close to $1000 a server. When you have 100s of servers the incentive to build your own is high. Large enterprises already have a global monitoring solution so any additonal monitoring tools need to be motivated for and also have to intergrate seemlessly with the enterprise monitoring solution.

    Any DBA worth his salt needs to be able to collect and analyse peformance data using native tools.

    Having said that I would always welcome articles, sessions and reviews on monitoring products provided that the licensing and ongoing support costs are clearly listed in the article as well as associated infrastructure costs in implementing the software.

    thanks

    SQL_EXPAT

  • It seems to me we have a forum here with lots of people rolling their own solutions - surely this would be an ideal open source project to harness that work into a collaborative alternative to the big money solutions that so many cannot afford or justify

  • Of course your are right. But as a DBA you also should understand how Software development and Applications are working with Databases.

    So from my point of view a DBA should also develop a small application and maintain it if possible.

    A good example is a monitoring tool.

    I our company we have been developing such a tool with a central Repository (SQL Server database). This repository consolidates infos from our SQL Instances, databases,

    databases files etc...

    I' m still working on it but not all the time.

    When I' am dealing with our development team, i can be more concrete.

    Yves Walter

    DBA from a Insurance company in Switzerland.

  • James Horsley (9/15/2009)


    It seems to me we have a forum here with lots of people rolling their own solutions - surely this would be an ideal open source project to harness that work into a collaborative alternative to the big money solutions that so many cannot afford or justify

    I too would like to suggest that some other folks should produce a free open-source tool for server management 😉

    And I know what it'll look like if they do.

    On the other hand:

    (1) How much commercial software in this category is really useful, and how much falls into "executive toy" class? That is, it lets you brag over the colleague or competitor who doesn't have CIO Dashboard Xtreme for Microsof SQL Serverz, and you can lean back in your chair and watch LED-styled indicators run up and down and on and off l!all day, but it doesn't serve a purpose (and maybe neither do you)?

    (2) Even if it is useful, is it value for money, or is it expensive because SQL Server is expensive?

    For comparison, look at the price of "apps" for iPhone, or for Palm computers, alongside similar programs for desktop Windows - say a diet calculator/log or a game of checkers - sold are "shareware". The desktop product may represent only a similar amount of creative effort but the price ticket is probably a multiple of the handheld-device edition. The executable file is also probably many times larger. Well, the server is a step up in price again.

    On the other hand, would your boss authorise paying $20 for software to manage 100 SQL Servers? Or would it look suspicious? Better to charge $200 PER server, then the customer feels they got their money's worth.

    Oh, there's also the question of who's liable if an open-source product actually interferes with your servers. Or if you feel like claiming that it did. I am mentally speculating how much of the price tag on SQL Compare is the vendor's liability insurance. You can speculate too if you like.

    Maybe that isn't insuperable. For instance, we're proposing an open source tool. How about something that takes the form of merely generating a tailor-made Transact-SQL script that produces a health report on your servers, but technically you use the script at your own risk and are responsible for understanding what it does?

  • I depends 😉

    $$$ caused me to have to build my own inventory solution, providing simple history information for instance settings, databases, tables, indexes and volumes.

    And a good lesson it has been. Learning and detecting where and how DMO and SMO can help out in a performant way ... or not.

    For the great solutions that are available on the market, $$$ and licensing types are the big setback issues.

    Another huge problem is the add-ons to as well your target server(s) as well as your user databases.

    Especially if you want to remove the product afterward, in many cases leaving unused stuff on the os or in your db.

    One of the "problems" with many state of the art packaged solutions, is generalisation. There seems to be no simple way of defining high priority servers vs low priority using e.g. profiles and exception rules.

    This would come to studying your products xml config, ... in detail or accepting the vast amount of alerts being generated and overlooking the single one you needed.

    And yes, in many cases, the HW you would need to collect and process your needed data would be way to expensive (on top of the product ! ) to get your total projects budget approved, because for the product, you can start small and buy extra licenses when needed, but your infrastructure must be in place as of day 1 !

    In many cases, there is a different product to do your firefighting vs doing your evolution analysis. No problem with that, but many times, these products aren't using the same set of data or can't even share the provided data.

    Accountants that manage your purse, aren't interested in the great stuff you see when evaluating a product.

    Johan

    Learn to play, play to learn !

    Dont drive faster than your guardian angel can fly ...
    but keeping both feet on the ground wont get you anywhere :w00t:

    - How to post Performance Problems
    - How to post data/code to get the best help[/url]

    - How to prevent a sore throat after hours of presenting ppt

    press F1 for solution, press shift+F1 for urgent solution 😀

    Need a bit of Powershell? How about this

    Who am I ? Sometimes this is me but most of the time this is me

  • I worked out the ROI and cost-of-implementation for buying monitoring software vs building it ourselves. Buying would, long-term, cost less, and short-term, be in place immediately, while building would take much longer (lots of wheels to re-invent) and cost more (even if we just count salary for those involved in building it), but has no immediate "today cost". They decided to insist on building it. For a company that exists to manage money, this place has a huge tendency towards "penny wise, dollar foolish".

    - Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
    Property of The Thread

    "Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon

  • After reading some of the comments I just have to ask a question.

    How much is your database environment worth?

    Sure you can "roll your own", I've created / use it with our 3rd party monitoring software and we have over 100 server. Yes it's not cheap to buy but neither is our data and / or our environment. If we have a problem with our data ware house, for example, then the dollars lost would have paid for the monitoring license.

    Open source SQL monitoring project, great count me in but I still believe that you can have both. Use your 3rd party tool for what it does best and use "rolled" code for things it doesn't.

    Rudy

  • As someone relatively new to the World of the SQL DBA, I'd be only too glad to hear what third party products others are using to help monitor their servers and databases. Being new it's not even easy to know what to monitor, let alone work out which system tables and DMVs to use to do so 🙂

  • GSquared (9/15/2009)


    I worked out the ROI and cost-of-implementation for buying monitoring software vs building it ourselves. Buying would, long-term, cost less, and short-term, be in place immediately, while building would take much longer (lots of wheels to re-invent) and cost more (even if we just count salary for those involved in building it), but has no immediate "today cost". They decided to insist on building it. For a company that exists to manage money, this place has a huge tendency towards "penny wise, dollar foolish".

    Absolutely. We had been working in house on a monitoring solution for a piece of our monitoring and it had been under development for a year and still didn't work right on one server, let alone on 10 or the 30 we needed it for. I finally found a resource that did about 85% of what we wanted. The cost wasn't too high (although it was high enough) but I installed it and had it monitoring all our production servers within a couple of hours. As you say, measure the cost, not just in time spent developing, but in immediate use of the tool and all the benefits you get and compare that to not having the tool. It frequently makes the decision easier.

    "The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
    - Theodore Roosevelt

    Author of:
    SQL Server Execution Plans
    SQL Server Query Performance Tuning

  • chrisn (9/14/2009)


    Working for a development house, we understand that our clients may not have all the tools of our choice, and that we need to be able to understand what is going on with SQL Server given the native utilities. If you don't understand the anatomy, it's hard to prescribe the cure.

    No it's not, send 'em for an MRI.

    Oh, wait, wrong debate, sorry....:-P

    ---------------------------------------------------------
    How best to post your question[/url]
    How to post performance problems[/url]
    Tally Table:What it is and how it replaces a loop[/url]

    "stewsterl 80804 (10/16/2009)I guess when you stop and try to understand the solution provided you not only learn, but save yourself some headaches when you need to make any slight changes."

  • dead nuts on! gui tools are great only if you understand where the data is coming from and the limitations of the data. I know many developers who can use the gui tool from work but are lost when they don't have the tools available. I know others who can figure out how to get the data when they don't have the gui tool. I consider the last group to be the senior developers.

  • 1) I think it's really important to understand the basics, so writing one's own tools is excellent for that. That said, there are plenty of people who are far more knowledgeable about monitoring the servers than I, which is a good reason to take advantage of some third-party tools.

    2) What I would like to see is a comprehensive comparison of some of the commmon tools available. I am reminded of the product reviews I see in magazines, like PC Magazine or even Popular Photography, where a set of experts in the field do comprehensive tests of common products and list the pros and cons. Then the consumer can easily determine which product fits their particular needs. Of course, those people are getting paid to be the experts, but the key is that this panel reviews all the products, so they offer some consistency in evaluating them.

Viewing 15 posts - 16 through 30 (of 38 total)

You must be logged in to reply to this topic. Login to reply