SCOME - Centralize Monitoring with ASP.NET - Part 1
-
I'm glad there's been interest. Got to admit, I was a bit anxious with it being my first article and all.
Andrew Peterson (4/20/2009)
Step One - get a server.Most people need to see it to believe it. No matter what the topic.
So, rather than spend time justifying a new server, I take the 'show them' route...
A Do'er!
Andrew Peterson (4/20/2009)
Follow up:For those not wanting to learn and/or write ASP web pages, I highly suggest you consider using SSRS as the reporting tool. You have less control with the vanilla framework, but its up and running fast. ~ But, Drew's ASP.net approach will look better and will be more impressive.
Of course it will. 😀
sean hawkes (4/20/2009)
On the flip side, I am going to use this as an excuse to get back into coding so I really can't wait.Sean
Wise move. I think that was originally why I decided to write them in ASP.Net. The thing with using reporting services is that there is always a possibility that 6 months or a year down the line, you think of this great report to do, but realise after a bit of research that it's just too tedious to do in SSRS or it can't be done at all because, as Andrews says it's vanilla. Over the last year many of the reports have been injected with steroids. For example, one of the articles to come is the Test Restore report, that consists of two pages. One page to set up which databases on which servers I want restoring to the Test Server, and another to tell me the outcome of restores.
Set up what we want restoring when and where to:
Last nights outcome:
In the background, we're using .Net and T-SQL to copy the databases across the network, specify which backup sets to restore and so on...
Another report is not a report at all, but an auditing page that allows me to record any work done on any server, upload attachments used etc.
Bill Scott (4/20/2009)
We did something like this at a previous employer. We ended up monitoring so many servers and so many processes we had a consolidated display with stoplights. Green lights for the things that went according to schedule, yellow if there was a nonfatal, and the loudest meanest red for when things went bad....That's so cool. You didn't take a copy with you, did ya? 🙂
Bill Scott (4/20/2009)
heck, we even went so far as to set up monitoring on all the monitoring programs 🙂
Yes, to come in article Part 6! It's no good having a monitoring system if the jobs that trigger the monitoring system themselves fail.
Incidently, there are many online vb.net to c# convertors for the c#'ers, but I can supply the scripts in C# if you want.
Thanks to all, speak to you soon.
Drew
-
Looking good Drew! I don't have quite that many servers to contend with, but I'm always looking for ways to streamline monitoring. I've got two years of VB.NET application development under my belt, but until reading this, I hadn't given any thought to writing a central monitoring app like this. I'm looking forward to your next article.
Thanks!
Thaskalas
'When are you going to understand that being normal isn't necessarily a virtue, it rather denotes a lack of courage."
-
Drew,
Thanks for the article. Looking forward to the upcoming installments.
When I first saw the title of your article, I actually thought it was about using Microsoft System Center Operations Manager (SCOM) for SQL Server monitoring. It seems though that, except for the first 4 letters of your acronym, and the common mandate for centralized monitoring, your approach has nothing to do with that other SCOM. What a weird coincidence! 🙂
In our organization we are using SCOM for system monitoring and we harness the reporting capabilities of SCOM for some of our reporting. For the rest of our monitoring and reporting we use SSRS and some in-house solutions.
I always find it interesting to learn about different ways of monitoring SQL environments. Every approach has its own strengths, and I think your approach's greatest strength is its versatility, something lacking to some degree in canned products like SCOM, SSRS etc.
I have always maintained that DBAs need to keep training their development muscles - myself being a former developer turned DBA some time ago. It is approaches like yours that help in this direction.
Keep up the good work!
__________________________________________________________________________________
SQL Server 2016 Columnstore Index Enhancements - System Views for Disk-Based Tables[/url]
Persisting SQL Server Index-Usage Statistics with MERGE[/url]
Turbocharge Your Database Maintenance With Service Broker: Part 2[/url] -
Marios Philippopoulos (4/20/2009)
Drew,Thanks for the article. Looking forward to the upcoming installments.
When I first saw the title of your article, I actually thought it was about using Microsoft System Center Operations Manager (SCOM) for SQL Server monitoring. It seems though that, except for the first 4 letters of your acronym, and the common mandate for centralized monitoring, your approach has nothing to do with that other SCOM. What a weird coincidence! 🙂
In our organization we are using SCOM for system monitoring and we harness the reporting capabilities of SCOM for some of our reporting. For the rest of our monitoring and reporting we use SSRS and some in-house solutions.
I always find it interesting to learn about different ways of monitoring SQL environments. Every approach has its own strengths, and I think your approach's greatest strength is its versatility, something lacking to some degree in canned products like SCOM, SSRS etc.
I have always maintained that DBAs need to keep training their development muscles - myself being a former developer turned DBA some time ago. It is approaches like yours that help in this direction.
Keep up the good work!
That completely passed me. I was asking Steve Jones of SQLServerCentral.com what I should call the artcles and he suggested using the acronym. I hope people haven't thought that was deliberate :w00t:
What's SCOM like to use generally?
Drew
-
Drew Salem (4/21/2009)
That completely passed me. I was asking Steve Jones of SQLServerCentral.com what I should call the artcles and he suggested using the acronym. I hope people haven't thought that was deliberate :w00t:
What's SCOM like to use generally?
Drew
SCOM has a painful learning curve, partly because of poor documentation. Its greatest strength is the ability to "detect" new SQL instances, databases, servers etc. and automatically add those to the monitoring process. For example, if one wants to check for failed jobs, all one has to do is write a script to configure all jobs to write to the Event Log on failure. The script is deployed ONCE and can be targetted to ALL sql instances in the environment, without having to reference them by name. Every computer on the domain, in which the SCOM agent is installed, will pick up and run that script on a pre-configured schedule and alerts will be sent out when jobs fail. The failures are also listed on the SCOM console.
The strength of your solution is primarily in the reporting aspect. SCOM is somewhat lacking there.
There are ways to also customize the type of information to gather using SCOM. I wrote an article some time ago on this subject:
http://www.sqlservercentral.com/articles/Blocking/64038/
__________________________________________________________________________________
SQL Server 2016 Columnstore Index Enhancements - System Views for Disk-Based Tables[/url]
Persisting SQL Server Index-Usage Statistics with MERGE[/url]
Turbocharge Your Database Maintenance With Service Broker: Part 2[/url] -
yep my thoughts too re SCOM/SCOME. If your company has the resources to pay for 47 SQL licenses then perhaps it should invest in SCOM which via built-in and downloadable management packs provides out-of-the-box monitoring across systems and servers/services. I don't work for Microsoft, but if there is a business case/need to provide good quality monitoring/support then I'd prefer to buy a product rather than engineer a 'free' version.
Hey I like the idea of SCOME, especially when there is no chance of purchasing anything to do the job for me 🙂
-
I have created a few SQL Reporting Services reports for my daily DBA tasks but keep adding servers (dev and stage) that I want to check. So this article is timely....if he shows me how to find errors in SQL Server error logs so my SQL reports can read it, I'll have to send him a present....
Tammy
-
tjmcols (4/21/2009)
I have created a few SQL Reporting Services reports for my daily DBA tasks but keep adding servers (dev and stage) that I want to check. So this article is timely....if he shows me how to find errors in SQL Server error logs so my SQL reports can read it, I'll have to send him a present....Tammy
"He" has a report for it! It uses xp_readerrorlog to retrieve the logs and does a wildcard search on keywords such as "failed".
-
Drew,
I read your article on SCOME and it is very interesting.
I do have a separate test server for myself to do various dba testing before implementing it in prod.
In my previous job, I used to monitor 50 sql server db.
I did write and schedule T-SQL scripts like failed jobs written to a table on the same server and later copy it over to the test server table using dts/ssis.
I used a sql server procedure which can generate a html report of the table query and wrote to the network drive.
This html report can then be viewed from any desktop using browser with a periodic refresh.
I was able to write to check for Disk Space, Failed jobs etc.
Looking back at it, I think it is a poor man's way of doing things.
Let me test your approach and see if this can be useful in my current job.
I have never worked on ASP (have some power builder and excel vba experience).
I assume this is a good place to start with what i know and what i can learn new.
Thanks for sharing your experience.
Murali
-
Yes, please keep it simple. I'm an Oracle DBA who now has to also run MSSQL databases. I don't know VB, .Net, C#. I'm learning some TSQL (this site is a big help) but I'm really not that eager to learn VB, .Net or C#. A project like this one, with what I need to type in a c/p format, may make it interesting to start to learn some of those things.
I look forward to your series of articles.
-
Drew, the thing that confuses me about sites like the one you describe - which I know are perfectly common - is how they manage the licensing - both logistically, and the cost!
My SQL Server is heavyweight piece of hardware - 128 GB memory, 8x2 core, multi-channel io, Equalogics array - with multiple instances on it and most database packed into a single instance (we add multiple instances only when a 3rd party product has unreasonable demands like "must have server adm" or "keeps flushing the whole datacache").
The was MS licenses, that keeps the license cost low. Splitting that across 4 2x2 core machines would be twice the cost. Unless your virtual 2003 servers are all on one box - I sense they aren't - doesn't that waste a lot of licensing money?
That's what I never understand about sites that don't have this consolidated. We have a strong culture of centralization and consolidation, and we were doing client/server databases long before MS took over the Windows port of Sybase SQL Server, so we never fell into that "SQL Servers everywhere", but I know it's a real common issue.
Roger L Reid
-
Roger L Reid (4/22/2009)
Drew, the thing that confuses me about sites like the one you describe - which I know are perfectly common - is how they manage the licensing - both logistically, and the cost!My SQL Server is heavyweight piece of hardware - 128 GB memory, 8x2 core, multi-channel io, Equalogics array - with multiple instances on it and most database packed into a single instance (we add multiple instances only when a 3rd party product has unreasonable demands like "must have server adm" or "keeps flushing the whole datacache").
The was MS licenses, that keeps the license cost low. Splitting that across 4 2x2 core machines would be twice the cost. Unless your virtual 2003 servers are all on one box - I sense they aren't - doesn't that waste a lot of licensing money?
That's what I never understand about sites that don't have this consolidated. We have a strong culture of centralization and consolidation, and we were doing client/server databases long before MS took over the Windows port of Sybase SQL Server, so we never fell into that "SQL Servers everywhere", but I know it's a real common issue.
Consolidation makes a lot of sense, but there is still one problem with it: proper resource management and allocation. Even with your resources one still needs to ensure that one database/application will not end up hogging all resources at the expense of all others. Having individual instances makes resource management easier and more efficient by making it possible to define min/max memory and CPU-mapping per instance.
In SQL 2008 Enterprise Ed there is something called Resource Governor which I think addresses this problem, but there is nothing else like it that I am aware of.
__________________________________________________________________________________
SQL Server 2016 Columnstore Index Enhancements - System Views for Disk-Based Tables[/url]
Persisting SQL Server Index-Usage Statistics with MERGE[/url]
Turbocharge Your Database Maintenance With Service Broker: Part 2[/url] -
Roger L Reid (4/22/2009)
Drew, the thing that confuses me about sites like the one you describe - which I know are perfectly common - is how they manage the licensing - both logistically, and the cost!My SQL Server is heavyweight piece of hardware - 128 GB memory, 8x2 core, multi-channel io, Equalogics array - with multiple instances on it and most database packed into a single instance (we add multiple instances only when a 3rd party product has unreasonable demands like "must have server adm" or "keeps flushing the whole datacache").
The was MS licenses, that keeps the license cost low. Splitting that across 4 2x2 core machines would be twice the cost. Unless your virtual 2003 servers are all on one box - I sense they aren't - doesn't that waste a lot of licensing money?
That's what I never understand about sites that don't have this consolidated. We have a strong culture of centralization and consolidation, and we were doing client/server databases long before MS took over the Windows port of Sybase SQL Server, so we never fell into that "SQL Servers everywhere", but I know it's a real common issue.
Roger, don't even get me started on licensing. It's a bit of a sore subject where I work. I've covered my ar*e and made my recomendations to the people at the top, but Microsoft are on their cases now. All the servers are hosted on 4 big VMWare boxes. They've been going Server-CAL for the past few years, but I've been trying to get them to go EE CPU for each VM box. All SQL Servers on each box are licensed that way.
-
Drew,
I read the article with the anticipation of seeing how you managed to create the sidebar seen here.
How did you manage to get that?
PatrickPK
-
Viewing 15 posts - 16 through 30 (of 32 total)
You must be logged in to reply to this topic. Login to reply