September 24, 2010 at 9:59 am
Ok, I'm at my wits end, run out of ideas and called in all my favors to the super DBA's I know. I can NOT solve this problem.
This is a Win server 2008 Sp2 32-bit os with 32-bit sql server 2008 sp1. It's 32-bit because the vendor software will not support (they say it actually breaks it) 64-bit, end of discussion on that point.
This system itself is a fairly beefy two cpu quad core with 144gb of ram (can you say waste?) and emc fibre attached san.
For several days now something I can't pinpoint is causing this error to throw over and over in the sql logs.
It is cleared up only by restarting sql (with SLA's this is a REAL problem). I've found several postings regarding this, including this forum and a few MS forums people suggesting the -g512 startup param (which I'm running). When these first crept up I was running DB mirroring, and a tlog backup every 15 min that backs up 10 user db's and a trace that runs constantly (audit compliance requirement). The vendor software also uses reporting services. I have since disabled mirroring, the trace and reporting services. The issue still occurred today. I'm fairly certain it is a process the vendor is running, but I'm really not sure how to narrow it down.
If I let the system run with these errors continuing, it will eventually get to a point where I can't even open the sql logs. I have to go to them through the filesystem.
I'm really quite stumped on this one, so I really could use some help.
September 24, 2010 at 3:05 pm
Just throwing this out there, are there any OLE DB Linked Servers?
The vendor application you mentioned, is it performing a bulk insert?
Can you post the error log?
How much memory do you have allocated to SQL?
Do you have AWE enabled?
And finally - are locked pages enabled? Please read post - http://technet.microsoft.com/en-us/library/ms190730.aspx
Chris Powell
George: You're kidding.
Elroy: Nope.
George: Then lie to me and say you're kidding.
September 24, 2010 at 3:52 pm
No linked servers, I've been going through the trace file trying to find bulk inserts actually and haven't seen any. So I'm going to say inconclusive on that.
Attached is part of the error log, it's mostly filled with this error though and it overlaps with the tlog backups occasionally. The MSDTC error I had forgotten about when I posted here, it only showed up on day one, it's never reappeared. I'm thinking it may have been a casualty not a cause. Still though... not sure about that.
I am using AWE and I had SQL allocated to use the default very large value but I dropped it to 12gb today.
As for Locked pages, yes they are enabled I read that post when I was building this system actually. 🙂 Good to know that I was on the right track.
September 24, 2010 at 4:16 pm
Not sure what all you've looked into yet, sure you've exhausted a lot, buuut:
Is there a lot of databases on this server? http://support.microsoft.com/kb/316749 (near the bottom)
Looking at the error log it's complained about DTS taking a dive, mirroring taking a dive, and DTC taking a dive before the errors really kick in for real.
How's the connection to the mirror?
Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.
For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA
September 24, 2010 at 4:39 pm
The mirror broke on that day, it recovered nicely though once everything smoothed out. The mirror issues and the MSDTC issues never recurred. There are only 10 databases. So I'm no where near the limits MS is expecting.
September 27, 2010 at 10:05 am
Does anyone have any troubleshooting ideas? I'm working on getting Microsoft support involved but it looks like that's going to take a little while.
Viewing 6 posts - 1 through 5 (of 5 total)
You must be logged in to reply to this topic. Login to reply