SqlServer Stack Dumps
-
Hi all,
Have been experiencing some problems with our SqlServer 2k recently, we're running NT4 sp2, Sql2000 sp3a and we are getting frequent assertion errors, then stack dumps and then..........nothing, no response, sql terminated !.
Here's some of the log file details;
2003-12-10 12:39:13.45 spid54SQL Server Assertion: File: <p:\sql\ntdbms\storeng\drs\include\record.inl>, lin
2003-12-10 12:39:13.45 spid54Stack Signature for the dump is 0x6FAC1FFF
2003-12-10 12:38:59.48 spid54Using 'dbghelp.dll' version '4.0.5'...
2003-12-10 12:38:59.46 spid54Could not open FCB for invalid file ID 24320 in database 'DEVELOP'..
2003-12-10 12:38:59.46 spid54Error: 5180, Severity: 22, State: 1
2003-12-10 12:38:59.45 spid54SQL Server Assertion: File: <p:\sql\ntdbms\storeng\drs\include\record.inl>, lin
2003-12-10 12:38:59.43 spid54Stack Signature for the dump is 0x2E0DE171
2003-12-10 12:38:45.51 spid54Using 'dbghelp.dll' version '4.0.5'...
2003-12-10 12:38:45.51 spid54Could not open FCB for invalid file ID 24320 in database 'DEVELOP'..
2003-12-10 12:38:45.51 spid54Error: 5180, Severity: 22, State: 1
2003-12-10 12:38:45.51 spid54SQL Server Assertion: File: <p:\sql\ntdbms\storeng\drs\include\record.inl>, lin
2003-12-10 12:38:45.50 spid54Stack Signature for the dump is 0x2E0DE171
2003-12-10 12:38:31.46 spid54Using 'dbghelp.dll' version '4.0.5'...
2003-12-10 12:38:31.39 spid54Exception in destructor of Rowset
2003-12-10 12:38:31.37 spid54Stack Signature for the dump is 0x923ABDAB
2003-12-10 12:38:30.67 spid54SqlDumpExceptionHandler: Process 54 generated fatal exception c0000005 EXCEPTIO
2003-12-10 12:38:30.67 spid54Error: 0, Severity: 19, State: 0
2003-12-10 12:38:17.15 spid54Using 'dbghelp.dll' version '4.0.5'...
2003-12-10 12:38:17.10 spid54Error: 3624, Severity: 20, State: 1.
2003-12-10 12:38:17.10 spid54SQL Server Assertion: File: <p:\sql\ntdbms\storeng\drs\include\record.inl>, lin
2003-12-10 12:38:17.10 spid54Stack Signature for the dump is 0xA84BDAC1
2003-12-10 12:37:58.98 spid54Using 'dbghelp.dll' version '4.0.5'...
2003-12-10 12:29:03.00 spid53Using 'xpstar.dll' version '2000.80.760' to execute extended stored procedure '
2003-12-10 12:28:24.42 spid51Using 'xpsqlbot.dll' version '2000.80.194' to execute extended stored procedure
And the dumpfile details;
Dump thread - spid = 60, PSS = 0x495c11b0, EC = 0x495c14d8
*Stack Dump being sent to D:\MSSQL7\log\SQLDump0008.txt
* *****************************************************************************
** * * BEGIN STACK DUMP: * 12/10/03 13:13:03 spid 60 * * * * Input Buffer 34 bytes -
* exec sp_spaceused
* * * MODULE BASE END SIZE
* sqlservr 00400000 00B2CFFF 0072d000
* ntdll 77F80000 77FFAFFF 0007b000
* KERNEL32 77E80000 77F34FFF 000b5000
* ADVAPI32 77DB0000 77E0AFFF 0005b000
* RPCRT4 77D40000 77DAFFFF 00070000
* USER32 77E10000 77E73FFF 00064000
* GDI32 77F40000 77F7BFFF 0003c000
* OPENDS60 41060000 41065FFF 00006000
* MSVCRT 78000000 78045FFF 00046000
* UMS 41070000 4107CFFF 0000d000
* SQLSORT 42AE0000 42B6FFFF 00090000
* MSVCIRT 780A0000 780B1FFF 00012000
* sqlevn70 41080000 41086FFF 00007000
* NETAPI32 7B750000 7B79EFFF 0004f000
* SECUR32 7B7A0000 7B7AEFFF 0000f000
* NETRAP 7B7B0000 7B7B5FFF 00006000
* SAMLIB 7B7C0000 7B7CFFFF 00010000
* WS2_32 7B7D0000 7B7E2FFF 00013000
* WS2HELP 7B7F0000 7B7F7FFF 00008000
* WLDAP32 7B800000 7B828FFF 00029000
* DNSAPI 7B830000 7B853FFF 00024000
* WSOCK32 7B860000 7B867FFF 00008000
* ole32 7BB50000 7BC45FFF 000f6000
* XOLEHLP 7BC50000 7BC57FFF 00008000
* MSDTCPRX 7BC60000 7BD0EFFF 000af000
* MTXCLU 7BD10000 7BD1FFFF 00010000
* VERSION 7BD20000 7BD26FFF 00007000
* LZ32 7BD30000 7BD35FFF 00006000
* CLUSAPI 7BD40000 7BD4FFFF 00010000
* RESUTILS 7BD50000 7BD5CFFF 0000d000
* USERENV 7BD60000 7BDBCFFF 0005d000
* rnr20 7BDC0000 7BDCBFFF 0000c000
* iphlpapi 7BE10000 7BE22FFF 00013000
* ICMP 7BE30000 7BE34FFF 00005000
* MPRAPI 7BE40000 7BE56FFF 00017000
* OLEAUT32 7BE60000 7BEFAFFF 0009b000
* ACTIVEDS 7BF00000 7BF2DFFF 0002e000
* ADSLDPC 7BF30000 7BF51FFF 00022000
* RTUTILS 7BF60000 7BF6DFFF 0000e000
* SETUPAPI 7BF70000 7BFFCFFF 0008d000
* RASAPI32 7C000000 7C031FFF 00032000
* RASMAN 7C040000 7C050FFF 00011000
* TAPI32 7C060000 7C081FFF 00022000
* COMCTL32 7C090000 7C118FFF 00089000
* SHLWAPI 7C120000 7C169FFF 0004a000
* DHCPCSVC 7C170000 7C188FFF 00019000
* CLBCATQ 7C5A0000 7C624FFF 00085000
* winrnr 7C6D0000 7C6D7FFF 00008000
* rasadhlp 7C6E0000 7C6E4FFF 00005000
* SSNETLIB 7CEB0000 7CEC4FFF 00015000
* SSNMPN70 410D0000 410D5FFF 00006000
* security 413E0000 413E3FFF 00004000
* msafd 41BF0000 41C0EFFF 0001f000
* wshtcpip 41C50000 41C56FFF 00007000
* SSmsLPCn 41CE0000 41CE6FFF 00007000
* ntdsapi 41D80000 41D90FFF 00011000
* SQLFTQRY 42420000 42451FFF 00032000
* sqloledb 42460000 424D2FFF 00073000
* MSDART 424E0000 424FEFFF 0001f000
* comdlg32 42510000 4254DFFF 0003e000
* SHELL32 42550000 42791FFF 00242000
* MSDATL3 427A0000 427B4FFF 00015000
* oledb32 00390000 003F5FFF 00066000
* OLEDB32R 00BF0000 00BFFFFF 00010000
* xpsqlbot 00C10000 00C15FFF 00006000
* msv1_0 01A50000 01A6DFFF 0001e000
* xpstar 01B80000 01BC6FFF 00047000
* SQLRESLD 42AC0000 42AC6FFF 00007000
* SQLSVC 01BD0000 01BE6FFF 00017000
* ODBC32 01BF0000 01C20FFF 00031000
* odbcbcp 01C30000 01C35FFF 00006000
* W95SCM 01C40000 01C4BFFF 0000c000
* SQLUNIRL 10000000 1002CFFF 0002d000
* WINSPOOL 01C50000 01C6CFFF 0001d000
* SHFOLDER 01C70000 01C77FFF 00008000
* odbcint 01DC0000 01DD5FFF 00016000
* NDDEAPI 01DE0000 01DE6FFF 00007000
* SQLSVC 01DF0000 01DF5FFF 00006000
* xpstar 01E00000 01E08FFF 00009000
* msdasql 01E20000 01E69FFF 0004a000
* MSDASQLR 01E70000 01E73FFF 00004000
* comsvcs 01F00000 02061FFF 00162000
* TxfAux 02070000 020D0FFF 00061000
* SYODASE 02260000 02288FFF 00029000
* syodutl 02290000 022DAFFF 0004b000
* syodbas 022E0000 02329FFF 0004a000
* libct 02330000 0237AFFF 0004b000
* libintl 02380000 02388FFF 00009000
* libcomn 02390000 023D8FFF 00049000
* libtcl 023E0000 023F9FFF 0001a000
* libcs 02400000 0240BFFF 0000c000
* syodutlR 02420000 02424FFF 00005000
* syodbasR 024B0000 024B7FFF 00008000
* SYODASER 024C0000 024C5FFF 00006000
* odbccp32 024D0000 024E7FFF 00018000
* msadce 02B90000 02BDAFFF 0004b000
* mswstr10 02BE0000 02C74FFF 00095000
* msadcer 03090000 03094FFF 00005000
* DBNETLIB 031B0000 031BEFFF 0000f000
* crypt32 031C0000 03234FFF 00075000
* MSASN1 03240000 0324FFFF 00010000
* SQLOLEDB 03390000 0339EFFF 0000f000
* msdaora 034F0000 03525FFF 00036000
* MSDAORAR 03530000 03533FFF 00004000
* ociw32 03570000 03578FFF 00009000
* ORACLIENT8 03580000 03685FFF 00106000
* oracore8 03690000 03715FFF 00086000
* oranls8 03720000 0376CFFF 0004d000
* WINMM 03770000 0379FFFF 00030000
* oravsn8 037A0000 037A5FFF 00006000
* oracommon8 037B0000 03814FFF 00065000
* orageneric8 03820000 03941FFF 00122000
* oranl8 03950000 0397AFFF 0002b000
* oran8 03980000 03A28FFF 000a9000
* orancrypt8 03A30000 03A3CFFF 0000d000
* oranro8 03A40000 03A76FFF 00037000
* orannzsbb8 03A80000 03AE0FFF 00061000
* oranldap8 03AF0000 03AFDFFF 0000e000
* oraldapclnt8 03B00000 03B1FFFF 00020000
* oranhost8 03B20000 03B25FFF 00006000
* oranoname8 03B30000 03B35FFF 00006000
* orancds8 03B40000 03B45FFF 00006000
* orantns8 03B50000 03B55FFF 00006000
* orannds8 03B60000 03B67FFF 00008000
* oranms 03B70000 03B76FFF 00007000
* oranmsp 03B80000 03B90FFF 00011000
* ORATRACE8 03BA0000 03BDFFFF 00040000
* orapls8 03BE0000 03E02FFF 00223000
* oraslax8 03E10000 03E17FFF 00008000
* orawtc8 03E20000 03E25FFF 00006000
* orasql8 03E30000 03EA2FFF 00073000
* orantcp8 043B0000 043BAFFF 0000b000
* mswsock 04F20000 04F31FFF 00012000
* dbghelp 02800000 028B4FFF 000b5000
*
* Edi: 495C14D8: 495C11B0 495ED2C0 495ED2C0 454AEAD8 495D9418 495D8
030
* Esi: 00000000:
* Eax: 00000000:
* Ebx: 495C11B0: 0000003C 00000000 00D9CB31 0000944B 00000000 00000
000
* Ecx: 0533D8E0: 00000000 0001003F 00000000 00000001 00120A84 00000
08C
* Edx: 00000000:
* Eip: 0087A90E: 4D830088 75FFFFFC 0875FF0C E8E04D8B 0000002D 64F04
D8B
* Ebp: 0533D8C4: 0533E0E4 0087C41F 495C11B0 0533E248 0533E4EB 78024
52E
* SegCs: 0000001B:
* EFlags: 00010246: 00500049 0044005C 0049004D 0057005C 006E0069 00320
033
* Esp: 0533D898: 495C14D8 00000000 495C11B0 0533D8E0 495C1000 0533D
898
* SegSs: 00000023:
* *****************************************************************************
**
* -----------------------------------------------------------------------------
--
* Short Stack Dump
* 0087A90E Module(sqlservr+0047A90E) (streambuf::setbuf+0011061A)
* 0087C41F Module(sqlservr+0047C41F) (streambuf::setbuf+0011212B)
* 00931097 Module(sqlservr+00531097) (streambuf::setbuf+001C6DA3)
* 005B1DA8 Module(sqlservr+001B1DA8) (CGenIndexStrategy::GenOptimalSolution+000
00023)
* 004074FF Module(sqlservr+000074FF) (TMatchPattern<unsigned short,CWCharBinary
Helper>::match_pattern+000003A7)
* 00405C0C Module(sqlservr+00005C0C) (wpattern+000000DC)
* 0042A639 Module(sqlservr+0002A639) (CnstNormalize+00000320)
* 0042A4F6 Module(sqlservr+0002A4F6) (CnstNormalize+000001DD)
* 00490503 Module(sqlservr+00090503) (CSubTextLockBytesSS::GetComplexCol+000000
12)
* 004301A9 Module(sqlservr+000301A9) (CStmtDropIndex::XretExecute+00000041)
* 0042F735 Module(sqlservr+0002F735) (CallDropObject+00000249)
* 0042F71F Module(sqlservr+0002F71F) (CallDropObject+00000233)
* 0042E20C Module(sqlservr+0002E20C) (CObject::FResolveName+00000086)
* 0042E010 Module(sqlservr+0002E010) (SetConstraintsNotTrusted+000002B7)
* 0040DEE3 Module(sqlservr+0000DEE3) (CFetchIntoVars::LinkToEc+00000005)
* 0041D472 Module(sqlservr+0001D472) (CSPModifyParams::BuildParamedSql+000002F3
)
* 00429DFF Module(sqlservr+00029DFF) (InsertBldmastrColumns+0000015D)
* 0041B442 Module(sqlservr+0001B442) (SPCursor::Option+000002F4)
* 0041AA88 Module(sqlservr+0001AA88) (SPCursor::Fetch+00000385)
* 0041B9B6 Module(sqlservr+0001B9B6) (CSPOpenParams::CheckMinParamCount+0000004
E)
* 00498A8A Module(sqlservr+00098A8A) (exp_builtin+000000BA)
* 00498926 Module(sqlservr+00098926) (exp_literal+000004FC)
* 0041B442 Module(sqlservr+0001B442) (SPCursor::Option+000002F4)
* 0041AA88 Module(sqlservr+0001AA88) (SPCursor::Fetch+00000385)
* 0041B9B6 Module(sqlservr+0001B9B6) (CSPOpenParams::CheckMinParamCount+0000004
E)
* 0048A63B Module(sqlservr+0008A63B) (MkColBitmap+00000073)
* 0042708C Module(sqlservr+0002708C) (initODS+000003E6)
* 41075002 Module(UMS+00005002) (ThreadStartRoutine+00000032)
* 41074698 Module(UMS+00004698) (UmsFiberScheduler::ConvertThread+00000018)
* 542868FF Module(UNKNOWN+00000000)
-------------------------------------------------------------------------------
* Location : 0087A90E Module(sqlservr+0047A90E) (streambuf::setbuf+0011061A)
* Return Addr: 0087C41F Module(sqlservr+0047C41F) (streambuf::setbuf+0011212B)
Frame : 0533D8C4
Parameters:
[1] 495C11B0: 0000003C 00000000 00D9CB31 0000944B 00000000 00000000
[2] 0533E248: 00000000 00000000 0098BDC0 0533E258 00000189 000091A0
[3] 0533E4EB: 4042D100 5ED2C000 5ED2C049 00000049 5ED2C000 00000049
[4] 7802452E: 83EC8B55 458B20EC 45895608 E04589E8 C710458D 0042EC45
160 bytes of stack data from 0533D824 to 0533D8C4
0533D824: 0533D898 0000001B 00000003 00000000 [..3.............]
0533D834: 495C11B0 0533E248 0533E4EB 7802452E [..\IH.3...3..E.x]
0533D844: 00000000 00000001 0533E0E4 00000000 [..........3.....]
0533D854: 00000000 00000000 00000000 00000000 [................]
:::: 1 Duplicate line detected
0533D874: 00000004 6FAC1FFF 0533D8B4 009825D3 [.......o..3..%..]
0533D884: 00000000 0533D8C4 0087A922 00000000 [......3.".......]
0533D894: 0533D824 495C14D8 00000000 495C11B0 [$.3...\I......\I]
0533D8A4: 0533D8E0 495C1000 0533D898 0533D4E4 [..3...\I..3...3.]
0533D8B4: 0533E0D8 00488754 009B88F0 FFFFFFFF [..3.T.H.........]
If anyone could help it would be most appreciated.
Thanks in advance.
j
-
did you run a checkdb in database DEVELOP?
-
Hi racosta,
yep, ran checkdb and got this;
Server: Msg 8966, Level 16, State 1, Line 1
Could not read and latch page (1:1490) with latch type SH. sysindexes failed.
DBCC execution completed.
And we have a similar problem with a couple of our db's but checkdb(repair_rebuild) * (repair_allow_data_loss) did not fix them !
help!
j
-
I have to run checkdb with the repair_Rebuild option a couple of times, and usually to repair a database, have to execute it several times (sometimes at least 5).
If there is no way you can repair the db, then check in the output of checkdb the tables or indexes that are having problems. If possible, maybe you can re create them.
Also you need to know why the db got wroken. Maybe you should check previous sql logs to find the cause of the error in the db.
-
Hi racosta,
Can't run repair_rebuild at the moment as we have users in the system, so i will have to wait until later for that.
Original cause is not known but we did have a power outtage a week ago and the sqlserver has not been the same since. we were told the server was shut down correctly, but i have my doubts if it was.
j
-
Go back and check the SQL Server logs from the time of the power outage and you can verify if SQL Server was shutdown gracefully.
K. Brian Kelley, GSEC
http://www.truthsolutions.com/
Author: Start to Finish Guide to SQL Server Performance Monitoring
K. Brian Kelley
@kbriankelley -
I assume you meant NT5 sp2 and not NT4 sp2, or?
Unless this isn't a typo, no matter what this issue is, you should really consider to apply some more recent SPs for NT4.
Where are the South Sandwich Islands?
Frank
--
Frank Kalis
Microsoft SQL Server MVP
Webmaster: http://www.insidesql.org/blogs
My blog: http://www.insidesql.org/blogs/frankkalis/[/url] -
Hi,
Ok, we've fixed the error now, thanks for pointing me in the right direction. I shall try to summarise for you;
Originally our first error was a failure when users were querying, they were receiving a message about 'latch' problems !.
So I ran the checkdb command and received error messages, with some errors on system tables in the db's.
Then we started to get the error messages in our errorlog that I first posted and eventually ( after reading the posts here and some investigation ) we linked it to the corruption of the system tables within our corrupted db's. Basically what was happening was as soon as some system stored procs ran we saw the 'assertion file' error in the logs and thereafter the 'scheduler hung' errors, so something as simple as clicking on the properties tab of the db in enterprise manager runs some system stored procs ( sp_spaceused etc etc ) and caused the 'assertion file' errors.
So, we tried to fix the db's but everytime the server tried to access the system tables in the corrupted db's it discovered that they too were corrupted and just failed.
Result was we had to restore to previous backups and so far we've seen none of the same errors in our logs.
So, many thanks for point us in the right direction, bkelley you were spot on with the server shutdown & racosta you were spot on with the checkdb.
Bit disappointed that Microsoft would allow minor corruption of a db's system tables to escalate to a point where our server either could not function or simply hung/crashed.
Regards.
j
p.s Frank it was a typo and should've been NT5 although the patch level was correct !! - oh and south sandwich islands are located here http://www.cia.gov/cia/publications/factbook/geos/sx.html
Viewing 8 posts - 1 through 7 (of 7 total)
You must be logged in to reply to this topic. Login to reply