December 17, 2014 at 6:26 am
I produce a quarterly report that compares the prior 12 months with the same 12 months a year ago. It is done relationally, not multi-dimensionally.
In the current cycle I noticed that the prior period results were larger than the current period results for the same swath of history, which is not reasonable because more claims got paid, so something is rotten in Denmark.
Another untoward event is that I have been hammering on SSRS over the past six weeks, yet, I noticed that the Reporting Server backups have been getting smaller, not bigger.
This makes me suspect that there is some kind of data loss happening, but I don't know where to look to prove it. When I do my ETL, I create a control total in addition to the data for import, but my predecessor did not collect control totals, so I cant be certain of what is correct for the past. I can prove that I have loaded what was sent since I started there, but I don't have a standard available for the past.
1)the current values are SMALLER than the second quarter totals, and we know from the control totals that that cannot be true.
2)The Reporting Server is not showing deployed reports we have already reviewed, and the new deployments suddenly not showing data.
3)The backups of the Reporting Server are getting SMALLER (see screen shot followed by Event View logs below).
The error logs report the following, yet, the disk seems to have plenty of free space on it.
Here's what I found in event viewer, there is nothing remarkable in the SQL Server logs.
Am I losing data? My backup chain only goes a month deep, so I don't know yet if restore saves my bacon. I do have the starting text files, so it is possible to reload everything into a test db and rerun the report queries against them. The networking resources say connectivity and network are fine, and disk is 15% free and no other evidence of hardware failure.
What would you do? I'm freaking out.
Thanks a lot
Windows cannot access the file C:\Program Files\Microsoft SQL Server\100\Setup Bootstrap\Update Cache\KB2979597\ServicePack\x64\fr\microsoft.sqlserver.connectioninfo.resources.dll for one of the following reasons: there is a problem with the network connection, the disk that the file is stored on, or the storage drivers installed on this computer; or the disk is missing. Windows closed the program Windows host process (Rundll32) because of this error.
Program: Windows host process (Rundll32)
File: C:\Program Files\Microsoft SQL Server\100\Setup Bootstrap\Update Cache\KB2979597\ServicePack\x64\fr\microsoft.sqlserver.connectioninfo.resources.dll
The error value is listed in the Additional Data section.
User Action
1. Open the file again. This situation might be a temporary problem that corrects itself when the program runs again.
2. If the file still cannot be accessed and
- It is on the network, your network administrator should verify that there is not a problem with the network and that the server can be contacted.
- It is on a removable disk, for example, a floppy disk or CD-ROM, verify that the disk is fully inserted into the computer.
3. Check and repair the file system by running CHKDSK. To run CHKDSK, click Start, click Run, type CMD, and then click OK. At the command prompt, type CHKDSK /F, and then press ENTER.
4. If the problem persists, restore the file from a backup copy.
5. Determine whether other files on the same disk can be opened. If not, the disk might be damaged. If it is a hard disk, contact your administrator or computer hardware vendor for further assistance.
Additional Data
Error value: C0000010
Disk type: 3
Faulting application name: rundll32.exe_aepdu.dll, version: 6.1.7600.16385, time stamp: 0x4a5bc9e0
Faulting module name: aeinv.dll, version: 6.1.7601.17514, time stamp: 0x4ce7c45b
Exception code: 0xc0000006
Fault offset: 0x0000000000051811
Faulting process id: 0x21fc
Faulting application start time: 0x01d018f155fc5d3e
Faulting application path: C:\Windows\system32\rundll32.exe
Faulting module path: C:\Windows\system32\aeinv.dll
Report Id: 1edd8144-84e5-11e4-9a09-00237dd5c7ec
Faulting application name: rundll32.exe_aepdu.dll, version: 6.1.7600.16385, time stamp: 0x4a5bc9e0
Faulting module name: aeinv.dll, version: 6.1.7601.17514, time stamp: 0x4ce7c45b
Exception code: 0xc0000006
Fault offset: 0x0000000000051811
Faulting process id: 0x21fc
Faulting application start time: 0x01d018f155fc5d3e
Faulting application path: C:\Windows\system32\rundll32.exe
Faulting module path: C:\Windows\system32\aeinv.dll
Report Id: 1edd8144-84e5-11e4-9a09-00237dd5c7ec
December 19, 2014 at 3:06 pm
run checkdb on your databases!
December 30, 2014 at 6:10 pm
This certainly looks like a Windows problem on a disk. As the error message states, perhaps a bad disk or maybe just a bad sector.
First, verify that the files the errors are citing are actually there and then do the ChkDsk thing that it recommends. Be advised that could take some time and your system will be unavailable during that time but I don't see away around it unless you have a mirror that you can roll over to.
--Jeff Moden
Change is inevitable... Change for the better is not.
December 30, 2014 at 9:20 pm
Thank you both.
dbcc checkdb ran on all dbs without error, but pharmacy queries still did not agree with controls.
I felt I had no choice but to redo the ETL and load the pharmacy data from scratch. After that, the queries agreed with controls. For the uninitiated, control totals are an advice from the data source that tell you the number of records and the summations of billed, allowed and paid amounts, so data agreeing with control totals seems in hand.
The gorilla in the room is I still have to rely on a widgy disk to do the backups...I back the whole megillah up as full backup to the questionable disk and then copy the BAKs to a 2 TB external drive I picked up at Costco, then copy those BAKs to my local (storage is not mirrored, but the dbs are small enough to get a week's worth onto the drive) and restore them from there. So its a pita right now, but control still matches the queries and the SSRS reports run.
I haven't done check disk on the questionable drive. I have no idea what the consequences of running it will be but have to admit it raises some anxiety. I can try that out the day after tomorrow when no one is around, and if something else dies or data is lost during the operation, at least I have the data moved off in two places (one on my local and one on the external drive), so I will have to keep everything crossed.
Thanks for your help
January 1, 2015 at 1:10 am
Quick suggestion, check the self monitoring data (SMART) of the hard drive,a non-intrusive operation. Utility for this can be found here www.smartmontools.org
😎
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply