Server Not Responding

Question

Post reply

Server Not Responding

VastSQL

SSC-Dedicated

Points: 36097
More actions
December 21, 2017 at 12:23 am

#409883

Hi Experts,
The active server in 2 Node Active\Passive Cluster having another Always On AG node was not responding till we restart the server.
SQL server didnt failover to another node . DBAs observed the issue when they received below SCOM alerts.
1.An error occurred during recovery, preventing the database 'ABCCorp' (14:0) from restarting. Diagnose the recovery errors and fix them, or restore from a known good backup. If errors are not corrected or expected, contact Technical Support.
2.description: fcb::close-flush: Operating systemerror (null) encountered
3.The operating system returned error 170(The requested resource is in use.) to SQL Server during a read at offset 0000000000000000 in file 'I:\Log\ABCCorp_NEW_01162017_log.ldf'. Additional messages in the SQL Server error log and system event log may provide more detail. This is a severe system-level error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online.

Got below error from Eventviewer.
Cluster service failed to update the cluster configuration data on the witness resource. Please ensure that the witness resource is online and accessible.
The cluster service detected a problem with the witness resource. The witness resource will be failed over to another node within the cluster in an attempt to reestablish access to cluster configuration data.
The system failed to flush data to the transaction log. Corruption may occur in VolumeId: Q:, DeviceName: \Device\HarddiskVolume130.
({Device Busy}
The device is currently busy.)
Cluster resource 'Quorum' of type 'Physical Disk' in clustered role 'Cluster Group' failed.
Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it. Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet.
The system failed to flush data to the transaction log. Corruption may occur in VolumeId: J:, DeviceName: \Device\HarddiskVolume137.
({Device Busy}
The device is currently busy.)
I tried to use handle to figure out if the file is held by any other process but unfortunately the handle file was also through resource in use error.
Please Help.

Viewing 7 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic. Login to reply

VastSQL SSC-Dedicated Points: 36097 More actions · Answer 1

VastSQL

SSC-Dedicated

Points: 36097

December 26, 2017 at 12:30 am

#1973397

Experts..Any idea on what went wrong?

Steve Jones - SSC Editor SSC Guru Points: 736212 More actions · Answer 2

Have you run a DBCC CheckDB? I'd be concerned about corruption here

Jeff Moden SSC Guru Points: 1004339 More actions · Answer 3

Before that, have you tried accessing any of the drives that have been reported as a problem? I'm not an expert here but it sounds like a disk failure.

--Jeff Moden

RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
First step towards the paradigm shift of writing Set Based code:
________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

Change is inevitable... Change for the better is not.

Helpful Links:
How to post code problems
How to Post Performance Problems
Create a Tally Function (fnTally)

VastSQL SSC-Dedicated Points: 36097 More actions · Answer 4

Steve Jones - SSC Editor - Tuesday, December 26, 2017 8:15 AM
Have you run a DBCC CheckDB? I'd be concerned about corruption here

Thanks Steve for the reply.

Databases was inaccessible and later did a failover to Availability Group. Ran CheckDB in all databases and everything looks healthy.

VastSQL SSC-Dedicated Points: 36097 More actions · Answer 5

Jeff Moden - Tuesday, December 26, 2017 11:04 AM
Before that, have you tried accessing any of the drives that have been reported as a problem? I'm not an expert here but it sounds like a disk failure.

Thanks Jeff.

That was the first thing i did . Check all disk ,was expecting disk offline but all disk were online . I am suspecting Antivirus but the AV team said nothing was there in their log.

MMartin1 One Orange Chip Points: 27795 More actions · Answer 6

Do you know if something was thrashing one of the disks? That is, some process doing some intense read or write? An anti-virus would make sense. If you approach the server and it has a disk drive you may hear the strong sound of a disk spinning at 100%.

----------------------------------------------------