SQL Mirroring didn't failover

  • Hey All,

    I was wondering if anyone came across the following issue...

    We have mirroring setup with automatic failover. Last night our Principal server restarted (12:15 am), but failover never occurred. We found errors right before the Principal server restarted, but don't see then after the server came back on line.

    Thanks for any help!

    SQL Server Ent SP 3 *I found a MS article about some of the error message, but we already have the latest sp installed on our database servers.

    We found the following in SQL Server error log messages on the Principal Server:

    *read from bottom up.

    09/22/2010 00:15:30,Server,Unknown,SQL Server is terminating because of a system shutdown. This is an informational message only. No user action is required.

    09/22/2010 00:12:49,spid68,Unknown,Autogrow of file 'Log' in database 'Testdb' was cancelled by user or timed out after 1875 milliseconds. Use ALTER DATABASE to set a smaller FILEGROWTH value for this file or to explicitly set a new file size.

    09/22/2010 00:08:04,spid28s,Unknown,Database mirroring is inactive for database 'Testdb'. This is an informational message only. No user action is required.

    09/22/2010 00:08:04,spid29s,Unknown,The mirroring connection to "TCP://remoteserver:5022" has timed out for database "Testdb" after 10 seconds without a response. Check the service and network connections.

    09/22/2010 00:08:04,spid29s,Unknown,Error: 1479<c/> Severity: 16<c/> State: 2.

    09/22/2010 00:06:17,spid67,Unknown,Autogrow of file 'Log' in database 'Testdb' was cancelled by user or timed out after 9515 milliseconds. Use ALTER DATABASE to set a smaller FILEGROWTH value for this file or to explicitly set a new file size.

    We found the following in SQL Server error log messages on the Mirror Server:

    *read from bottom up.

    09/22/2010 00:15:43,spid20s,Unknown,Database mirroring connection error 4 'An error occurred while receiving data: '10054(An existing connection was forcibly closed by the remote host.)'.' for 'TCP://remoteserver:5022'.

    09/22/2010 00:15:43,spid20s,Unknown,Error: 1474<c/> Severity: 16<c/> State: 1.

    09/22/2010 00:15:43,spid20s,Unknown,Database mirroring connection error 4 'An error occurred while receiving data: '64(The specified network name is no longer available.)'.' for 'TCP://remoteserver:5022'.

    09/22/2010 00:15:43,spid20s,Unknown,Error: 1474<c/> Severity: 16<c/> State: 1.

    09/22/2010 00:08:06,spid28s,Unknown,Bypassing recovery for database 'Testdb' because it is marked as a mirror database<c/> which cannot be recovered. This is an informational message only. No user action is required.

    09/22/2010 00:08:05,spid28s,Unknown,Starting up database 'Testdb'.

    09/22/2010 00:08:05,spid28s,Unknown,SQL Server has encountered 1 occurrence(s) of cachestore flush for the 'Bound Trees' cachestore (part of plan cache) due to some database maintenance or reconfigure operations.

    09/22/2010 00:08:05,spid28s,Unknown,SQL Server has encountered 1 occurrence(s) of cachestore flush for the 'SQL Plans' cachestore (part of plan cache) due to some database maintenance or reconfigure operations.

    09/22/2010 00:08:05,spid28s,Unknown,SQL Server has encountered 1 occurrence(s) of cachestore flush for the 'Object Plans' cachestore (part of plan cache) due to some database maintenance or reconfigure operations.

    09/22/2010 00:08:05,spid24s,Unknown,Database mirroring is inactive for database 'Testdb'. This is an informational message only. No user action is required.

  • What about the witness log?

    It looks like the mirroring was broken a few minutes before the principal reboot, that explains why there was no failover when the principal rebooted (mirroring already dead), but not why it did not failover on network failure.

    Now if for some reason the principal lost connection with the mirror and the witness at the same time, the mirroring would be broken from the Principal point of view.

    If the mirror lost connection with the witness and the Principal, the mirroring would also be broken from it's point of view.

    If something happen to the principal after that, there would be no failover.

    Other solution is that you have no witness, then there would be no automatic failover 🙂

  • bluefox,

    Is it [testdb] that is mirrored? What happened that caused the autogrow to fail?

    What is the growth increment on the tlog? How much space is left on the drive it is on?

    -Dan B

  • Thanks for your reply SSCRookie.

    We do have a witness server setup. Here's the log entries from the 12 am hour. There weren't any further entries about the mirroring connection before or after 12am. -Thanks

    09/22/2010 00:15:43,spid19s,Unknown,Database mirroring connection error 4 'An error occurred while receiving data: '64(The specified network name is no longer available.)'.' for 'TCP://remoteserver:5022'.

    09/22/2010 00:15:43,spid19s,Unknown,Error: 1474<c/> Severity: 16<c/> State: 1.

    09/22/2010 00:15:43,spid19s,Unknown,Database mirroring connection error 4 'An error occurred while receiving data: '64(The specified network name is no longer available.)'.' for 'TCP://remoteserver:5022'.

    09/22/2010 00:15:43,spid19s,Unknown,Error: 1474<c/> Severity: 16<c/> State: 1.

    09/22/2010 00:00:15,spid18s,Unknown,This instance of SQL Server has been using a process ID of 508 since

    9/21/2010 6:31:58 AM (local) 9/21/2010 10:31:58 AM (UTC). This is an informational message only; no user action is required.

    09/21/2010 07:28:11,spid52,Unknown,Using 'xpstar90.dll' version '2005.90.4035' to execute extended stored procedure 'xp_instance_regread'. This is an informational message only; no user action is required.

    09/21/2010 07:28:11,spid52,Unknown,Using 'xpsqlbot.dll' version '2005.90.4035' to execute extended stored procedure 'xp_qv'. This is an informational message only; no user action is required.

    09/21/2010 07:28:11,spid52,Unknown,Configuration option 'show advanced options' changed from 1 to 0. Run the RECONFIGURE statement to install.

  • skrilla99,

    1. testdb is the mirrored database.

    2. I don't know the cause for the autogrowth to fail. *looking for errors

    3. growth increment: autogrow by 200MB, restricted growth to 25651 MB

    4. Currently there is 113,683 MB or 79.6% of free space left on the drive the database is on.

    Thanks!

  • Hi,

    I've fallen into similar issue. Principal (which is actually windows failover cluster) crashed as usual but this time, mirror just noticed it, tried to become principal but with no luck:

    25.3.2013 1:03:20 The mirroring connection to "TCP://principal:5022" has timed out for database "purpleDB" after 300 seconds without a response. Check the service and network connections.

    25.3.2013 1:03:20 Database mirroring is inactive for database 'purpleDB'. This is an informational message only. No user action is required.

    25.3.2013 1:03:47 Starting up database 'purpleDB'.

    and finally:

    25.3.2013 1:03:57 Bypassing recovery for database 'purpleDB' because it is marked as a mirror database, which cannot be recovered. This is an informational message only. No user action is required.

    Thanks for any info, tips..

    Pavel

Viewing 6 posts - 1 through 5 (of 5 total)

You must be logged in to reply to this topic. Login to reply