July 8, 2015 at 2:35 am
Hi,
Yesterday I ran into a problem that some databases on my Secondary Server were in Recovery Pending mode. I solved this by kicking the database out of the AAG Group, and then add it again from the Primary.
How could this have happened? I did do a reboot of both servers just before, so I assume that had something to do with it.
I did:
1) failover from DB01 (Primary at that time, becoming Secondary) to DB02 (using Management Studio)
2) reboot DB01
3) failover back to DB01 (Secondary at that time, becoming Primary) from DB02
4) reboot DB02
After this some databases (4 of approx. 100) were in Recovery Pending mode....
July 8, 2015 at 3:45 am
was the instance still present as a replica in the AG?
was the database still listed as joined to the AG?
-----------------------------------------------------------------------------------------------------------
"Ya can't make an omelette without breaking just a few eggs" 😉
July 8, 2015 at 3:54 am
yes and yes
I have a gut feeling I must reboot differently. I can't find on Google anywhere best practise how to reboot a clustered SQL sytem
July 8, 2015 at 4:10 am
How did you perform the FO using the wizard in SSMS or the cluster manager ?
Recovery pending is not the same as bringing the databases to sync.
How long did you wait before you kicked the databases out of the AG. Did you review the error logs to see how far along the recovery was ?
July 8, 2015 at 4:24 am
Hey, what I did, using the SSMS, was:
Connect to Server DB01 and navigate to AlwaysOn High Availability | Availability Groups | AAG-MyGroup (Primary)
Right-click, Select Failover
Check that Failover Readiness is "No data loss"
Follow the wizard and perform the failover to DB02, without errors
After the operation, when seeing AAG-MyGroup (Secondary), rebooting the DB01 server.
Reboot done by Windows Restart; so I didn't manually shutdown SQL service or anything alike.
Once the server DB01 was up again basically do the same operation but then failover the other way around, so that DB01 is primary and DB02 is secondary again. Then reboot DB02. Once DB02 was up again, and that showed in AAG-MyGroup Dashboard, those databases on DB02 were in Recovery Pending mode.
Thanks!
July 8, 2015 at 4:38 am
July 8, 2015 at 4:48 am
Do you remember if the status of the databases were Synchronized or Synchronizing before you performed the seconda FO ?
My guess is Synchronizing.... I'm kinda new to AAG so I have to learn what to watch for. As we reboot once a week I will keep a close eye on Synchronized / Synchronizing.
Thx!
July 8, 2015 at 5:07 am
Jayanth_Kurup (7/8/2015)
I cant remember if a windows shutdown cleanly shutsdown SQL Server Services but i guess it should.
Yes it will
-----------------------------------------------------------------------------------------------------------
"Ya can't make an omelette without breaking just a few eggs" 😉
July 11, 2015 at 8:46 am
Synchronizing means that records are being moved around and havent been hardend in the log. This would result in recovery of the databases since redo undo phase is applicable.
Typically network latency or unexpected shutdown might cause a replica to go out of sync with the primary but given enough time they will eventually get back in sync.
Assuming the first shutdown cause the sync to break the second shutdown might aggrevate the process since both system are now in recovery so to speak
Viewing 9 posts - 1 through 8 (of 8 total)
You must be logged in to reply to this topic. Login to reply