1. Are you using a common AD account ?

2. Have you checked your firewall policy ? try and telnet port 5022

I have experienced this same issue intermittently on few of our servers. I agree with Elliott, this is definitely due to I/O and network bottlenecks.

If you have the luxury...