Log Message: IO Completion Listener non-yielding

Question

Log Message: IO Completion Listener non-yielding

Viewing 9 posts - 16 through 23 (of 23 total)

You must be logged in to reply to this topic. Login to reply

Erwin Dockx SSC Eights! Points: 852 More actions · Answer 1

We are no longer experiencing problems:

We're not using the memory tweak, but HP has updated the firmware. This fixed the problem and the error hasn't occured againg in the last 3 months. You can find more information here:

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&objectID=c00688313&jumpid=reg_R1002_USEN

This article may also apply:

http://support.microsoft.com/default.aspx?scid=kb%3Ben-us%3B810885

Greetz

_____________________________________________________
Do not go past the mark you aimed for, but learn when to stop.

You can find me on LinkedIn.
I support The Programmer's Bill of Rights.

MCITP, MCDBA, MCSD

audunj Valued Member Points: 73 More actions · Answer 2

Thanks for this.

Do you know if you updated to version 1.10.xxxx or to version 1.9.xxxx as referenced in this article?

According to our hosting provider we are already running on version 1.9.xxxx.

Thanks,

Erwin Dockx SSC Eights! Points: 852 More actions · Answer 3

I'll have to ask the sysadmin about that info... I'll post it in here when I get his reply.

_____________________________________________________
Do not go past the mark you aimed for, but learn when to stop.

You can find me on LinkedIn.
I support The Programmer's Bill of Rights.

MCITP, MCDBA, MCSD

Erwin Dockx SSC Eights! Points: 852 More actions · Answer 4

I had to get the info myself en for what I can see, the firmware of the iLo hasn't been upgraded to 1.9 or above, it's at version 1.8.3790.0. From what I've heard from another colleague, HP has upgraded the firmware of the NIC. But I can't help you with the specifics for that unfortunately.

_____________________________________________________
Do not go past the mark you aimed for, but learn when to stop.

You can find me on LinkedIn.
I support The Programmer's Bill of Rights.

MCITP, MCDBA, MCSD

audunj Valued Member Points: 73 More actions · Answer 5

audunj

Valued Member

Points: 73

April 16, 2007 at 3:17 am

#700638

Ok, thanks for your help on this issue.

alen teplitsky SSC-Dedicated Points: 30014 More actions · Answer 6

we are on sql 2005 SP1 and getting this error. Opened a case with PSS. Many possibile causes.

We did the following:

upgraded the drivers and some software for our Emulex HBA's that link to the EMC SAN

moved some small databases that hold logs and business rules to another server from the main production servers. since there was no room for separate volumes for different db files we had multiple db files on the same volume

removed some old backup software we were testing. since the software put's iteself in the I/O path it can be a problem. other culprits are AV software.

we still get these once in a while, but only during alter index or gathering index frag statistics. also seen them come up a few times when someone with MS Access causes blocking.

MS says it can be caused by hardware, software, blocking and there is a bug fixed in build 3161 where the checkpoint process can cause these

voron999 SSC Eights! Points: 836 More actions · Answer 7

Bringing this topic back...

We've migrated and running on ESX-based VM for this particular SQL instance.

So far OK.

Discount for high known IO latencies that we still need to fix (some LUN moves in order).

First serious issue on the VM - today at about 9:00am the SQL instance ran out of memory.

As a result, even transaction backups set for 9:00 failed to execute.

By 9:30 things recovered without any intervention (we simply did not know what to do..).

Free RAM back to normal 1.6GB.

First simptoms appear at about 8:19 - memory dump with error "Non-yielding IOCP Listener" and "IO Completion Listener (0x59b8) Worker 0x0084E0E8 appears to be non-yielding on Node 0. Approx CPU Used: kernel 0 ms<c/> user 0 ms<c/> Interval: 15156.,"

Attaching the log.

It maybe that some rogue Exchange server is competing for the same physical IO that our SQL instance is using.

I think that the general issue is that underlying IO was not able to keep up with the SQL-issued IO requests.

Last night we moved the tempdb onto is own IO channel.

But the latency on it after the move is about 180ms (vs. 10-20 ideally).

So this alone could be a problem.

Thanks for any ideas.

OomBoom Hall of Fame Points: 3481 More actions · Answer 8

Hi

I've had the same problem and it was caused by the dbcc checkdb that hung on the log file not being able to write for some reason. I am unable to investigate because it only happened 2 times, but what I could gather in the short time I had to trouble shoot, it had something to do with the backup procedure that runs dbcc checkdb. I would suggest checking the other locks on the system at that time. This should tel you what the actual problem / lock is.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Do not reinvent the wheel.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

voron999 SSC Eights! Points: 836 More actions · Answer 9

No, dbcc checkdb is not at fault in my case.

I was not running it at the time.

But I maybe found a possibility - other external app could be jamming the VM (antivirus, sys backup, etc).

Typical Working Set for this SQL instance is 1600MB.

Just minutes before the jamming, it was only 700MB (per my logs).

At the same time free RAM was about 160MB (typically around 1200-1500MB).

So SQL instance could not aquire any more RAM because it was used up by something else.

Sent email to net admins asking about any other procs running at the same time - no reply.. 🙂

Still though.. Similar incidents happend before, but never as severe.

SQL would just run slow, but never "took a dump".

The total "physical" RAM on this VM is 4GB.

Just default config, no /3GB switches and the such are used.