Have you checked in the repsonse tab that it isn't running the job as a response to the alert being raised ?

I had this problem where through Management Studio I...