Beware the Small Things

  • Comments posted to this topic are about the item Beware the Small Things

  • Years ago we had construction going on to update our UPS and power for our Data Center. One business day our entire Data Center went down. Come to find out a painter accidentally hit the emergency power off button for the Data Center. It took us about 4 hours to get everythign back up and running. The week after that a box was put over the Emergency Power button to avoid accidental power downs.

    With more new systems we have here now I cannot imagine how long it would take us to bring everything back up from an incident like that if it were to happen today.

  • Markus (12/29/2014)


    Years ago we had construction going on to update our UPS and power for our Data Center. One business day our entire Data Center went down. Come to find out a painter accidentally hit the emergency power off button for the Data Center. It took us about 4 hours to get everythign back up and running. The week after that a box was put over the Emergency Power button to avoid accidental power downs. ...

    Same thing happened to us, only in our case it was the halon discharge button that was positioned just right to be hit by the opening computer room door. It, too, was by a clear acrylic cover. Fortunately when the computer room manager accidentally hit it he was able to hit the abort before it discharged.

    When I was at the police department we had a pretty impressive backup: a room that had two walls lined with lead acid batteries that were about 2' tall and about 18x12" wide and deep. The techs tested them with a hygrometer faithfully. I think it was over a summer weekend that we had a big storm and lost the mains, a short time later they found out that after the last time that the generator was serviced that it hadn't been switched back in to the circuit, and the entire data center went dark.

    That was not a good day.

    -----
    [font="Arial"]Knowledge is of two kinds. We know a subject ourselves or we know where we can find information upon it. --Samuel Johnson[/font]

  • Some places only have certain systems on generator backup and one place I worked immediately lost power when road works cut through the main powerline to the business estate. A lot of people lost a mornings work as they did not save their work even occasionally. One person hadn't saved all week.

    Sometimes what you are responsible for backing up is small too i.e. your own work until it makes it to a central repository with redundancy supplied as part of the service!!!

    Gaz

    -- Stop your grinnin' and drop your linen...they're everywhere!!!

Viewing 4 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic. Login to reply