Think your office is secure? You have guards during the day checking badges, you have key card systems to control and audit access and alarms set at night. You don't worry if you have various system names or passwords scribbled on scratch paper. After all, who would target your office?
You never know. With the large number of people working in technology, there are bound to be a few people with less than prefect morals. The turnover of employees in technology can be high, and it might not be as easy to recognize an outsider as you think. Even in companies that only have 50 or so employees, it's easy to assume the person you see in the company is new hire, contractor, or other individual that has reason to be there.
Especially if that person gets into your locked server room because they've hacked the alarm or key card system. At the Black Hat conference, hackers demonstrated how they can get past many alarm systems and talked about their methods for cloning RFID cards by walking near someone. The fact that these technologies are available, relatively cheap, and can be implemented by people that have some knowledge of the systems means we have to be more careful than ever about the physical security of our systems.
Be aware of the authorized users that can physically access systems. Ensure that you don't have passwords or other critical information easily available, and encrypt your systems. Even if someone breaks into your data center, they won't want to spend a lot of time there. If someone can easily get information, or copy files, they will. Adding a few security hurdles to bypass might mean the difference between keeping control of your data and losing it.