I used to have an office. I guess I have an office now, but it's not mine; it's shared with my wife. As I look back at my career, I realize that I've often taken a job that switched me from a more open working environment to an office, and back. Early in my career, we had cubes, which a few of us combined into an open 4 desk space at one point. Then I took a job with an office, then back to an open plan (10 developers in a room), then my own office, before ending up with my shared office. Actually, I'm always sharing a space. When I travel to the Redgate offices, they have an open plan, so I get a desk that's open to a few dozen other people.
I've gone back and forth on whether or not I want a shared space. I've come to the conclusion that it really depends on my job. When I worked as a sysadmin, where we were constantly collaborating with others, an open, shared space worked well. We could easily talk to each other, which was an efficient way to work.
When I've worked as a developer or a manager, I've preferred having my own office. The ability to concentrate for longer periods of time is helpful for me, and while headphones separate me, they aren't as private, as there can still be visual distractions as people move around. I've had people come up to tap me on the shoulder, which can ruin my concentration at times.
There's an article at Ars Technica that talks about a study showing that open plans result in much less productivity and even less interaction. That seems counterintuitive, as I'd expect an open plan to encourage more interaction. Indeed, when I'm at the Redgate office, I find that someone will often turn and chat with a friend. Or if two people are talking, it can become a watercooler atmosphere where others join in. There seem to be plenty of interaction.
When I've been at Microsoft, where everyone has a small office and most with a window to the hallway, I find lots of open and closed doors, with the former inviting others in and the latter discouraging it. I do see plenty of workers get distracted if someone stops and looks in the window. At the same time, I also find there are lots of open spaces where people can engage with others and even work in an open space if they desire.
I don't know if one way is better or worse than the other. I know that I feel differently at different times about how I work. At Redgate, we do have some pods for private work if you choose, but they aren't completely closed off. They're more like a cube or a small space for one person. I do find that having a private space is important for me at times, and if I were to go back to a company's location on a regular basis, I'd want to ensure I had some sort of private office, even if it were only for a portion of a week and others used it when I wasn't there.