What's Worse than Announcing a Data Breach? You might think that's the worst thing that you could tell your boss these days. Imagine discovering that sensitive data has been exposed and you need to go inform an executive. Hopefully no one is looking to punish the messenger, but it will still be a very uncomfortable conversation.
Now imagine that you need to go back and have a second discussion a day or two later. Why? More data was exposed, and potentially lost. I'm not sure which conversation is worse, though if you don't have a list of things you've checked, changed, or fixed between the two meetings, I would argue the second one is worse.
That happened to a mortgage loan company. The data breach was already bad, with an Elastisearch server out there without any security. Data had been converted from paper documents through an OCR process, which resulted in no shortage of mistakes, but there was still plenty of sensitive information out there. Things got worse when security researchers discovered the source of the OCR process was an Amazon S3 bucket that container the original images, also without a password.
Before I comment on anything else, there should be NO shares, buckets, containers, databases, or any sort of server without a password. None, nada, zip, zilch, no excuses. At least protect everything with a password that meets your organizations password requirements. No "12345" or "asdf" or default passwords. Before you do anything else, go set passwords. If you have S3 buckets or Azure storage, write a script to check them all. Set. A. Password. On. Everything.
We will all make mistakes in configuration, and there might be security issues at times. These ought to be rare, with today's vulnerabilities scanners, static code analysis, configuration as code, global policies, etc. There really isn't any excuse why we don't set things up at the beginning, but if things change and mistakes are made, we ought to detect them quickly. And then fix them. That's why we should use automation, configuration as code, and regular evaluation of our systems.
One of the greatest strengths of the DevOps philosophies is that we can deploy changes quickly to fix things. We'll make mistakes, we'll have issues, but when we find them, there is no long waiting period to get the fix into our client's hands. That ought to be true for both software and infrastructure.