June 7, 2021 at 12:00 am
Comments posted to this topic are about the item The Tech Blame Game
June 7, 2021 at 2:19 pm
Years ago, while I was on vacation, code red hit our business. The Data Warehouse group I was in was separate from the ERP group. I had installed the latest service pack, and our servers were protected.
So although all the ERP and other IT servers were affected, there was no talk of trying to blame someone. All I remember was there was a lot of work driven towards insuring everyone tested and installed service packs in a timely manner.
Our DW was a lights out operation, with checks and balances throughout. Even when the ERP had a problem, energy was directed on how to detect this and alert the right person, not place blame. Placing blame on a person or group just causes more harm and fixes nothing.
For someone to place blame on a single person below them, they need to ask themself why they did not see this as a problem? They too should share the blame - it is their ultimate responsibility in ensuring the checks and balances are in place.
In our group, the common thought was if someone got hit by a bus, how do we insure business continues without interruption?
Single point of failure reminds me of the people who think they can insure there job security by being the only one that knows how to do things. Many of them were the first to be victims in any reorganization - I guess those above sensed the problem.
June 7, 2021 at 2:51 pm
I'd like to think this is how most organizations think, but I'm amazed at the number of high profile incidents blamed on a low level worker. That doesn't typically happen, at least, in my experience. Certainly someone low level might get fired, or blamed internally, but the gall of a CEO to say an intern or sysadmin caused millions of dollars in loss is crazy
June 7, 2021 at 3:49 pm
IMHO, the only thing that has changed in the "blame game" or even the "tech blame game" is that a whole lot more of it is seen by the public thanks to the internet. When I got out of the Navy, I started work in a fortune 500 company back in the '80's... even then, the "blame game" was rampant and not just limited to the "board rooms".
Yep... some of the problems themselves and how public they've become have changed because the tech has, but the "blame game" hasn't.
--Jeff Moden
Change is inevitable... Change for the better is not.
June 7, 2021 at 5:34 pm
Think about how many times code doesn’t work as expected. Many times the blame can be to not fleshing out all the requirements. Users tend to blame the programmer. In the end, they are as responsible as the Business Analyst to have made sure this happened.
When a CEO singles out an intern, just think of how many layers between the 2 have a share in what happened. There might be a scapegoat, but responsibility goes upward, whether directly or indirectly.
I had the pleasure of working for a CEO that was different from most. The big thing many noticed about him - he knew you by name. Whether you were in upper management, worked in the office, on the shop floor, or in the warehouse. He always would hear you out, even if he disagreed with your view. Best of all, he made sure when you did something well, he thanked you. It didn’t seem to matter whether it was a small project, or a huge one. He wanted you to know that he noticed your contribution. You felt more like part of a family, rather than just another body at a Fortune 500 company.
Del was there during the Code Red issue, and focused on how to improve the process rather than lay blame. We were a separate group, a dotted line to him, unaffected by Code Red. He made sure we attended the big IT meeting, and started by thanking us for having patched servers while everyone else was scrambling.
Del always liked all the numbers first thing in the morning. On the rare occasion we were late, it was a simple did you find the issue? And do we have a resolution so it won’t happen again? I guess when you grow up on a farm, your concern is less on the blame game, and more on fixing and moving forward.
The world could use more like people like Del - and not just at the top.
June 8, 2021 at 2:33 am
To me, this is the area that DevOps, GitOps, anything Ops, automated, or at scale, needs to mature.
Dont forget to add Agile to the list.
For those to mature means to be removed.
the concepts are immature by definitions.
The processes have been more or less mature before they've been introduced.
2 weeks deployment cycle means there is no space for any reasonable testing.
Same Devops in charge of development, testing, deployment and support means there is no barrier on the way of a bad solution into Production.
And with CD on board deployment of a faulty code "is not the end of the world" for anyone involved - "we can fix it in a following deployment".
_____________
Code for TallyGenerator
June 8, 2021 at 4:37 pm
Looking back at my career I don't think those who played the blame game were particularly long lived, professionally speaking anyway. The higher up the organisation they go the more toxic they are. Would you want to work for someone who was going to throw you under the bus?
The DevOps culture has helped with the technical aspects of software development. Instead of having to go before a disconnected committee for approval I'm working with people who are close to all aspects of the subject matter and have a less binary view of risk.
There's a world of difference between Agile as envisaged and practiced by its forefathers and the whole industry selling itself as Agile. It emphatically isn't "bodge it into production we'll fix it later". There's a huge emphasis on improving processes that deliver software and spotting problems early. It's been a long time since I heard anyone say "well it worked fine on my machine".
In the old world where development and operations were separate the operations folks had monitors on the walls showing metrics for all manner of mechanical behaviours. There was a lot of ranting and raving about IO, network traffic etc. Eventually the Dev teams pointed out that they didn't have access to any of those metrics and every request for access had been turned down. Imagine driving a car without any instruments and being pilloried for running out of fuel, speeding fines and overheating engines.
People forget that DevOps is about building a culture of collaboration between people with different disciplines. It is not a bunch of technologies, buzzword bingo or fancy job titles.
June 9, 2021 at 4:19 am
People forget that DevOps is about building a culture of collaboration between people with different disciplines. It is not a bunch of technologies, buzzword bingo or fancy job titles.
Absolutely spot on! You made my day with that statement, David! I'm glad to see that someone other than me gets that!
--Jeff Moden
Change is inevitable... Change for the better is not.
July 2, 2021 at 12:22 am
David.Poole wrote:People forget that DevOps is about building a culture of collaboration between people with different disciplines. It is not a bunch of technologies, buzzword bingo or fancy job titles.
Absolutely spot on! You made my day with that statement, David! I'm glad to see that someone other than me gets that!
The key - those are still “different disciplines”. There should not be such thing as “DevOps engineer”. And yet - there might’ve just couple of people who get that.
_____________
Code for TallyGenerator
July 2, 2021 at 12:29 am
Another thing I was never able to understand - why index tuning is a job for ops?
they are definitely a part of table design, and changes to it should go through a proper dev cycle.
If not cutting this corner, if requests for index change would be passed to dev teams for consideration - then there would be a proper feedback channel from Prod to Dev and there would not need for that DevOps nonsense.
_____________
Code for TallyGenerator
July 2, 2021 at 9:56 pm
Another thing I was never able to understand - why index tuning is a job for ops?
they are definitely a part of table design, and changes to it should go through a proper dev cycle. If not cutting this corner, if requests for index change would be passed to dev teams for consideration - then there would be a proper feedback channel from Prod to Dev and there would not need for that DevOps nonsense.
I've found teams willing to take on far more than they are credited as being able to do. Teach them what they need and get them comfortable asking questions of you. A good question teaches a lot.
Viewing 11 posts - 1 through 10 (of 10 total)
You must be logged in to reply to this topic. Login to reply