July 29, 2024 at 8:24 am
IT often gets compared negatively with physical engineering, but we should remember that people have had centuries more practice at buildings, bridges, etc. yet we still have tragedies like Surfside in the US and Grenfell in the UK. That said, it does seem that many people are mistakenly using continuous development as a shortcut. It's all very well being able to deploy a fix within days (or hours) of releasing a faulty product, but that does rely on the recipient machine still being in a working state so that it can receive the update.
July 29, 2024 at 12:27 pm
Thank you all for the feedback and comments.
I would like to point out that I did emphasize "collectively" we stink. Individually, there were people running Crowdstrike on Windows who weren't affected. Either they didn't accept updates, they didn't have production machines attached the internet, or whatever it was. Proof that, again, collectively, we don't have to stink.
Also, several people have pointed out that sometimes the people responsible for the decisions aren't actual technologists. Way too true. Yet, still part of the collective "we" that stink.
Keep that feedback coming and thanks again.
"The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
- Theodore Roosevelt
Author of:
SQL Server Execution Plans
SQL Server Query Performance Tuning
July 29, 2024 at 2:47 pm
One of the things in the software industry that I would consider as a point of stench is that software companies release software that mostly works, just to get their product out on the market. The consumer installs the software, it has to check for updates and install them, and the consumer believes they have a working application. Then there are the patches, and patches and patches, which have to be tested, and tested and tested before you apply them to your production system.
Then there is the subscription licensing, so that you pay the company that keeps your I.T. department busy testing and installing updates, then a yearly fee to ensure you have the latest version of their software, that your I.T. department will be forcing down time to do the upgrades as they become available and are tested, involving user hours to verify functionality. And with this model, you are committing to probably 1-3 years, depending on the contract, of hoping the software company doesn't drop one of these little bombs of code on your system that shuts you down for days. But for I.T. people, this provides job security.
Imagine the auto industry, building vehicles that mostly work, and you have to keep taking them back to be patched on a monthly basis. Imagine vehicle patches as, fixing the braking system, or stopping random airbag deployments, or engines that shutdown at a specific operating temperature, etc. And when they have patched your vehicle as much as they can, then you need to upgrade to a new vehicle and start the patching process all over again. Then, if you want to guarantee you have the latest version of your vehicle, you can not only purchase the vehicle, but then also pay a yearly fee to keep it up to date. And remember, you are taking the chance that a patch may just render your vehicle inoperable at any time whether in your garage, or going 65 mph down the highway.
The point is that profit is the driving force behind the stinky software business. Software companies want to get the product into the market to start reaping the financial benefits, before the software is proven stable and secure. We see this in operating systems, productivity software, gaming software, communications devices, and most other areas of our electronic lives. The cellular phone industry expects consumers to spend hundreds, or even thousands on cellular phones that "mostly work" on a yearly basis. If you are a person that doesn't throw things out on a regular basis, take a look and see how many devices you have packed away that are no longer updateable or no longer function because the software is outdated, and yet the device is probably in good shape otherwise. But at some point, you had to spend the cash to purchase a new version or technology to replace the device and keep up with the software on a device that mostly worked, but can't be updated. Thus keeping the cash flowing to the software and device manufacturers.
July 29, 2024 at 4:07 pm
Imagine the auto industry, building vehicles that mostly work, and you have to keep taking them back to be patched on a monthly basis. Imagine vehicle patches as, fixing the braking system, or stopping random airbag deployments, or engines that shutdown at a specific operating temperature, etc. And when they have patched your vehicle as much as they can, then you need to upgrade to a new vehicle and start the patching process all over again. Then, if you want to guarantee you have the latest version of your vehicle, you can not only purchase the vehicle, but then also pay a yearly fee to keep it up to date. And remember, you are taking the chance that a patch may just render your vehicle inoperable at any time whether in your garage, or going 65 mph down the highway.
As I understand it, this is exactly the model we are moving toward. I was having trouble with my dashboard in my car and one of their solutions (which didn't work) was to update the software and reboot.
Tesla is super deep into this model. They designed their cars to accept new software downloaded from satellites. As I understand it, Tesla is constantly downloading new software into their cars. And I could be wrong, but I thought I read that the downloads don't always go that well... And don't get me started about their lack of privacy with their model!
July 30, 2024 at 1:46 pm
IT often gets compared negatively with physical engineering, but we should remember that people have had centuries more practice at buildings, bridges, etc. yet we still have tragedies like Surfside in the US and Grenfell in the UK. That said, it does seem that many people are mistakenly using continuous development as a shortcut. It's all very well being able to deploy a fix within days (or hours) of releasing a faulty product, but that does rely on the recipient machine still being in a working state so that it can receive the update.
FWIW, people also mess up physical structures. It's one reason we have inspections and reviews. Engineers make mistakes as well.
IT gets sloppy, but it also gets a lot of stuff right. How many planes work every day? How many of you are productive every day without a blue screen? Lots of IT works really, really well.
July 30, 2024 at 1:51 pm
RayC-714046 wrote:Imagine the auto industry, building vehicles that mostly work, and you have to keep taking them back to be patched on a monthly basis.
...
As I understand it, this is exactly the model we are moving toward. I was having trouble with my dashboard in my car and one of their solutions (which didn't work) was to update the software and reboot.
Tesla is super deep into this model. They designed their cars to accept new software downloaded from satellites. As I understand it, Tesla is constantly downloading new software into their cars. And I could be wrong, but I thought I read that the downloads don't always go that well... And don't get me started about their lack of privacy with their model!
Tesla sends updates periodically, but they also know their updates need to be well tested. They've gotten way better across the last decade. It's one reason I chose a Tesla over a Ford/Hyundai/etc. The software part of vehicles is hard and it takes time to work out how to test, deploy, rollback, etc. easily.
It's unlikely we get to the place where you have to pay to drive your car or update it, especially for security/safety things. Regulations won' t allow that. At least I hope not. I'd hope most democracies would be upset if this was the case.
Forcing updates is important, and it's something software manufacturers don't like in the computer business. They want upgrades because they don't want to maintain software a long time. I bet a few auto (And other) companies will regret the need to maintain some of their software over time. I wouldn't be surprised if we forced them to provide patches for a decade or more.
As far as privacy goes, most companies stink at this. Tesla isn't any worse than others. At least with them, I can see my data.
August 1, 2024 at 2:59 pm
I'm not a system administrator, so forgive me for not understanding the nuances of why some servers would be down days after the fact.
According the Microsoft:
If fixing the issue is just a matter of restarting the server in troubleshooting mode and then restoring the system drive or recovering from a system state backup, then could this process not be automated and deployed across all Windows servers in the enterprise ?
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
August 1, 2024 at 3:28 pm
I'm not a system administrator, so forgive me for not understanding the nuances of why some servers would be down days after the fact.
According the Microsoft: https://support.microsoft.com/en-us/topic/kb5042421-crowdstrike-issue-impacting-windows-endpoints-causing-an-0x50-or-0x7e-error-message-on-a-blue-screen-b1c700e0-7317-4e95-aeee-5d67dd35b92f
If fixing the issue is just a matter of restarting the server in troubleshooting mode and then restoring the system drive or recovering from a system state backup, then could this process not be automated and deployed across all Windows servers in the enterprise ?
Maybe. From what I understand, you need safe mode, and potentially multiple reboots to get there. The initial reports were that each machine needed physical access to fix, so IT people had to walk around. I saw some of that in airports, with all these small NUC devices, where people had to go reboot and connect a keyboard to change something. I saw it at my gym as well, where last Tue (5 days in), some displays had rebooted but were still blue screened.
Viewing 8 posts - 16 through 22 (of 22 total)
You must be logged in to reply to this topic. Login to reply