Slow Fixes

  • There's an interesting piece at the Washington Post on Microsoft's delays in releasing patches, with some analysis showing that when the flaw is disclosed to the public, a patch comes out much quicker. After some analysis over the last 3 years and researched the dates Microsoft knew about the issue and the dates that the patches were released.

    Surprise, when everyone knows, the patches come out quicker. It seems that the piece is intended to take a shot at Microsoft's patching process, and maybe it is, but there are some interesting things in there to talk about. First of all is the time lag.

    Is this any different from any software vendor or even internal corporate software? If your boss knows, or the client knows, don't you work a little harder and a little quicker? Isn't it more critical and don't you rush things in addition to working harder when it's a "public" patch that is needed?

    I'm sure we all do. And it's human nature to put more effort into something that's widely perceived as an issue and less effort if you know that you may have more time. We all do that and our work schedules, effort, and productivity change depending on a variety of things, including the importance of the work.

    The piece also leaves open a number of questions and mentions this, noting that the analysis might be flawed. There's no mention of if the rushed (or delayed) patches had to be repatched later. There's concern over patches applying to one area, but other similar flaws found in other parts of the software remaining unpatched. That's something for sure that should be examined in looking for re-patches or whether things are rushed. There's also the lack of examination on what else was happening inside Microsoft, and whether people working on other projects had to be pulled off them. We all know that delays things as well. They take time to get their head back into code, they may be annoyed, a critical person could be out, etc.

    Not to mention that the statistical methods might not be the best ones. I'll leave that to the mathematicians to figure out.

    Patching is hard. As is finding bugs. I think Microsoft has done a much better job over the last 3-4 years and the quality of software, at least SQL Server, has improved. However there is still definitely room for improvement.

    Steve Jones

  • From my experience:

    If a user reports a bug then the QA time would be right on it.  If an in-house tester found a bug after release then we'd probably wait until the next scheduled release to fix it.

    Of course, the severity of the bug is always taken into consideration: I assume Microsoft do the same.  For example, when writing financial software, if a bug was found that caused figures to be incorrect, we'd patch that straight away.  If the GUI was awry then we'd leave it until the next release.

    I think severity should be the main consideration, and then you can take into account who knows about it.  If it's non-severe and the user doesn't know, then why bother alerting them?

  • I begin my software development career using CPM to make machine control systems with the Motorola 6809 processor.  I remember it taking 4 to 5 hours after identifying a minor problem in code, such as a timer change, to get the changes made, compiled and burned back into the EPROM's.  Today, that kind of change would take minutes or seconds depending on the nature of the control system.

    When DOS came along and I discovered how easy it was to write code and use real programs to make my work easier, I became a fan of Mr. Gates.  Through the years the Microsoft team has amazed me with how they have met the competition and eventually chosen paths that for the most part, have helped the whole IT industry.

    Please don't get me wrong, everyone needs competition to help them rethink bad decisions and I wish all MS competitors’ success as they help us all see new and better ways to use our computers for greater productivity.  But, come on, give the big guy his due...  It's a better world with MS than it ever was without it...

    Patches, who uses stinking patches anyway???

     

  • It really comes down to a certain level of integrity. Either you have it or not. Just because the client doesn't know about it, doesn't mean that you should lolly gag around and wait until the next scheduled release to fix a bug.

    In fact, the motivation for working harder and quicker should be to fix the bug before the customer finds out, not solely because they already know and you have to get it out the door. This kind of attitude is exactly why software has become the bloated, inefficient code it has.

    I long for the days where your salary is based on the quality and accuracy of your coding. There would be far fewer "engineers" out there to compete with in this market.

    Just my 2 pence.


    Cheers,

    Alex

    Rogue DBA

  • Hi Steve,

    I'm not surprised. When you think about it, it makes sense.

    The risk of releasing patches without extensive testing is high. Of course, nobody expects patches and hotfixes to get the same amount of testing service packs do. But I'm sure that the people at Microsoft try to get as much as possible of the more important tests done before releasing anything.

    If a potential security hazard has been identified in-house but is not public yet, the risk of customers being exploted by hackers is low. After all, they'll have to find the exploit first.

    However, once a security hazard is known to the public, it's also known to the hackers. You can be sure that every hacker will try to use it as much as possible before MS releases a patch. The risk for customers to be exploited gets extremely high.

    For patches to secure non-public security problems, the choice MS has is to release before finishing all tests (high risk) or postpone release (low risk). For patches to secure publicly known security problems, the choice MS has is to release before finishing all tests (high risk) or postpone release (extremely high risk). In both cases, MS will choose the option with the lowest possible risk.

    Best, Hugo


    Hugo Kornelis, SQL Server/Data Platform MVP (2006-2016)
    Visit my SQL Server blog: https://sqlserverfast.com/blog/
    SQL Server Execution Plan Reference: https://sqlserverfast.com/epr/

  • I think the delay makes plenty of sense - in an ideal world (where the PR guys don't exist), things should be prioritised according to needs, not demands. As an old boss of mine was very fond of saying "You seem to have mistaken this place for a democracy!"

     

  • Here is a key point in the article:

    Toulouse pointed to one particularly problematic patch that took the company 200 days to fix: a vulnerability in a component of Windows (and many other networking applications) known as ASN.1, at the time considered the largest vulnerability in the history of the Windows operating system. In the course of testing the patch for that flaw --  reported by security researchers at Aliso Viejo, Calif.-based eEye Digital Security -- Microsoft was forced to reset the process at least twice as internal developers found additional problems that were being masked by previously unknown glitches in the fix.

    I would prefer that Microsoft (or any other software vendor) take the time to fully test the patch before releasing it. This can be a lengthy process but if the vulnerability has not be disclosed to the public then the software vendor responsible to take the time to make sure that the patch is not going to cause any problems with the software and, more importantly, not expose any additional vulnerabilities.

    I would guess (or at least give the benefit of the doubt) that a good amount of the delay in patches for vulnerabilities that have not received "full disclosure" has to do with testing and the other patches pose a bit more risk to the users. Then again, I am fairly optimistic

    [font="Tahoma"]Bryant E. Byrd, BSSE MCDBA MCAD[/font]
    Business Intelligence Administrator
    MSBI Administration Blog

  • Don't be naive!


    Kindest Regards,

    The art of doing mathematics consists in finding that special case which contains all the germs of generality.

  • I'm sorry, but I'm afraid that I can't buy into the "let he who is without sin cast the first stone" defense.  When I slap down my several hundred dollars for a MS product I expect it to be secure and bug free.  Of course, it never is.  Am I surprised? No.  Am I disappointed?  Continually.

    Continual hot fixes, patches and service packs are a sad testament to the quality of MS products, and delaying any such updates for any reason other than to assure quality is unethical.

  • 1) As has been pointed out on numerous security lists, many of the big name vendors have problems with bugs and security. The list of Oracle bugs fixed in this latest release is some 82 in number. Some of those vulnerabilities have existed for years. As a matter of fact, David Litchfield called them out on this fact. Also, he pointed out that one of the fixes they did previously only prevented the sample exploit code he provided them. It didn't actually solve the problem. I'm not saying all this to bash Oracle but to point out as software gets more and more complex, more issues come about.

    2) The WMF vulnerability is something left over in the APIs from the 3.1 days. There was a reason for some of it in the 3.1 days, not necessarily all of it, but in those days all code was essentially trusted. Having to be backward compatible in large measure does cause these sorts of things to happen. And not being backward compatible can be a death sentence. Amiga, anyone (though Commodore's poor marketing has a lot of the blame, too).

    3) Microsoft was releasing hotfixes as soon as they were ready. And then they started getting complaints from sysadmins. Having to patch systems 5 or 6 times a month when no exploit code was known circulating was killing IT shops. It's one thing to patch a critical vulnerability that's being exploited. It's another thing to have a whole slew of patches to deploy. So Microsoft asked and the responses they got back led them to the once a month release. To which many, many sysadmins thanked them.

    K. Brian Kelley
    @kbriankelley

  • If you expect bug free software, you're in the wrong business. Every software has bugs, big or small company. And most of them have security issues. Mostly it's because the business is still evolving and the criminals' job of picking things apart for vulnerabilities is easier than the job of designing secure software.

    The being said, MS can do a much better job. There are a number of known "types" of bugs, buffer overflows, etc. that ALL code they own should be scanned for. At last back to 98 and possibly 95. Maybe they need to charge $25 or so for a backwards patch, but I think they should just release it.

    'Course, if you think you can write bug free software, then show us how easy it is. Linux/Open Source can't, Microsoft can't, Oracle can't, Sun can't. The list continues.

Viewing 11 posts - 1 through 10 (of 10 total)

You must be logged in to reply to this topic. Login to reply