Relying on Algorithms

Software will be the last bastion of hand made manufacturing and it is experiencing all the problems associated with that method of construction.

Tortuous time scales

Unpredictable project outcomes.

Inconsistency of quality even from the same software houses.

Lack of scalability in production

Non standardisation

Difficult to maintain and repair

Real difficulty in finding talented staff

Costly

My hope is that we will reach a stage where significant platform change is irrelevant.

This will be the start of the era when software can really take advantage of its true digital promise - immortality. Software then will truly run the world.

I thought that the software engineering push in the 1990s would take us closer to what you say but as an industry we have since u-turned. Sadly.

Our SW company was bought out. We are now in the process of extracting our customer's data to put it into the new SW.

We have encountered so many flaws in the data being extracted, and we think much of it is based on the design flaws of the software. For example there is a relationship code between the patient and their contacts. We have facilities that have added over 25 different relations for daughters with misspellings, secondary types like "daught/POA", "daugt/POA" and "daughter/POA", etc. We think the cause is that the end user wasn't limited to the dropdown list but could add them on the fly. Where a good design would have forced them to pick from a dropdown list only when adding the contact. Then have a second location to add to the dropdown list. It might be able to be restricted who could add relationship types, but even if not, it still is a deliberate choice.

The same issue is cropping up all over. So now we have to cleanup the data as we go and try and figure out what was in the end-user's mind at the time.

+1

My Netflix homepage is a dozen preschool shows (Sesame Street, Dr Seuss movies, Leap Frog, Curious George, Disney, Thomas) followed by "critically-acclaimed violent movies" including Pulp Fiction and Reservoir Dogs. Hmmm.

Fudgeable! This is a concept expressed by one of my software design heroes, Alan Cooper. He points out that any system must enable and be resiliant to process changes made by experts. For example, if your customer support system doesn't allow a representative to make a priority change for a disgruntled customer, you have a big problem. This is the feature that helps to fix the problem of rigid algorithmic responses.

We work in the context of a very young industry and that fact is oft forgotten by the that many of us have worked in it for decades.

The materials, standards, frameworks, data, capabilities are all moving forward at a tremendous rate and our period of flux is a natural response to that environment. We are in the midst of a confluence of 'Revolutions': the ones most affecting our industry are Materials and Information (I see the 'Social' revolution as a derivative of the Information revolution).

We'll have to wait for those Revolutions to subside or for regulatory bodies to get teeth (meaning people have to die) before we have any realistic expectation of comprehensive quality control and truly robust and reliable commodity systems. There are great examples of artisans doing incredible work in our field and adhering to quality self-imposed standards. But in reality, the capabilities of the artisan are incredibly potent given the power of our computing systems and the publishing capabilities of the Internet so we have many more examples of well intentioned but immature solutions published and put in place.

You wouldn't buy your house at a craft fair, but many companies buy their line-of-business software in a similar fashion, from a well meaning small company who rely on frameworks for power but may not understand the full responsibility of production software.

We will get to the maturity of the construction industry at some point (we model many of our industry roles on theirs...) and our architects and engineers will have a stable set of materials, regulations and standards to rely on such that our end product will be capable of standing 100 stories tall and safely housing the digital assets of people who rely on it for their finances, health... and will within reason be able to withstand the naturally foreseen hazards (fire, flood, decay, malicious access...)

But right now, our architects are bombarded by new materials that provide quantum elevations in capabilities, our engineers are constantly under pressure to adapt to new frameworks and languages as the underlying platforms increase performance levels by orders of magnitude and our programmers are able to build reasonable and highly functional solutions that carry out the line-of-business requirements for the most part; so they do.

So let's not despair too much about our predicament, it will be boring soon enough and we'll be slogging along hammering nails into 2x4s according to pattern on 16" centres with well understood load thresholds and tensile strengths. Let's revel in the chaos and do our best to provide robust systems with full consideration for security, standards, auditing, logging, data integrity, performance, scalability, maintainability, user experience and whatever other system aspect you deem significant. Our industry is a teenager, raging with hormones, indestructible, capable of everything, let's embrace that for the time being -the mortgage and kids are coming soon enough.

We live in interesting times...

Cheers,

Derek.

I don't know about Netflix, but on Amazon there is the "Why was this recommended" link. If you click on that it brings up a list of stuff that was looked at or bought by whoever has been accessing the account. You can mark stuff to not matter and it changes the whole listing when you do searches. For example I bought stuff for my nephews for Christmas that I would never normally care about. So I went in and took that out and all the juvenile stuff went away.

Whilst I agree in part with dtinney, my biggest issue was that serious inroads were being made but at some point we did another u-turn.

I should have noted my agreement with this statement as well. Late 90's, Rational RUP, full SDLC, hierarchy of design responsibility, testing methodologies evolving - including UI testing, maturing application deployment platforms...

Then we are blessed with free software, open source, Internet repositories, code sharing - all empowering programming for the masses.

But the biggest factor in my estimation was the emergence of the browser as the application deployment platform, brand new infrastructure element with support for scripting and requiring a completely new set of coding tools. Programming went back to text editors and script, hacking HTML and Java to get what the platform couldn't do out of the box, emerging issues of session, web site data repositories, data ownership... followed by evolution of application service frameworks so we're dealing with loss of core functionality and then a rapid evolution of capabilities which completely negated the benefit of design. By the time you architected a solution the framework had evolved past it.

2 steps forward for deployment and user experience, one step back for structured programming.

Cheers,

Derek.

All very good points. A good round up in my opinion.

Well after visiting the science museum in London and having been in Paris at the Louvre two weeks prior I ironically noted that the Sumerians who lived 4500 years ago and who recorded things in cunieform on clay tablets appeared to have had greater consistency and rigour to their financial records than some of the institutions that I have worked for…

I have noticed that Amazon always seems to recommend the last thing I bought, like I might want to buy another wall clock, garlic press, or pressure cooker because one is just not enough.