Representative Data Challenges

  • The short answer, there isn't a simple way to view the world of software. It isn't that we allow all failure or have all the data to cover all cases, but we need to find a balance. Aiming for perfection in software or people isn't often realistic. At the same time, allowing too much, especially in data sets, can be problematic.

    While many people, especially managers, want a simple rule, the world isn't quite so simple. There are reasons to tolerate or not tolerate some failure. There are also reasons to go with less data or seek more.

    My intent here was to point out some failures in data sets, and one of those is in diversity. Some of you note you don't see color or differences in people. I'm not sure I think that is the answer to anything. Instead I think we should appreciate diversity. There problems in the articles I've listed are real problems to a portion of the world. While you might not think they apply to you, or aren't necessary to examine, I'd hope that anyone doing image recognition or related work  would consider the diversity of the world.

  • Well, for the past couple hours I've been otherwise occupied with my other project, besides this one but I've still been thinking about this discussion of design and implementation.  I'm working in my basement installing vapor barrier, framing, insulation and wallboard prior to building permanently attached storage.

    I'm sure the architectural drawing (the design phase) of  my home assumes a smooth level cement floor, 90 degree angles in the corners, and straight 90 degree perpendicular cement walls.  In reality (the implementation phase), NONE of these are true.   In addition, there are little nubs of cement sticking out from the walls where the forms had developed holes and there is a small ridge along the floor surface where the forms did not exactly meet the floor.  Further, the distance from the floor surface to the beams is not consistent since the cement is not perfectly smooth.  Thus the storage unit cannot be scaled perfectly but must be custom measured and cut to fit.  My task is to now implement a new phase of maintenance of these 'features'  by adapting my design of the framing, wall installation, and storage construction.

    Our design and creation of database systems is very similar in that while we can't always assume a perfect design by the architect and a perfectly straight result by the builder, we hope that the architect did an acceptable workable design, and that the builder adapted where necessary without jeopardizing the integrity of the design and the adaptability to future needs by making good choices.

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

  • Steve Jones - SSC Editor wrote:

    My intent here was to point out some failures in data sets, and one of those is in diversity. Some of you note you don't see color or differences in people. I'm not sure I think that is the answer to anything. Instead I think we should appreciate diversity. There problems in the articles I've listed are real problems to a portion of the world. While you might not think they apply to you, or aren't necessary to examine, I'd hope that anyone doing image recognition or related work  would consider the diversity of the world.

    Well, Steve, I only partially agree with this.  I personally believe that we have gone way overboard on appreciating diversity, to the point it is now divisive.  If someone doesn't think I appreciate their diversity, my response is 'Get over it.  We have a job to do'.  If the image recognition doesn't work, let's fix it.  No need to make it an issue.   Software may not adequately recognize my image because some time ago I passed out, fell and broke my nose which is now off-center.  Is that a failure of diversity?  "I'm insulted that your program fails to match me to my photo.  Why didn't you consider that in your design?"  If you tell me  my nose looks crooked, I may tell you it's because you're drunk.

    Hers's a good example of the huge implications of making too great an issue of diversity.  There is movement forming to make reparations to people of color due to the history of slavery.  OK, while I agree that slavery was the worst part of our greet national history, the practical problem with 'appreciating diversity' is this:  Who gets to decide who GETS the reparations and who PAYS the reparations?  Who draws the line?  How 'black' or 'white' do I have to be to qualify either way?  And whose record of heredity do we trust and whose do we reject?

    Again, not to push any buttons here, but to illustrate the absurdity of some things that don't really need to be an issue.  Inclusivity is absolutely the most divisive thing I've observed in my 77-plus lifetime.

    Let's put THAT in  our software and figure it out.

     

    • This reply was modified 3 years, 11 months ago by  skeleton567. Reason: added thoughts

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

  • Well, one final observation on this topic before I go begin 'merlot time' with my wife.  While we've gotten way sideways on this whole 'appreciating diversity' thing, and I have never actually done any graphic recognition software myself, my purely uneducated guess is that those 'faulty' bathroom devices' that fail are likely due not to 'color' differentiation, but instead are due to bad motion detection.

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

  • Bad motion detection like infrared light reflection that doesn't reflect enough light off of darker skin tones to actuate the device?

  • jschmidt 17654 wrote:

    Bad motion detection like infrared light reflection that doesn't reflect enough light off of darker skin tones to actuate the device?

    Bingo!  Fix the problem instead of p!ssing and moaning about a lack of diversity.

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

  • I believe you have labeled "identifying the lack of diversity in test plans and data" as "p!ssing and moaning".

  • jschmidt 17654 wrote:

    I believe you have labeled "identifying the lack of diversity in test plans and data" as "p!ssing and moaning".

    Wrong, what I labeled as that was attempting to make a restroom hardware failure as a diversity failure instead of just fixing the problem.  Please do not attempt to misquote my comments.

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

Viewing 8 posts - 16 through 22 (of 22 total)

You must be logged in to reply to this topic. Login to reply