April 30, 2012 at 12:08 am
Comments posted to this topic are about the item Being Responsible for Code
April 30, 2012 at 3:21 am
Hi, Would have thought the ‘tester’ should be more accountable for allowing bugs through rather than the programmer – metrics should be based on the bugs identified at testing stage not after release
April 30, 2012 at 4:56 am
A. Hughes (4/30/2012)
Hi, Would have thought the ‘tester’ should be more accountable for allowing bugs through rather than the programmer – metrics should be based on the bugs identified at testing stage not after release
I quite agree - if a bug has got through to the live environment there has been a failure in the QA process, rather than just one individual.
April 30, 2012 at 5:13 am
The best way to avoid any bugs is to not write any code...
Be careful what you incentivise.
April 30, 2012 at 6:47 am
I understand the point which is "to be accountable for your mistakes" and I praise this way of life but for several reasons I would not work for such company.
Over time I've seen several developer codes in SQL (and .Net in general) and with a rule like this, I would be force to be accountable for their code and would require an entire regression testing for a single fix every time. (Even changing a line text, some software I've seen was relying on text to execute different operations)
Only a few developers uses the "Coding defensively" method and most of the time they are pushed forward to implement so-so solution (that will do the work) but are not maintainable or worst have side-effects which are not easily spotted.
To my eyes, (it's a rough idea) I would prefer having a few top knowledgeable person who are dedicated to validate what ever code is generated (which require great knowledge of the overall application & the technology, yes this is rough), have co-worker check what's being done is working. (Unit testing could also be implemented)
Having a QA that test particular functionality, have QA that test regressions, have a series of automated tests that run with each build.
Finally business owner that need to test their toy.
But that chain is very, very costly, but you get what you paid for in the end...
April 30, 2012 at 6:55 am
I would hope that similar metrics are tracked of code/peer reviewers and QA/UAT as well.
Also, I would hope that success is tracked and rewarded as well as failure. X-number of deployments without disruption should get some sort of incentive as well as X-number of deployment failures would send a developer through the trap door.
April 30, 2012 at 7:33 am
Some comments on the specific example in the article and on accountability in software development in general.
First: The specific example. I'd say that Facebook is doing it completely wrong. If the website is disrupted and a post-deployment fix is required, the developer is NEVER to blame. The first to blame would either be the QA department for missing something during test, or management for not allowing sufficient time for testing.
Second: Should we then say that Facebook is doing the right stuff at the wrong moment? In other words, should they, instead of assessing developers on bugs that brought the website down, assess developers on bugs found during QA? That would be less of a mistake - but I think it's still a mistake, for two reasons: (a) a bug in developed code is sometimes not the developers' but the designers' fault, and (b) as someone already mentioned, it can stimulate developers to program defensively, going over their code twenty times, running countless tests, etc - that would surely bring down the number of bugs, but it would also minimize the developers output.
If you really want to measure developers, you first have to define what a good develoepr actually is. How will you measure my performance? By number of bugs? - I can code almost bug-free if you want to - if you don't mind that I take a few weeks for even the simplest program. By number of programs built in a week? I can give you tens, maybe hundreds, but don't expect the QA department to be happy. By lines of code? Come on, that's the worst metric of them all; I could build a macro to insert a line break after every keyword to give you lots more lines of code, but do you really want that?
I think the best developer is the one who manages to strike a perfect balance between the two extremes of spending way too much time to minimize the chance of bugs or going so fast that the code will be littered with them. If I spend two days on an average program, then hand it to QA and start on the next, then return to the program once QA is done and fix the three bugs they found, I am probably more effective then when I spend seven days on the same program to ensure that QA finds nothing, or when I finish it in three hours and then have to go through seven cycles of bug fixing and returning it to QA before it's good.
So how do you measure that? Counting the time from starting the first version until getting final signoff of QA might be a good idea, but that does not take into account that not all programs are equally complex. Also, some developers brong value to a team that would not be measured by this. How about a senior developer who spends two hours per day helping other developers solve complex issues? How about a developer who, instead of just coding what the design says, first takes the time to read it critically and ask the designer some questions? If those questions are all useless, it's a waste of time for both the designer and the developer, but I've also experienced (both as developer and as designer) that those critical questions saved a lot of time because they pointed out a flaw in the design. In that way, a developer can be very effective without ever coding a single line of code!
Bottom line: measuring performance of a developer is hard, maybe even impossible. Measuring the time between getting the assignment and getting signoff from QA would be a fair start (if complexity is taken into account, for instance using function point analysis), but that does not measure other qualities the developer may have. If used as one element in assessing the developer, applied by a human who is also able to understand other qualities of the developer and circumstances that may have led to unusual low or high values, it can be a good element. But only then. Automated translation of this measure in salary adjustment would, for me, be a good reason to find a different company. Not because I would be paid bad (I am quite confident that I could work the system), but because I believe that being a good developer also involves being critical of the design before you start coding, also involves helping your colleagues if they are stuck and you happen to know your way around their problem, and also other stuff that may be even harder to measure.
Apologies for the rant. I've gone much further off-topic than I intended.
April 30, 2012 at 7:35 am
Rob,
You said: "Be careful what you incentivise."
You beat me to the punch on that comment.
If you have 2 developers and one writes 99% of the new and innovative code.
The other meanwhile basically cuts and pastes a way to a paycheck. Never writing anything new himself.
Who will release a bug first?
Who is more valuable to the organization?
April 30, 2012 at 7:53 am
I want to first comment on the posts I have seen, then your article.
First, I understand what everyone is saying (so far) about this leading to lower output, and about testing. I have no argument with these comments at all.
As a confirmation of how this works, I once worked with someone who worked for "the phone company" writing code. Allegedly he would write about 10 lines of code per year or month, I don't recall which. Whatever it was, it was extremely low. He explained that the review process was so extensive, it would take 6 months for any changes to get approved! Sounds wrong, until you think about the impact of bringing down the phone system in the country. Even now with a few companies running it instead of just one like it was decades ago, I would think a change in one could cause issues in another company pretty easily. Right or wrong, when there are incentives to do things perfectly, and punishment for any errors, the quantity of the output will go down significantly.
Second, I like the idea of holding people accountable for their quality. Won't happen. Companies don't want to do so. There are a million reasons, a few of them are:
If you hold the worker responsible, do you hold the CEO responsible?
Other managers?
Since managers are tasked with far more work than they should be, they don't have time to look at who is at fault, and would simply blame the easiest scape goat.
Since employees would know about it, they would do everything they could to "hide" in the system. Wait, that happens now!
Unfair managers would find a way to blame those employees they don't like.
One of the biggest issues in the workplace today is that the best employees are not rewarded, the worst employees get the same raises, bonuses as everyone else. The reason is companies are afraid of lawsuits and don't want to manage their workforce. They may claim they do, evidence shows not.
So I am in favor of doing so, as soon as someone comes up with a fair way to evaluate quality, that also includes all other factors about what is required of an employee, and holds managers accountable for NOT doing their job, with penalties at least as harsh as those for non managers.
Dave
April 30, 2012 at 8:00 am
I guess FB doesn't have any sort of QA or test methods in place. We used to have a programmer here that thought his stuff was fine because "it compiled" (his words).
Cheers
April 30, 2012 at 8:28 am
Hugo Kornelis (4/30/2012)
Some comments on the specific example in the article and on accountability in software development in general.So how do you measure that? Counting the time from starting the first version until getting final signoff of QA might be a good idea, but that does not take into account that not all programs are equally complex. Also, some developers brong value to a team that would not be measured by this. How about a senior developer who spends two hours per day helping other developers solve complex issues? How about a developer who, instead of just coding what the design says, first takes the time to read it critically and ask the designer some questions? If those questions are all useless, it's a waste of time for both the designer and the developer, but I've also experienced (both as developer and as designer) that those critical questions saved a lot of time because they pointed out a flaw in the design. In that way, a developer can be very effective without ever coding a single line of code!
I worked for an ASP company (before the "cloud" got caught up in marketing) and was tasked with fixing a bug in the software that loaded an array and then passed it to a third party product for statistical analysis. I quickly found that it was not doing anything like what they thought! It is somewhat complex, but worth explaining.
If a company sells 10 widgets a month, every month, you should make sure you always have enough to cover that. It doesn't work that way though, because you have lead times for delivery, seasonal fluctuations, and other issues that affect demand. So you need to know how many you sold each month, along with a lot of other data, to decide how much to buy. In our case you would load that data into an array, where the array might be defined as 24 months, and let's also assume we were using C++ so the array indices would go from 0-23.
The developer never worked out good test cases with the designer. The code was written and tested and seemed fine. The problem was that the test cases were not written to identify errors in the code. So when I reviewed the issue and found that in VB 6 (we weren't using C++ but that is all the developer had coded in previously) you have the option of using 0-based or 1-based arrays. So the developer assumed we were using one, when in fact we were using the other. Not sure which way anymore, but essentially he was doing something like loading elements 1-24 and passing it to the third party tool while telling it he was using 0-23. Whatever it was, one element always got evaluated as 0 demand!
To bring this back on point, the only way I could convince everyone I was right was to show them documentation on how C++ worked, how VB worked, how the third party used the array, how the designer actually wanted it to work, testing with the designer with proper test data, and bringing in a very intelligent math expert to prove my calculations were correct!
Finding and fixing the bug took me a couple hours.
Proving I was right took two full weeks!
Don't get me wrong, the people I worked with were very intelligent, and did the right thing by taking the time to check. The point is, had I been evaluated on quantity of code, I would never have got a raise. I ended up playing a role like a bounty hunter - finding root causes, doing extensive research into WHY it worked the way it did, and then fixing it. That is not something companies are generally willing to pay for.
Dave
April 30, 2012 at 8:29 am
Hugo Kornelis (4/30/2012)
Apologies for the rant. I've gone much further off-topic than I intended.
No apology needed. Good rant, and good points.
April 30, 2012 at 8:33 am
djackson 22568 (4/30/2012)
One of the biggest issues in the workplace today is that the best employees are not rewarded, the worst employees get the same raises, bonuses as everyone else. The reason is companies are afraid of lawsuits and don't want to manage their workforce. They may claim they do, evidence shows not.So I am in favor of doing so, as soon as someone comes up with a fair way to evaluate quality, that also includes all other factors about what is required of an employee, and holds managers accountable for NOT doing their job, with penalties at least as harsh as those for non managers.
A bit of a generalization, but not far off the way things are. Too many companies are afraid of lawsuits, but many are not. Some of the tech companies are brutal with reviews, almost like medical school where they look to prune off the low performers, even if they are good workers.
I think your earlier points about people hiding in the systems and managers abusing power is human nature. It happens all too often, and it's no wonder that we end up with workforces that are achieving far less than they could.
April 30, 2012 at 8:38 am
Steve Jones - SSC Editor (4/30/2012)
djackson 22568 (4/30/2012)
One of the biggest issues in the workplace today is that the best employees are not rewarded, the worst employees get the same raises, bonuses as everyone else. The reason is companies are afraid of lawsuits and don't want to manage their workforce. They may claim they do, evidence shows not.So I am in favor of doing so, as soon as someone comes up with a fair way to evaluate quality, that also includes all other factors about what is required of an employee, and holds managers accountable for NOT doing their job, with penalties at least as harsh as those for non managers.
A bit of a generalization, but not far off the way things are. Too many companies are afraid of lawsuits, but many are not. Some of the tech companies are brutal with reviews, almost like medical school where they look to prune off the low performers, even if they are good workers.
I think your earlier points about people hiding in the systems and managers abusing power is human nature. It happens all too often, and it's no wonder that we end up with workforces that are achieving far less than they could.
I have to agree with most of what you said. I take issue with
A bit of a generalization
.
That was a rather LARGE generalization! 😀 We tend to write like we speak, and sometimes the lack of facial expressions and other clues affect the accuracy of the written word. I should have been more clear in qualifying what I said. I think you moved my comments much closer to reality.
Dave
Dave
April 30, 2012 at 8:59 am
Hugo, I think what you qualified as a rant is rather a pretty descriptive picture of what happens in real life. I certainly don't see it as a rant.
As it happens, my brother use to work for 'the phone company'. His only real measure was testing, and for the 6 years he managed the software systems that handled rebilling (i.e. pay up, deadbeat) he never had an installation backed out. The first installation after he retired was backed out and his successor even said that if P### had been here, this would have never happened. There is no incentive payment to anyone in any of those successful situations he was involved in. Just a gang of programmers who wrote increasingly better code because they knew the level of testing that was going to happen.
I would also liken it to this...the more times you get called at 2:30 am to fix something, the better you write/test/configure software. EVERYONE should be on call for a while. Then the incentive is sleep...pure, blissful resting sleep. How much of a benefit does THAT pay?
------------
Buy the ticket, take the ride. -- Hunter S. Thompson
Viewing 15 posts - 1 through 15 (of 34 total)
You must be logged in to reply to this topic. Login to reply