December 7, 2016 at 10:49 pm
Comments posted to this topic are about the item Testing Coders
December 7, 2016 at 11:41 pm
Years ago (longer than I care to think about 🙂 ) I worked for a friend for a little bit. Jim sold electronic components like resistors and so forth. It was like US$4.00 for 100 or some (to me) ridiculously low amount. I was resolving some issues with his system that was built on the Pick operating system. For those unfamiliar with it the Pick O/S was, essentially, a three dimensional representation of data. Sort of like arrays that could hold arrays of arrays. :smooooth:
Chatting with Jim one day about a particular resistor a client was insisting be a specific brand (Fairchild over Motorola) Jim said that while both were in fact manufactured to exactly the same specification it was a known issue that often one would work and the other would not. With no real reason on the face of it.
I think sometimes that there are often unknowns that tests, like specifications, just do not work in the real world. No one's fault, just one of those realities that just refuse to fall into what the textbooks say should be. Which, for me, is actually the real issue; that of using the incorrect measuring tool. Maybe "do not work" is too strong whereas "incompletely satisfy" is better.
I also think that textbooks (using the above analogy) are often thought about as defining reality rather than an attempt to codify and represent it. Think bureaucrats vs entrepreneurs - which most accurately defines reality? In other words we're most likely expecting the wrong thing from a test. I'm looking for "good judgement" and how can a test fish that out? I think so to some degree but given that "good judgement" implies responding to a dynamic set of conditions I think that should not be forgotten. I read in Steve's post just that, "How do you think and respond?" That's different than, "What do you remember?"
I can think of fifty things that fall outside of any sort of test that could have an impact of the convergence of designs, plans, specifications, talents, commitments and decisions with the reality of the often mal-defined phrase "Done".
Maybe we're looking for a digital answer in an analog world?
December 8, 2016 at 2:19 am
I just read about Grant's sins and, funnily enough, was discussing abuse of NOLOCK with a fellow SSC'er just this morning.
As for interviewing, for a C# developer role I was asked to code the classic FIZZBUZZ (which I hadn't knowingly come across before). I was coding it and offered the interviewer to continue asking questions as I was coding (for me this was an opportunity to stand out by coding whilst a little distracted). He declined. When I had been working there for a few weeks I asked the interviewer why he hadn't taken the opportunity to push me further and his answer was astounding. He had looked over my shoulder and could see that I could code and was not wanting to give himself a reason not to take me on as I was doing well and most of the other candidates were so bad. One opened up Visual Studio and just stared at it because he didn't have a clue what to do. Another didn't even manage to launch Visual Studio. For a contract role. For an experienced .NET developer. Recruitment company failure!!!
With this level of candidates, and it doesn't appear to be any better for permanent recruitment, it is no wonder that we have little opportunity to hone our skills when there is so much effort filtering out candidates who should never have made an interview stage.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
December 8, 2016 at 6:56 am
it is no wonder that we have little opportunity to hone our skills when there is so much effort filtering out candidates who should never have made an interview stage.
I've had similar experiences. Even when I think I have the right guy, just as in the article some make it, some don't. I even had one quit on me because he just didn't want to work anymore.
December 8, 2016 at 7:12 am
This potentially could be a good test, if the example is well chosen.
We read of all kinds of tests, some of which border on the ridiculous: throwing bizarre (often not even IT related) examples at someone to see if they can jump out of the box fast enough. These kooky tests, however, seem to be more about the ego of the interviewer than any real world capabilities of the candidate.
Another extreme is a test about knowledge or deeply arcane functionality of SQL or whatever to see if they have the photograhic memory to recall syntax. Realistically, however, a good admin/developer needs to know these rarely used capabilities are there, most sensible people check the documentation when they need to use them.
The ideal test, I think, involves presenting a problem (like above) and asking: how would you go about improving this? Not the detailed syntax, but instead listening to the candidate's thought process.
For a humorous take:
https://jobmob.co.il/blog/funny-ikea-job-interview-cartoon/
...
-- FORTRAN manual for Xerox Computers --
December 8, 2016 at 7:49 am
How about this?
Develop a project plan that includes either writing new code, modifying code or both. Come up with a few specific tasks that fit. Then write up something that explains that we are working on a project, and we have some unfinished work. Give the person the choice of which task or tasks they want to tackle. Give them the choice of writing code, writing pseudo code, or simply explaining what they would do. As you compare answers, you will get a feel for who was more productive as far as volume, who produced better quality, who was able to complete work versus who just got started but couldn't finish, some idea of their thought processes on how to tackle project tasks, and I am sure many other insights.
It would be inappropriate to use an actual live project, you aren't paying them people to do the work, but I believe many of us could come up with something from their past, or something they have considered building in the future. For SQL, it should be easy to come up with tasks from one of the sample databases Microsoft provides. You could build some SPs and provide modification tasks.
When I am involved in hiring my time is limited, but I still utilize this to some degree. I have found that I know exactly what I am getting. A recent experience allowed me to determine who had confidence, who didn't, who was honest and provided accurate information regarding their skills, and who was blowing smoke.
I feel this is much better than what I have experienced, give someone a pencil and a sheet of paper and ask them to code...
Dave
December 8, 2016 at 8:36 am
Another example of wacky interview questions:
I really feel this is just smoke and mirrors, more about the interviewer's interpretation of themselves and/or their company as hip and innovative. But in reality it tells NOTHING about performance on the job.
To get a reality check on this, would you choose a doctor based on how well they answer these questions?
...
-- FORTRAN manual for Xerox Computers --
December 8, 2016 at 8:57 am
I have an advantage as an interviewer as I have full self-awareness that I am not hip.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
December 8, 2016 at 9:03 am
At a job interview a number of years ago, I was given a listing of a C program and asked to explain what it does. I did that and also explained that technically, the program should not work because it was writing to a file that was opened in read-only mode.
December 8, 2016 at 9:23 am
WOW, very interesting article, Steve! I've had the opportunity of interviewing several candidates over the years. We've used a variety of techniques. Sometimes, depending upon where we worked, your hands are kind of tied by HR. For example, in my previous job all questions had to be submitted to HR for their approval. When it came time to interview candidates we had to read through the questions like it was a script. And stay on script!!
I really like the idea that you said Ore Eini did. We did something like that even in my previous job. One of those scripted questions was to show the candidate 2 tables, give the table structures and a few records for each table, and also have a simple left join SELECT. We asked them what the results of the SELECT would be. The ones who knew how to do a simple SQL SELECT answered the question in about 5 minutes. The others who struggled, we just moved on. And didn't hire.
Kindest Regards, Rod Connect with me on LinkedIn.
December 8, 2016 at 10:54 am
A few years ago, I was fortunate to work with a great company that gave our team the freedom to design our own recruiting strategy. We were in an isolated, yet vibrant part of the world so we wanted to make sure the candidates we hired would work out (since it was expensive to bring them over).
Our process evolved over time and in the end, it was pretty good but certainly not 100% error proof. You just don't know how somebody is going to work out until you've got them in the office for awhile, contributing to projects.
Our process was this ...
- find out about candidates through recruiters, job sites and recommendations from other employees
- we spent a lot of effort briefing recruiters on what we were looking for, to avoid wasting time on those who were obviously not suitable
- administer an on-line personality test and an on-line general, timed aptitude test; both of these were somewhat controversial even within the firm but we used them anyway ... I thought they were reasonable
- hold a couple of Skype interviews ... one would be more of a "meet and greet" style to explain the company, the work and to get a sense of the candidate; the second would be more technical in nature
- if we felt the candidate was reasonable, we would let our non-technical supervisor Skype with the candidate
- if the candidate was still in the running at this point, we would bring them to our location for at least two or three days
- during this time, we would have more interviews and bring the candidate out to dinner (again, to see how they behave socially)
- also, we would give them a small, mini-project to do, using a technology of their choice; they would "present" it to a few of us at the end of the stay
It was admittedly quite involved and it didn't always work out. However, we had some great hires as a result. Like anything, you get out of it what you put into it.
December 8, 2016 at 2:19 pm
djackson 22568 (12/8/2016)
How about this?Develop a project plan that includes either writing new code, modifying code or both. Come up with a few specific tasks that fit. Then write up something that explains that we are working on a project, and we have some unfinished work. Give the person the choice of which task or tasks they want to tackle. Give them the choice of writing code, writing pseudo code, or simply explaining what they would do. As you compare answers, you will get a feel for who was more productive as far as volume, who produced better quality, who was able to complete work versus who just got started but couldn't finish, some idea of their thought processes on how to tackle project tasks, and I am sure many other insights.
I like that idea.
December 8, 2016 at 2:42 pm
At my current job, our organization actually exists for the Testing department who writes tests that are supposed to test someone's aptitude for the job instead of someone's measure of "likability", and the test I took really felt like it was designed to figure out if I would have that aptitude.
The test was a pared-down version of the website that we actually work on, with some errors built in that kept it from building correctly. I was allowed to use the internet to solve problems, just like in real life, and I had a task list for what someone could theoretically get done in the 2 hours I was given. Knowing next to nothing about Visual Studio or .NET or C#, I had to learn quickly. It turns out that my manager didn't care if I could walk onto the job and replace him; he wanted someone who was able to learn, and the fact that I documented my code to make it easier for him to find the changes I had made, made my test really stick out.
Gary Varga (12/8/2016)
With this level of candidates, and it doesn't appear to be any better for permanent recruitment, it is no wonder that we have little opportunity to hone our skills when there is so much effort filtering out candidates who should never have made an interview stage.
I'm not sure that there's a way to filter out people who are going to fail the test without filtering out people who are going to pass in ways that are really useful to your organization. I really liked my organization's policy of "test first, ask questions later", because it got me around the personality tests that I always seem to fail, and the interviews where I'm supposed to talk about how amazing I am, instead of getting to show that I can be a useful addition to your team because I'm willing to learn even when I'm in over my head. And really, what could be a better test for someone than working on code like the code that your organization works on? Yes, you'll end up with some really terrible candidates, because the average guy applies if he only meets 60% of the qualifications you post, and every 20-something guy who's been tinkering with computers his whole life thinks he can code. But you'll also end up with some good surprises.
December 8, 2016 at 3:11 pm
Regardless of whether we're interviewing for a DBA or a developer who will code SQL, one question I ask is:
"For the next 15 minutes, explain everything you know about locking, blocking, and isolation level."
Yes, that's a broad topic, but I can tell a LOT about someone's past experience and areas of expertise based on that one question.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
December 9, 2016 at 12:33 am
hammackk (12/8/2016)
...I'm not sure that there's a way to filter out people who are going to fail the test without filtering out people who are going to pass in ways that are really useful to your organization...
I am mostly referring to people offered up for intermediate and higher roles who clearly do not have a clue. Absolutely none. Lower skill level than a junior i.e. none. This a recruitment agency must be able to do.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
Viewing 15 posts - 1 through 15 (of 20 total)
You must be logged in to reply to this topic. Login to reply