Today we have a guest editorial from Simon Galbraith

One of the abiding memories of my childhood is our family dinners. My father ran a small, struggling building firm and over some plain food I got the equivalent of an MBA. My father would talk about what he was up to with the firm and I would ask endless questions. One of the answers I remember fondly was “If something sounds too good to be true, it almost certainly is.”

Four people, one week

At Red Gate it normally takes a minimum of three months to produce a piece of software; larger, more complex projects can take 12-18 months.  In an effort to reduce development time, we recently ran an experiment to see if we could produce something useful in one week.

We assembled a team of four people, rented them a house by the sea, and sent them off for the week to come up with something.

They came back with SQL Search, which makes it much faster and more efficient to search your database for schema objects and text strings in SQL Server Management Studio.

So simply by sending a great team out of the office we were able to improve their productivity by a factor of 12? It’s the sort of statement that would cause my father to draw in a deep breath and say “If something sounds too good to be true…”

The truth reveals itself

It is too good to be true. That team was great, and we might be onto something having a prototype done in this way.  But, you can’t change productivity by a factor of 12 just by sending people offsite and having them work 18-hour days and eat takeaway pizzas.

The team came back with something that looked and behaved a lot like a product, but it wasn’t one. It was buggy, had some design problems, and was very slow in many circumstances due to a rushed architectural decision.

It was, however, an excellent prototype – it showed people very clearly what we meant when we said SQL Search, but we couldn’t have given it away because it would have hugely annoyed users.

Another team took the prototype and spent three months working through the hundreds of bugs, and recoding some key elements – that’s the product you can now download on our website.

The second team worked in the way that Red Gate engineers usually work – in a scrum-like team with short daily meetings, a project manager, prioritized backlogs and all of that good stuff. They had managers, they ate normally, and went home at sensible times. They weren’t as fast as the first team, but they worked in an environment that allowed them to think clearly and do great work. I’m still not sure which team was more productive. The second team complained of code written in a hurry, but the first team did manage to produce a richly featured prototype very rapidly.

The Windows NT story

Our SQL Search experience reminds me of the difference between the official and unofficial stories of Windows NT.

The official story: Unix-hating genius Dave Cutler designed and built a UI-based operating system that won the historic server OS battle for Microsoft, leaving the once-mighty Novell doomed

The unofficial story: Unix-hating genius Dave Cutler led the team that designed and built a UI-based operating system that he wouldn’t have been able to ship in a decade. Microsoft quietly realized that Dave Cutler was unsuited to large-scale project management and put a brilliant, but anonymous, project manager on the job. It was this team effort with Dave Cutler taking a step back towards the end that shipped a product that changed the OS world.

The version I suspect is truer still has Dave Cutler as a visionary genius and doing lots of important implementation, but the details of how it really happened confront engineering realities that we’re often unwilling to confront: Writing production code is hard and slow, fixing bugs is necessary even when a genius has thought deeply about the approach, and a different approach can be needed for different parts of a project.

If you use Management Studio I really hope that SQL Search comes in useful for you. And next time someone tells you about a production quality product produced in an unfeasible length of time, listen for my father drawing in his breath.

SQL Search is now available here as a free add-in to SSMS.

To find out more about the Coding by the Sea project that led to the SQL Search prototype, read Robert Chipperfield’s story.  To find out what happened in the three months of product development that followed, read the interview with Red Gate project manager Tanya Joseph.