At Redgate Software, we've been trialing Copilot from GitHub with our developers. I managed to get access for this experiment and have tried a few things, though I'm not sure I've found it very useful. I'll continue to work with Copilot, but for now, I just don't find Copilot AI helping me with the types of tasks I do.
However, our chief of technology, Jeff Foster, was interviewed about how he sees AI and Copilot in Redgate. It's a good read, and I found a few fascinating things in the piece. First, I was worried about this a bit, as we have explicit guidelines in developers that say we can't take the output from an AI and put it in our codebase. We worry about legalities since we sell software. How does that work with Copilot?
Well, I learned that there are a couple of things in here. First, with a business subscription, we can have the AI not show us copies of public code. If more than 150 characters of public domain code is in a suggestion, it's hidden. This is in the Copilot docs as well. I don't know if 150 is a good length or too long, but at least there are limits. We also found that Copilot doesn't retain any snippets, which is important for us. We don't want to share Redgate code with others, especially competitors unless we are explicitly putting a public repo out there.
The second thing I found really interesting is that we have a lot of guardrails around the way we produce code. We have tests, lots of automation to evaluate things, and plenty of peer review. I see requests for PRs in Slack constantly amongst our teams. We know that the code which gets generated likely needs some editing, but also that we want to be sure that whether it's sent as is or edited by a developer, the code is still reviewed. Having these guardrails, test, etc. help us experiment with new ideas without impacting our productivity.
Perhaps the most interesting part for me was near the end. Jeff doesn't think that Copilot makes us faster. He says "We probably invest ten times more time finding the right problem to solve than on the actual coding." That makes sense, given what I've observed over the last 15 years. We are deliberate and careful about picking problems to solve and producing high-quality code. Sometimes it makes me a little crazy as I want things to move faster, but I've learned to appreciate that overall, this is an approach that doesn't get us into places where we struggle to onboard developers to new projects or adjust our code to meet new requirements.
I've got a few quiet weeks coming up, so I'm going to give in more and use Copilot a bunch to see what I can get out of it. If you've tried it, let me know what works or doesn't. If you want me to try and solve some problem for you, drop a note in the discussion with the challenge you have.
