AI: is it starting to speak our language?

When someone once commented on the obvious African influences in his paintings, Pablo Picasso famously said, “Good artists copy, great artists steal.” Programmers practise a similar kind of predation – like magpies, a good programmer will feather their nests with the shiniest bits that they can spy. And with a good search engine, well, there’s a lot there to spy. Nearly every problem in programming has been solved before by someone, somewhere, at some point – and if a programmer can find that solution, they can save themselves a fair bit of effort, or at least study the handiwork of others in pursuit of their own answers.

It’s this promiscuity of resources that separates the young and inexperienced programmers from those (such as myself) who have been slinging code most of their lives. A few years ago, a study showed that the most productive programmers were those who consulted Google (and Stack Overflow, which is a kind of Wikipedia of great code examples) most frequently. These days, I code with a whole set of browser tabs open – into Google, Stack Overflow, documentation pages, and so forth. It’s like having an extra brain – well, millions of extra brains. It definitely makes programming less tiring than it might have been 20 years ago, when programmers spent a lot of their time solving problems others had already solved, but had no idea others had solved them.

“Good artists copy, great artists steal.”

Pablo Picasso

These practices that programmers learn over a lifetime’s experience with their tools have been dramatically short-circuited by one of the latest advancements in artificial intelligence. At the end of June, programming website GitHub – a “repository” used by millions of programmers to store and share their code with their teams, or, in the case of open source, with the world – announced a new tool: Copilot. Developed from a very rich artificial intelligence model known as OpenAI Codex, it purports to do what good programmers have always done for themselves – find the best bit of code to solve the problem at hand.

That sounds like quite a big ask, because code can be difficult for any programmer who didn’t write it to read and understand, and because the entire act of coding – laying out a structure, then filling in that structure with the necessary details – seems like a very human, cognitive task. It’s never been something a programmer could ask Siri or Alexa or Google Assistant to do for them. In AI terms, it’s next level.

I wrote a single line of comment that stated my requirements and – abracadabra! – the computer provided a complete solution.

When I recently got access to GitHub Copilot (it’s still in limited release) I had very low expectations. I reckoned it would do little more than make a few hints while I sat and typed out my code. What I got, though, was rather more than that. From the examples provided by GitHub, I could see that I could frame what I needed as a “comment” – a bit of code designed to be read by a human and ignored by the computer. So I wrote a single line of comment that stated my requirements and – abracadabra! – the computer provided a complete solution.

I inspected the code burped up by Copilot, and realised the computer had given me exactly the result I was looking for. Just what I would have done – if not in exactly the same way. Coding styles differ among programmers, and Copilot has its own, very straightforward style. That makes it easy to read – always a good thing.

Yet after some time playing with it, I could see that Copilot is far from perfect. I asked it for code to do something quite straightforward, and while it came up with suggestions that would have done the job, none of them made sense in the context of the programming language I was using. It may be that I’ll need to learn to “speak” to Copilot in a language that it can understand – just as we’ve all learned how to shape our requests to Siri and Google and Alexa. And there’s no question that it feels uncanny to have the computer quickly and silently come up with just the bit of code that you’re looking for. Copilot will make bad programmers better, and good programmers more productive. That’s a good thing. But it’s just the tip of the iceberg.

It may be that I’ll need to learn to “speak” to Copilot in a language that it can understand.

Copilot sits atop OpenAI Codex, which in turn sits upon something known as GPT-3 – the third iteration of a “Generative Pre-trained Transformer”, an artificial intelligence program that has been “taught” 175 billion “parameters” (think of them as rules) about human language. All of that data came from a huge vacuuming of the Web – GPT-3 sucked in much of what we’ve been publishing online over the past 30 years. With so much fed into it, and so many rules inferred from all of that hoovering, GPT-3 could do things that no computer program had done before, such as craft a summary of a technical article, find the salient points in a press release – even “read between the lines” of CEOs public reports to discern the real health of the business. There’s no magic here, no “thought”, but those billions of rules make it appear as though GPT-3 can do things only humans can do. Mo rules, mo smarts.

But GPT-3 is over 18 months old – dog years in the rapidly evolving world of artificial intelligence. Earlier this month, Microsoft and Nvidia (who make the pricey display cards hardcore gamers prefer) unveiled the latest, greatest and biggest program, MT-NLG. MT-NLG has over half a trillion parameters within its model of human language – three times the number on offer from the suddenly shabby GPT-3. What does that get you? Expanded powers of inference. In one example, Microsoft fed MT-NLG a statement, then asked it a question:

Famous professors supported the secretary.

Question: Professors supported the secretary. True or False?

MT-NLG replied:

True. The secretary was supported by famous professors.

That’s all the more significant because MT-NLG showed its work, explaining why it answered as it had.

In the same way the steam engine relieved human beings of the mechanical drudgeries of work, artificial intelligence looks to be fulfilling its promise to relieve the drudgeries of desk work.

Much of what we read consists of factual statements. MT-NLG, like GPT-3 before it, can digest these factual statements, draw inferences, then make decisions based on those inferences. Is it “understanding”? The question we need to ask at this point isn’t the unanswerable, “Does MT-NLG understand what it’s reading?” But rather, “Does MT-NLG have enough rules to give the correct answer?” The answer there is (mostly) yes.

Microsoft recently bought GitHub; Copilot and MT-NLG look to be on a collision course. This means the kinds of suggestions Copilot provides programmers will soon get even better. Will this put programmers out of work? That seems unlikely. Instead, programmers will be able to focus on the interesting bits, using Copilot to provide workable solutions to all of the necessary “boilerplate” within every computer program.

In the same way that the steam engine relieved human beings of the mechanical drudgeries of work, artificial intelligence looks to be fulfilling its promise to relieve the drudgeries of desk work. It won’t be long until we have tools analogous to Copilot employed for basic business communication – drafting simple press releases, answering customer emails, and so forth. Automation is coming to the boring bits of our white-collar lives, leaving us with the interesting bits – the weird and unexpected, those things no computer has ever learned, nor any human has ever seen. Humans are good at dealing with exceptional circumstances – and with some help from AI, we’ll have more time to get better.