ARTIFICIAL INTELLIGENCE

Intro: Humanity faces an uncertain fate as experts brace for superintelligent AI. The tech industry claims looming “singularity” will change everything

Every time one of the world’s top artificial intelligence companies unveils a new system, employees at the US research organisation METR put it through its paces. Its ability is tested to complete a series of increasingly complex tasks.

The tasks are measured by how long each one would take a skilled human. They range from trivial arithmetic (two seconds) and completing a game of Wordle (13 minutes) to building complex military satellite software (taking a human expert 14.5 hours).

The test then serves as a gauge as to how capable AI has become – and where it might go.

The first version of ChatGPT, released in 2022, could only perform simple tasks that would take a human a few seconds.

But as AI systems have become more powerful, they are able to complete more complex actions that would take humans hours or days, such as breaking into a medical website and downloading all its data.

METR has found that AI capabilities are doubling in power every 196 days. Plotted on a graph, this progress starts slowly then rapidly accelerates to a near-vertical plane.

Converse with anyone in the AI industry for any length of time and the likelihood of them pulling up a version of the chart approaches 100pc, to the point where it has become a meme in its own right. It is being referred to as the most important chart in the world. The chart goes off the scale.

Last month, the AI lab Anthropic announced it had developed a new system, called Mythos, that it said was too powerful to release to the public because of its ability to find gaping holes in online security systems.

When METR’s researchers released the results of Mythos’s capability and function, they scored the system at 16 hours – meaning the world’s most powerful AI can now automate tasks that would take a human two full eight-hour shifts.

Nonetheless, they said the model was “at the upper end” of their ability to test. In other words, progress has become too fast for them to measure.

Not everybody is convinced by the results because the test only measures if a machine can do something half the time, not if it can do it consistently. The METR chart has, however, captured many people’s imaginations for two reasons.

First, the exponential growth looks strikingly similar to “Moore’s Law”, the maxim that has governed the electronics industry for more than half a century, stating that microchips roughly double in power every two years.

Second, it measures abilities, rather than intelligence. While many AI “benchmarks” resemble university exams and gradings, dealing in abstract reasoning or maths, the METR test studies whether AI can actually work.

It suggests that on current trends, vast amounts of human tasks could be automated in the next couple of years – including, most crucially of all, the art of developing AI models itself.

At that threshold, known in the tech industry as “recursive self-improvement”, all bets are off.

The concept is closely linked to superhuman AI because an AI that can make itself smarter could act like an evolutionary chain reaction, rapidly building to a system vastly more capable than mankind.

AI would have become – as IJ Good, the Bletchley Park codebreaker, predicted in 1965 – “the last invention that man need make”. Almost Orwellian in thought.

For 60 years, the idea seemed out of reach. But much of Silicon Valley believes this is about to change – and the US government is starting to notice.

The vast majority of people’s experience of AI has not changed much in the last couple of years. The release of ChatGPT in 2022 generated an initial flurry of excitement and fear in equal measure but, since then, progress has been less obvious.

The AI experience for many people comes in seeing an obviously fake video on their social media feeds, seeing an AI overview at the top of their search results, or having a bot that “helpfully” offers to summarise their emails.

But at the coalface, people are rapidly bringing forward their timelines for the day that superintelligence arrives.