Artificial Intelligence, Society, Technology

AI is spiralling out of control: it can be stopped

ARTIFICIAL SUPERINTELLIGENCE

Intro: East and West collaborated to end nuclear proliferation – it is time to do the same for the latest advancing technology. Washington and Beijing must come together to rein in AI’s growing threat

After the Cuban Missile Crisis brought the world to the edge of nuclear war, global powers embarked on a concerted effort to pull it back from the brink. The non-proliferation treaty (NPT) of 1968, which limited the spread of nuclear weapons, has been a resounding success. Only a handful of countries today have access to the 80-year-old technology and those that do have not used it.

In the decades since, no technology has proved as dangerous as nuclear weapons as to require international co-ordination.

Now, however, many believe that the advance of artificial superintelligence requires a similar global effort to prevent an AI-led disaster.

Anthropic, the world’s most valuable AI company, has called for a mechanism to slow down or pause the development of advanced AI. It has warned that the technology could get out of control sooner than many think.

The company believes it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. It says it would “likely be a good thing” if development could be delayed.

Anthropic – recently valued at $965bn (£720bn) – said it had raised the alarm because it believed AI was improving much faster than our ability to understand and control the systems.

Within the company itself, bots are not just writing code; they are also ordering around other bots and even carrying out their own research. Before long, AI could be building itself, a process called recursive self-improvement. This could start a feedback loop in which progress goes parabolic.

Sceptics insist this is just mere marketing. Anthropic has announced that it has filed for an initial public offering and is expecting a value in excess of $1tn. What could be more valuable than a technology so powerful that world leaders need to rein it in? AI that builds itself has been a premise the company has used to raise money for years.

David Sacks, a high-profile critic of Anthropic, and Donald Trump’s former AI tsar, suggested the warning was an attempt to secure a public bailout, implying it was a sign of getting the frontier AI lab nationalised.

Nonetheless, concerns about powerful AI are becoming increasingly prominent. Anthropic has kept its most powerful AI system, Mythos, out of public hands because of its ability to find security flaws in critically important computer systems.

Andrew Bailey, the Governor of the Bank of England, has raised the alarm about AI crashing the financial system and has warned that Mythos meant “things that we thought might happen in the next year, two years, three years or four, have now come right into the foreground”.

AI labs fear that the next generation of models will be good enough to help terrorists develop bioweapons.

If AI were to start building itself without human oversight, it would by definition become much more difficult to control. In the extreme scenarios that safety experts are concerned about, AI’s goals become detached from our own, forcing it to eliminate humanity through evolution so that we do not get in the way.

There are those who dismiss this idea as sci-fi nonsense. But supporters of a pause say even a tiny chance of extinction should be enough to make us consider how to stop it.

Establishing the need for a pause would be the easy part. Making it happen is another matter altogether. If he so wished, Dario Amodei, Anthropic’s chief executive, could send everyone home today and shut down his company. At best, though, this would delay the rise of powerful AI by a couple of months. Its two major rivals, Google and OpenAI, are not far behind. OpenAI, the developer of ChatGPT, has said that it too sees “early signs of RSI [recursive self-improvement] in today’s systems”.

It added: “We expect this to increase competitive pressures among developers and nations, and create governance challenges that existing institutions are not equipped to address.”

Even if the US government ordered all three to stop work on AI, this might only cede ground to China, whose companies are typically seen as being just three to six months behind the US.

Earlier this year at the World Economic Forum, Amodei said: “The reason we can’t [slow down] is because we have geopolitical adversaries building the same technology at a similar pace… It’s very hard to have an enforceable agreement where they slow down and we slow down.”

Practically, it would require a government-level agreement and the two nations that matter are the US and China. This sort of agreement would require Trump and Xi Jinping to co-operate on a pause, something that looks far from likely given both have compared AI to a race.

Xi has said that China must “gain a head start and secure a competitive edge” in AI, while a Trump administration action plan states that “America is in a race to achieve global dominance in artificial intelligence”.

It has also emerged that the National Security Agency have been using Mythos to carry out cyber-attacks. This suggests the US government is making enthusiastic use of the latest systems instead of fearing their consequences.

Pessimists often compare the technology and its potential consequences to nuclear weapons, but the two are nothing alike.

The destructive capabilities of atomic warheads are undisputed, whereas AI’s safety risks can appear nebulous. The latter’s upside may also be significant: its supporters believe it can cure disease, lead to interstellar space travel, and make work optional.

What is more, pressing pause on the AI race is not without its own set of risks. Suspending work on AI could cause an economic crash. The chips and data centres that AI relies on have driven a stock market boom that has helped sustain the US economy. Inhibiting demand for them could do the opposite.

There have been signs that China and the US are changing tack. The White House has raised the alarm about Mythos and Trump has just signed an executive order calling for AI models to be reviewed before release.

Beijing has called for a “global AI governance framework” to rein in the technology. This is miles away from the global deal Anthropic has called for, but campaigners have taken it as a positive sign.

The political zeitgeist can move very quickly. The US and its allies have succeeded to a certain extent in deterring nuclear proliferation. To do so similarly with AI is going to be hard, but as we have seen with nuclear weapons, global governance can come together and work for the common good.

Standard
China, Digital Economy, Economic, International trade, United States

Essay: Hormuz has exposed flaws in Beijing’s economic model

CHINA’S ECONOMIC MODEL

Intro: As the US president prepares to visit Beijing, he may have ended China’s decades-long growth miracle. Trump’s war on Iran, and especially the closure of the Strait of Hormuz, has China in a chokehold

In the modern era, we have witnessed among many things, a robot that can win a half-marathon, a pilot-less plane, a soap opera made by AI, and an electric supercar with a top speed of 308 miles per hour.

Whenever a new bit of Chinese-made tech goes viral online, it seems to be something that goes further, faster, or smarter than ever before. And every time, it feels like a silicon-made metaphor for China’s coming of age as a global superpower.

As Donald Trump prepares to visit Beijing this month, he confronts a Communist Party regime that, on the face of it, looks on the up. Beijing controls access to critical minerals used by his military, and holds at least $700bn (£517bn) in outstanding loans to his government.

Little wonder that, even as he pours scorn on Sir Keir Starmer, Emmanuel Macron, Friedrich Merz and Mark Carney, Trump deferentially refers to his Chinese counterpart Xi Jinping as “the Highly Respected President of China”.

However, that doesn’t mean Trump is making things easy for Xi.

First he choked off the half-million daily barrels of oil that China was buying from Venezuela.

Now, his attacks on Iran, leading to the closure of the Strait of Hormuz, has squeezed another 40pc of Chinese crude supply.

An oil crunch is never good news, but Beijing was prepared for this. Through price caps, export bans, fuel stockpiles and home-grown alternatives to imported oil and gas, China has managed to stave off the early economic threat from the Iran War.

If the conflict drags on and the Strait of Hormuz remains closed, however, China could start to feel the pinch.  

A global slowdown or recession could cut demand for its exports, which have been almost single-handedly propping up the Chinese economy ever since the country’s great property crash of 2021.

And if that happens, Xi may struggle to keep a lid on three ticking time bombs sitting beneath China’s economy: its reliance on bloated factories churning out exports; its spiralling budget debts; and its alarmingly shrinking population.

Xi hopes to diffuse these problems by winning the global race on artificial intelligence, quantum computing, and robotics. But the Communist Party is also eager to stay in control, so it may struggle to make the most of this disruptive juggernaut.

The war against Iran isn’t going to plan for Trump. Yet whatever his intentions were, he may have triggered the tripwire that will blow up China’s decades-long growth miracle.

And this won’t be the only consequence of the Iran conflict that Xi is currently weighing up.

As ever, his thoughts will have quickly turned to Taiwan – an island whose reunification, he has previously said, is a task for his generation to fulfil.

He might like what he sees. The war in the Gulf may divert America’s attention, money, and resources away from the Pacific. It may strain alliances in which Taiwan has put faith.

If Iran’s asymmetric resistance to superior US forces is a play-book for Taipei, then Beijing can pick up strategies and tactics on how to push past it.

Chatham House, a think tank, says that a world in which the Trump administration is expressing a lot of transactionalism, and with the US maybe diminished by its failure in Iran, all of this opens up possibilities for Beijing vis-à-vis Taiwan.

Perhaps this is why Xi has been relatively quiet on Iran. Since the war began on February 28 he has only once picked up the megaphone, urging a swift reopening of Hormuz. This is likely to remain his top priority – the Chinese economy, and his regime’s legitimacy, may yet depend on it.

Economic shock

On Zhihu, a Reddit-style Chinese social media site, users enjoying a cloak of anonymity are swapping stories on the impact of the US-Iran war.

One user wrote: “My industry uses plastic as a raw material… Since plastic is a derivative of oil, the cost of raw materials has doubled and changes three times a day, with no stable price. As a result, global customers have stopped placing orders because they dare not order.

“Work has stalled, and the long-term high-pressure work is exhausting. I can’t change natural disasters or man-made calamities. Fine, I’ll go relax for a few days. But I realised that air ticket prices have also gone absurdly high, so I can’t go out.

“I have no choice but to squat every day in front of the Longjiang Braised Pork Leg Rice stall downstairs, asking, ‘Is it half price after 8pm?’”

China is powerfully exposed to the current of high costs rippling out from the closure of the Strait of Hormuz, the vital conduit for oil, gas, petroleum products, and fertiliser. China imports about 70pc of the oil it needs, of which about half comes from the Persian Gulf. Its shipments from there slumped by a third in March from a year earlier, and that number doesn’t even include the illicit Iranian oil.

On the gas front, about half of China’s gas imports come via pipelines from Russia, Myanmar and central Asia, with the rest arriving on LNG tankers. A third of that LNG comes via Hormuz, and imports plunged by 41pc in March.

For now, the damage may be less than these numbers might suggest, because China has spent the past decade building a huge and still growing domestic gas industry.

This insurance policy means home-grown output now satisfies almost two-thirds of China’s gas demand, so the Iran War has disrupted just 5pc of China’s total supply. It looks as if, in some ways, China has been preparing for this type of situation better than anybody else.

The Chinese have been similarly savvy with crude oil. Last year it bought 11.6 million barrels per day (bpd), taking advantage of lower prices to stockpile almost half a million bpd. By the end of 2025, that reserve totalled almost 1.4 billion barrels, vastly more than the rest of the rich world put together.

If China released supplies from this trove of black gold at a rate of 2 million bpd, a level that would largely negate the Hormuz closure, the stockpile would still last until at least the end of next year.

Still, the Chinese aren’t leaving anything to chance. Beijing told its refineries to stop exporting petrol, diesel, and jet fuel – in case supplies run low. But this has rebounded on Beijing. The move irked trading partners such as Vietnam, the Philippines, Singapore and Australia, whose fuel shortfalls have been exacerbated by China’s parsimony.

China appears to have now acknowledged that its stockpiles are almost too big, and that it needs to mend some fences. According to Bloomberg, exporters have now had approval to ship 500,000 tons to regular customers.

China’s biggest vulnerability to Iran, lies in its reliance on oil-based products like naphtha, which is vital to greasing the mighty Chinese industrial machine.

If you go back far enough in the chemical processing chain, or petroleum products, that’s where you’ll see the prices rise. Access to that is a continual problem, and that’s where the greatest fear is.

Producer price inflation, which was at or below zero last year, hit 1pc in March – the highest since the pandemic. Businesses, however, can’t pass these costs on to customers, largely because they are not spending.

Retail sales rose an annually sluggish 1.7pc in March, and in April car sales dropped by more than 25pc. The household savings rate was 37.8pc in the first quarter, the highest recorded (with the exception of the pandemic).

Consequently, if the situation continues to escalate and oil prices keep rising, then there is going to be some damage.

Exports to the rescue?

China’s get-out-of-jail card is its exports. The country’s use of coal, wind, and solar has insulated its producers from the global energy shock, helping to keep export prices competitive.

Export growth slowed to 2.5pc year-on-year in March, but a key indicator of activity at export-oriented firms in April hit 52.2, its highest level in more than five years.

That buoyancy in part reflects Chinese factories’ dominance of the supply chain for wind turbines, solar panels, batteries, and electric cars. The soaring oil price has brought green tech back into worldwide fashion.

Exports of solar cells doubled in March, and were up by more than a third for lithium-ion batteries.

Nonetheless, this tech, together with electric vehicles known in Beijing as “the new three”, still comprises just 7pc of China’s exports. The wider trading picture will be more worrying for China.

If the Strait of Hormuz remains closed for several more months, the blow to the global economy will get exponentially bigger – and the appetite to buy goods from China may wane. Exports to the Middle East are already suffering. On Zhihu, one post related how a toy exporter had seen a 20pc to 40pc drop in demand. “Even Saudi Arabia and the UAE are waiting and watching, afraid of escalating conflicts. No one dares to act recklessly.”

The real pain could come from south-east Asia. It has been a crucial shock absorber for Chinese exporters who have had to drop out of the US market since Trump returned to the White House. But many of these countries are Hormuz-dependent, and the war has hit them hard.

If its trading partners don’t want to buy so much, then China won’t be selling so much either.

This is the tripwire that could set off China’s first time-bomb. Beijing has spent decades subsidising its industries to produce more and more, but China’s massive domestic consumer base simply refuses to spend.

This means the only place for factories’ over-production is the world market. But, if demand for exports falters, the cracks in the Communist Party’s model will open wider and wider.

The trouble is, the only way out of this impasse could be a worldwide trade war.

A broken economic model

China’s most well-known company, these days, is probably the electric car giant BYD. In just two decades, it has risen to the pinnacle of the global EV market. Yet, all is not well. Its first-quarter profit is down 55pc, to a three-year low. Its inventory of unsold stock has stacked up by 16pc since the start of the year.

BYD and fellow carmakers are capable of producing more than twice the number of vehicles each year that they can actually sell within China.

At home, this forces BYD and its rivals into a price war, squeezing margins. Domestic sales, though, have still declined for seven straight months.

Meanwhile, exports are booming. BYD’s overseas sales jumped more than 50pc in the first quarter, accounting for 45pc of all sales.

Its market share in Europe might still be only 2.2pc, but aggressive discounting in Germany has helped more than double this figure in the past 12 months.

BYD’s story is a microcosm of the successes, failures, and risks of China’s economic model.

The system builds powerful companies, but relies on the rest of the world to pick up the tab.

China’s trade surplus with the rest of the world hit $1tn last year. This is, in effect, a monetary gauge of the de-industrialisation of the West, and the stymieing of industrial development in poorer countries.

The rest of the world’s tolerance for this is gradually waning. Trump may have stepped back from his tariff war, but many observers suggest it’s only a matter of time before China provokes a wider backlash.

China’s strategy is always outbound, export driven, because they don’t have the demand. So, it’s not going to change. But of course, individual countries could change, the world could change.

The reckoning is already approaching. Countries are realising that they need some economic security. That means maybe not getting the absolute rock bottom price but having to pay something in order to have diversified suppliers.

The only way Beijing can head off the consequences of an export downturn would be to bolster demand at home. However, this is now a deeply entrenched problem.

China’s consumer confidence index stood at 92 points in February. That was a three-year high, but before the property crash the index was never below 100. It appears that the Chinese do not want to buy consumer durables; they buy consumer staples, cheap goods that are consumed every day.

Households face insecurity on every front. Their real incomes grew 4pc in the first quarter, but have been slowing for years from the 6pc-plus rate pre-Covid.

House prices, which plunged after the authorities called time on a property bubble in 2021, are still falling everywhere except Beijing, Shanghai, and a handful of other large cities.

Property is still the biggest asset held by Chinese households. If housing continues shrinking in value, that just works against the authorities’ other efforts to try and stimulate consumption.

The jobs market is also failing to inspire confidence. The unemployment rate of 5.4pc is at a three-year high. The youth jobless rate has been improving, but is still high at 17pc.

Graduates of elite universities are still finding jobs, and some businesses are still hiring, however life is tough for young people further down the ladder.

The government has tried to stimulate consumption, but in a piecemeal fashion. Its signature effort last year was a trade-in scheme for white goods, which delivered at best a mild and temporary boost.

Many hoped that a recent meeting of the Communist Party’s Politburo would recognise that the economic model was unsustainable. Disappointment followed.

There were no concrete measures or targets around anything that would boost household consumption, say analysts at Capital Economics.

Without targets for lower tiers of government, they say, nothing will change.

“A lot of the high-level policies are focused on boosting domestic self-sufficiency, on AI and tech development… It feels like those will continue to be the priorities. That might come at the cost of, or instead of, boosting support for households.”

Beijing has apparently decided that its export-oriented model won’t trigger a trade war, or hobble the profits of its companies.

Some say the regime may see more risk in changing course than in sticking to what it knows, and projecting stability. However, sitting on their hands represents the real risk.

As soon as Western countries start threatening demand, China will be severely hampered. It will mean factory closures. It will mean spiralling deflation, on top of the deflation we’re already seeing because of overcapacity and over investment. This will become catastrophic for the average Chinese household.

Two more time-bombs: No babies, no budget

The deep-seated consumer anxiety may be one reason why many Chinese are choosing not to have children.

A recent Rhodium report noted that the 7.9 million births last year was half the number a decade ago. Births are at 1939 levels – below even the time of the one-child policy (which ended in 2016).

As the death rate creeps up, the country will lose nearly 60m people in the next decade – a population almost the size of France.

Beijing has offered a 3,600-yuan (£390) subsidy per child for three years, starting last year. But government policy seldom reverses a demographic decline.

The population loss is going to make the problem of domestic under-consumption much, much worse.

That’s going to present a massive headwind to China’s economy, and that’s why they will be hoping to gain productivity benefits from new technology like AI, to offset that.

Rhodium says this will create a massive budget headache for Beijing, as the government is forced to support an older population with fewer taxpaying workers.

However, Beijing is already fiscally neuralgic. Government revenue is now almost static, with tax income flatlining and land sales declining.

The public debt-to-GDP ratio is about the same as Britain’s. The budget deficit was 9pc of GDP last year, and could widen further. By 2030 this would be entering crisis-level territory.

Meanwhile, the public and private lending that washes through the Chinese economy, putting China “in a league of its own”, according to Capital Economics, is “a signal that all is not well”.

Rhodium’s warning is even more stark: “The era of fiscal trade-offs has already arrived, and the stress is deepening . . . This is what fiscal and financial decay looks like.”

Searching for a prop

Beijing hopes that AI and robots will defuse these time-bombs. Tech-fuelled productivity growth could prop up the economy if China’s population is falling and the world is turning away from its export wares.

China has one big advantage. In a command-and-control economy, the Communist Party can get businesses to adopt AI, and can build out the infrastructure to support it.

The Chinese also seem to be taking to it very quickly, to a greater extent than the average consumer in the US or in Europe. AI is being directly embedded into the commercial stacks that people use on a daily basis, the e-commerce or content platforms that the Chinese use.

The government is also on board. Technology isn’t just an economic panacea; it’s a must-have in Beijing’s relentless quest for self-reliance.

Just days ago, Ding Xuexiang, the vice-premier, said: “China’s AI must take the path of independent innovation, to have the confidence to withstand any containment or suppression”.

And yet, the Communist Party may also be its own worst enemy. Beijing’s refusal to rely on America for chips – instead pushing companies to use domestic Huawei alternatives – is holding back development.

Officials will also be fretting that AI might destroy more jobs than it creates – an outcome the regime wouldn’t be able to stomach.

What you might see is a lot of companies adopting AI, seeing productivity improve, but then not being able to follow through with the kind of lay offs that would result from that in most other economies. And, then, you don’t actually see a boost to labour productivity.

Beijing also has a broader insecurity about technology: it must rely for innovation on the private sector, which the leadership simply doesn’t trust.

The feeling is mutual. Xi’s 2020 attack on Jack Ma, founder of the fintech firm Ant, left lasting scars on China’s entrepreneurial class.

Such fears have been renewed with Beijing kyboshed Facebook owner Meta’s proposed merger with Singapore-based AI firm Manus. Manus’s Chinese founders had moved the company to Singapore, but not beyond the reach of the Communist regime. Some say that if you’re a private entrepreneur in China, what incentives do you have in trying to go global, if this can always happen, and the Party can just come in and say, ‘All of your work, it’s ours now’?

Other analysts are of the opinion that the Xi regime’s innate fear of losing control will be its undoing, and hobble its tech push. The signature feature of Xi Jinping and his administration is how conservative it is. It is not a radical, progressive, transformative regime. In the back of their mind, they’re always thinking, ‘We can’t change too fast’.

China has built a robot that can win a marathon. The question, though, is whether it can build an economic model that can keep the country competitive.

Trump’s war in Iran, if it lasts well into the year, may force out an answer.

Standard
Defence, National Security, Nuclear Weapons

Fears of a more dangerous Cold War

NUCLEAR WEAPONS

Intro: As weapons limits expire, the scene is set for a new nuclear arms race between the US and Beijing

Around midday on October 30, 1961, a Soviet plane flying above the Arctic archipelago of Novaya Zemlya dropped the most powerful nuclear bomb ever created.

The USSR’s “emperor bomb” was 3,000-times more powerful than the US atomic attack that killed 140,000 people in Hiroshima two decades earlier. On explosion, it unleashed a six-mile-wide fireball and a mushroom cloud that loomed more than 40 miles into the sky. And the Soviets were testing it at only half of its designed capacity.

Since then, decades of negotiations and arms-control treaties have massively reduced American and Russian warhead arsenals, with neither side testing a nuclear bomb in more than three decades.

But the last of these bilateral agreements has expired – and, with it, hopes that the nuclear arms race had been consigned to the history books.

The New Strategic Arms Reduction Treaty (START), which capped the number of deployed nuclear warheads held by the US and Russia, has now come to an end.

It is the first time since the 1970s that the two powers have had no agreement in place without at least negotiations for a new treaty under way.

At a time of huge geopolitical upheaval, analysts and diplomats are concerned that the stage is set for a new nuclear arms race – one that could prove even more dangerous than the world has seen before.

This is because the competition will not just be confined to Russia and the US.

China has also been developing nuclear weapons at a startling trajectory. It has more than doubled its stockpile of warheads over the last six years.

A three-way race will be hugely destabilising for the world order. If America tries to build an arsenal large enough to deter its twin foes at once, it will spur an even more dramatic increase in their respective stockpiles.

The director of the Project on Nuclear Issues at the Centre for Strategic and International Studies (CSIS), says that although this is the end of an era, it is not the end of arms control “but it is definitely the end of arms control as we know it.”

Smaller nuclear powers such as Britain and France will also face pressure to bulk up, particularly at a time when US security guarantees feel less reliable. And there will likely be a proliferation of new nuclear states.

Donald Trump has insisted for decades that he wants denuclearisation. But he seems to have no strategy in delivering this. His plans to build a new missile defence system – which he refers to as the “Golden Dome” – are only fanning the flames.

Standard