Category Archives: Science

Artificial Intelligence, Science, Society, Technology

Tech bosses have hidden motives in slowing AI progress

ARTIFICIAL INTELLIGENCE

Intro: What’s the real reason tech bosses are warning us about their AI creations? It’s certainly nothing to do with believing we are on the verge of wiping out humanity

The terrifying message from Silicon Valley is that we’re all doomed as the developers of artificial intelligence foretell the apocalypse for which they will themselves be responsible.

A high-profile British employee of Anthropic announced he was leaving over fears his company and its main competitor, OpenAI, were “gambling with our lives” by rushing to create a digital “superintelligence” that could destroy humanity.

Some working in AI – known as “doomers” – have expressed similar fears, but the warning by Anthropic’s departing employee went viral with many others in the company swiftly echoing the stark warnings. A senior safety researcher at the same company estimated there was more than a 10 per cent chance that AI “could kill all humans” within a decade.

Anthropic boss Dario Amodei largely agrees with the sentiments, saying AI had been “advancing drastically faster” and also called his industry to “slow down” and accept external regulation.

OpenAI boss Sam Altman (creator of ChatGPT) and AI developer Elon Musk followed suit.

Amodei acknowledged that the US government recognises the West is racing to develop superintelligent AI before China and other “autocracies” and that pulling back amounted to a “significant national security risk”.

But he warned that after a notorious incident this summer – in which a “swarm” of AI agents belonging to OpenAI launched sophisticated cyberattacks on rival AI company Hugging Face – a more capable swarm might “take over the entire internet” within as little as six months.

TWO

The media and Washington establishment have been listening agog to these apocalyptic pronouncements.

So, are we all genuinely doomed – or might there be another explanation for all this terrifying recent rhetoric?

Donald Trump certainly believes so, sticking to his firm beliefs on pushing ahead with AI. Critics insist that he is only thinking about himself: America has crucial midterm elections in November and Trump, they say, doesn’t want to do anything to harm the vast investment in AI that has pushed the US stock market to record highs and buoyed the country’s economy.

This may indeed be part of his thinking – but that doesn’t necessarily mean the President is wrong in wanting to pursue AI development.

Many experts claim that all those issuing dire warnings about AI being an imminent threat to the human race are wildly exaggerating. The doomers’ predictions, they argue, depend on AI achieving superintelligent, superhuman abilities – primarily by AI learning how to improve itself, a process known as “recursive self-improvement”.

We are, however, nowhere near such superintelligence, say critics in the tech world.

The doomers have also been accused of ignoring the essential truth that AI software and mathematical models aren’t innately malicious or rebellious but – as with the Hugging Face hacking scandal – are simply trying to meet objectives set for it by its human designers.

The risks of AI, they counter, are vastly outweighed by the potential benefits, from economic productivity to medicine, military gains, and much else.

These sceptics have dismissed previous warnings by Amodei and others as attempts to get free publicity – and they certainly might draw in investors and customers attracted by all the talk of how devastatingly powerful AI may be.

Critics believe there are other – rather more selfish – reasons behind the calls from Anthropic and OpenAI to slow down research and have their industry better regulated.

The pair may be the biggest players in AI but a multitude of rivals are trying to catch up.

Some industry observers have suggested that Anthropic and OpenAI may be trying to preserve their substantial lead by suddenly demanding a general moratorium on research and thus cementing their duopoly.

They may have additional financial motives, too. Both companies have shelled out vast sums on AI research and development but are struggling to find paying customers.

So when ChatGPT owner Sam Altman – an entrepreneur hardly famous for his ethical approach – announced a few days ago that he was pausing OpenAI’s stock market launch for another year, supposedly amid concerns over AI safety, some whispered that he had simply latched on to the controversy as an excuse to delay matters until his company is in a better shape.

David Sacks, a venture capitalist who served as Trump’s “AI Tsar” and now co-chairs the President’s council of science and technology advisers, has observed waspishly that AI leaders need to “stop pretending the motivation to ‘slow down’ is purely altruistic”.

Sacks, who like others in Trump’s administration believes that claims of an AI apocalypse are a thinly-veiled attempt to damage the Republicans in the mid-terms, has dismissed the gloomy predictions as scaremongering.

“We’ve seen this movie before,” he told Bloomberg. “We saw it with global warming: they’ve taken some legitimate concerns and blown it out of proportion.”

And there are further reasons why many are increasingly taking the doomsayers with such a hefty pinch of salt.

The most obvious question is: if these bosses genuinely believe there is a high risk that AI could imminently kill us all, why haven’t they stopped already?

Sceptics also point out the supposed scenarios for AI Armageddon are unconvincingly woolly. Ask doomers how AI will wipe us out and they become vague.

A lot of the theorising hasn’t gone much further than the notorious “paperclip maximiser” advanced by Oxford philosopher Nick Bostram in 2003.

He outlined how a superintelligent computer, asked to focus entirely on producing as many paperclips as possible, would take the instruction to a total extreme, co-opting the world’s entire resources and destroying humanity after deciding they were an obstacle to making more clips (having first harvested the iron in our blood to make more paperclips). Thought-provoking, perhaps – but surely fanciful.

Some have suggested that AI could detonate a nuclear bomb, launch a global drone war, create “misinformation” that could cause one country to attack another, or develop a lethal virus – though how it would physically do any of this is another matter entirely.

Alternatively, it could supposedly wipe us out by starving us of essential resources such as food and medicine. All these scenarios and more have been brandished by the doomers – though with few details on how it might actually happen.

Some experts complain that doomer predictions that the human race could be totally exterminated within years are unfeasible while an AI would find it very difficult to hack the entire internet. The real danger, they say, isn’t AI going rogue but humans misusing it.

Professor Gary Marcus, a renowned cognitive scientist at New York University, says the notion that AI will kill everyone within a few years is mostly “preposterous”.

Marcus believes AI could certainly be used to cripple infrastructure – perhaps hospitals or air or train networks, but that the total extinction of the human race is unfeasible.

He and other sceptics also insist that it would be vanishingly difficult for an AI to hack the entire internet, as Anthropic has now suggested.

THREE

Sceptics also claim it is non-sensical for anyone to try to put a mathematical number on the likelihood of AI destroying us all and emphasise how the doomers hardly help themselves given that their estimates of how long such a catastrophe will take to occur vary so widely.

For instance, Jack Clark, Anthropic’s co-founder, estimates that AI won’t start “doing dangerous stuff” for about 20 years.

Oren Etzioni, a professor at the University of Washington and founder of the Allen Institute for Artificial Intelligence, insists we are “very far away” from AI polishing everyone off – and that the risk of a deadly virus escaping a lab, for example, is far higher.

Some claim that the AI Cassandras have essentially lost all perspective. Having devoted their careers to researching AI because they believe so strongly in its potential, they are more susceptible to over-emphasising its power – and by extension, perhaps, their own importance.

President Trump may be somewhat blasé in dismissing all safety concerns around AI as just a “hoax”.

But it seems premature to be counting the days before our robot overlords carry out the extinction of our species.

Standard
Artificial Intelligence, Research, Science, Society, Technology

Superintelligent AI and its threat to humanity

ARTIFICIAL INTELLIGENCE

Intro: Humanity faces an uncertain fate as experts brace for superintelligent AI. The tech industry claims looming “singularity” will change everything

Every time one of the world’s top artificial intelligence companies unveils a new system, employees at the US research organisation METR put it through its paces. Its ability is tested to complete a series of increasingly complex tasks.

The tasks are measured by how long each one would take a skilled human. They range from trivial arithmetic (two seconds) and completing a game of Wordle (13 minutes) to building complex military satellite software (taking a human expert 14.5 hours).

The test then serves as a gauge as to how capable AI has become – and where it might go.

The first version of ChatGPT, released in 2022, could only perform simple tasks that would take a human a few seconds.

But as AI systems have become more powerful, they are able to complete more complex actions that would take humans hours or days, such as breaking into a medical website and downloading all its data.

METR has found that AI capabilities are doubling in power every 196 days. Plotted on a graph, this progress starts slowly then rapidly accelerates to a near-vertical plane.

Converse with anyone in the AI industry for any length of time and the likelihood of them pulling up a version of the chart approaches 100pc, to the point where it has become a meme in its own right. It is being referred to as the most important chart in the world. The chart goes off the scale.

Last month, the AI lab Anthropic announced it had developed a new system, called Mythos, that it said was too powerful to release to the public because of its ability to find gaping holes in online security systems.

When METR’s researchers released the results of Mythos’s capability and function, they scored the system at 16 hours – meaning the world’s most powerful AI can now automate tasks that would take a human two full eight-hour shifts.

Nonetheless, they said the model was “at the upper end” of their ability to test. In other words, progress has become too fast for them to measure.

Not everybody is convinced by the results because the test only measures if a machine can do something half the time, not if it can do it consistently. The METR chart has, however, captured many people’s imaginations for two reasons.

First, the exponential growth looks strikingly similar to “Moore’s Law”, the maxim that has governed the electronics industry for more than half a century, stating that microchips roughly double in power every two years.

Second, it measures abilities, rather than intelligence. While many AI “benchmarks” resemble university exams and gradings, dealing in abstract reasoning or maths, the METR test studies whether AI can actually work.

It suggests that on current trends, vast amounts of human tasks could be automated in the next couple of years – including, most crucially of all, the art of developing AI models itself.

At that threshold, known in the tech industry as “recursive self-improvement”, all bets are off.

The concept is closely linked to superhuman AI because an AI that can make itself smarter could act like an evolutionary chain reaction, rapidly building to a system vastly more capable than mankind.

AI would have become – as IJ Good, the Bletchley Park codebreaker, predicted in 1965 – “the last invention that man need make”. Almost Orwellian in thought.

For 60 years, the idea seemed out of reach. But much of Silicon Valley believes this is about to change – and the US government is starting to notice.

The vast majority of people’s experience of AI has not changed much in the last couple of years. The release of ChatGPT in 2022 generated an initial flurry of excitement and fear in equal measure but, since then, progress has been less obvious.

The AI experience for many people comes in seeing an obviously fake video on their social media feeds, seeing an AI overview at the top of their search results, or having a bot that “helpfully” offers to summarise their emails.

But at the coalface, people are rapidly bringing forward their timelines for the day that superintelligence arrives.

Pages: 1 2 3 4
Standard
Biotechnology, Health, Medical, Pharmaceutical industry, Science

Big pharma failing to address our greatest medical threat

GLOBAL HEALTH SECURITY

Intro: Drug-resistant infections now kill more people every year than HIV or malaria, yet only six companies remain active in antibiotic research

Writing in the last few days, Professor Lord Darzi, FRS, said that big pharma is failing to tackle our greatest medical threat.

The world-renowned and eminent surgeon says that every caesarean section, joint replacement, and round of chemotherapy depends on antibiotics. In medicine as in war, a successful attack needs a solid defence. Antibiotics are not medicine’s glamourous front line – they are its foundations. And those foundations are crumbling.

Citing that drug-resistant infections now kill 1.27 million people every year, by 2050 the toll could reach eight million. The current mortality rate is more than HIV or malaria. The World Health Organisation (WHO) has warned that one in six bacterial infections is already resistant to standard treatment.

Yet this growing threat has been neglected by the very industry that has the capacity and resources to confront it. The major pharmaceutical companies walked away from antibiotics when they stopped generating lucrative returns. In the 1980s there were 18 companies involved in antibiotic research. By 2020 the number had fallen to six. The rest have pivoted to focus on expensive but highly remunerative medicines to beat cancer and long-term conditions such as obesity.

The ways in which these new medicines attack disease is indeed transformative, but they do not save lives all by themselves. Patients undergoing treatment are at higher risk of infection, but without effective antibiotics, the surgeon cannot operate safely, the oncologist cannot deliver chemotherapy, and the transplant physician cannot suppress rejection.

It is strategically incoherent to innovate relentlessly in attack while underinvesting in defence. The defensive arsenal is not optional infrastructure. It is foundational.

Between 2011 and 2020, US venture capital invested just $1.6bn in antimicrobials, compared with $26.5bn in oncology. The antimicrobial pipeline has declined by 35 per cent since 2021, from 92 to 60 projects, according to the 2026 AMR Benchmark report by the Access to Medicine Foundation, last month. Half are led by GlaxoSmithKline (GSK), which is carrying a disproportionate share of the large-company burden.

There are now only 3,000 active antimicrobial resistance (AMR) researchers worldwide, against 46,000 in oncology. When antibiotic programmes close, 90 per cent of researchers leave the field entirely. The talent and expertise needed for these medicines is collapsing alongside the drug pipeline.

This weakness puts at risk the pharmaceutical industry’s own growth. In 2024, global oncology revenues exceeded $200bn and R&D investment surpassed $40bn. Yet one-third of cancer patients develop bacterial infections during treatment, and up to half of these are now resistant – causing delays, dose changes, and poorer outcomes.

Developing new antibiotics is especially challenging. Most drugs succeed commercially by reaching as many eligible patients as possible. But for antibiotics, good stewardship means reserving novel agents for resistant infections – precisely the behaviour that collapses commercial returns.

In 2020, a consortium of more than 20 major pharmaceutical companies committed around $1bn to bridge the “valley of death” between discovery and profitability by creating the AMR Action Fund. The fund’s ambition was to deliver two to four new antibiotics by 2030. To date, it has delivered one – pivmecillinam, for urinary tract infections.

Bold initiatives such as this $1bn scheme look impressive. But there is a danger of their becoming “guilt capital” – spending that looks responsible but does not change the underlying economics. Without genuine pull incentives, and without adequate investment in diagnostics, stewardship, and surveillance alongside drugs, the spending risks being perceived as reputational insurance rather than strategic investment.

Most tellingly, the fund itself acknowledges it “struggled to find investment opportunities in clinical development exactly because the pipeline is insufficient”. When a $1bn fund cannot find enough assets worth backing, the problem is not capital. It is upstream failure to generate candidates and downstream failure to create a market that rewards success.

The conclusion is quite simple. We need a new approach.

First, build a sustainable pipeline through modern discovery – including AI-enabled research that must prove itself with real-world data – and implement payment models that reward access rather than volume. The UK’s subscription-style scheme is now being expanded. Similar approaches in other countries could create a viable global market.

Second, reduce misuse through transformative diagnostics. Rapid pathogen identification and resistance profiling at point-of-care would cut inappropriate prescribing – the single largest driver of resistance – and protect new drugs from the fate of their predecessors. A deadline should be called: no antibiotic prescription without a diagnosis by 2030.

Third, strengthen stewardship, surveillance, and access so that new antibiotics are protected, monitored, and reach patients appropriately anywhere in the world – particularly in low-income and middle-income countries where the burden of resistance is heaviest.

In 2028, we will mark the centenary of Alexander Fleming’s discovery of penicillin at St Mary’s Hospital in London – a moment that launched the antibiotic era and transformed human health. The centenary should be a moment of celebration. It risks becoming a memorial if action is not taken.

Standard