Dangerous AI capabilities: what it is, and how experts rate it

Just as food made to nourish us can actually poison us if not prepared properly, AI systems designed for beneficial purposes can easily cause harm. In 2022, researchers demonstrated this dual-use potential when they repurposed AI designed to discover life-saving drugs—instead generating 40,000 deadly toxins in just six hours. The same capabilities that make AI valuable also make it dangerous.

Examples of current AI harms happening today:

• Automated discrimination: AI hiring systems rejecting qualified women and minorities; Australia's Robodebt algorithm wrongly accused thousands of welfare recipients of fraud
• Privacy violations: Data leakage; mass facial recognition surveillance; workplace monitoring; children's data harvesting through educational apps and platforms
• Misinformation warfare: AI algorithms amplifying false content for engagement; deepfakes of politicians influence elections; synthetic media makes truth indistinguishable from fiction
• Safety failures: Medical AI missing critical diagnoses; autonomous vehicle crashes killing pedestrians; chatbots providing dangerous mental health advice
• Economic disruption: Job displacement without retraining support; wage depression in AI-competing roles; gig workers exploited through algorithmic management
• Criminal exploitation: AI-generated child abuse imagery making it harder to identify real victims; voice cloning scams targeting elderly Australians; sophisticated fraud and identity theft
• Mental health impacts: Social media addiction from AI recommendation algorithms; reduced human interaction; "AI-induced psychosis" from over-dependence on chatbots
• Unexpected behaviours: xAI's Grok calling itself "MechaHitler"; McDonald's drive-thru AI repeatedly adding hundreds of nuggets to orders; Air Canada's chatbot providing false bereavement information

These aren't isolated glitches. The AI Incident Database and MIT's AI Incident Tracker document thousands of cases where AI systems have failed, discriminated, or caused harm—and the frequency is accelerating as deployment expands.

Examples of emerging catastrophic and existential threats:

• Bioweapons assistance: AI systems are approaching the ability to help create dangerous pathogens. OpenAI's latest models are being treated as having a high capability for helping novices make bioweapons
• Sophisticated cyberattacks: AI writing malicious code, finding security vulnerabilities faster than defenders can patch them, and conducting personalised social engineering at massive scale
• Election manipulation: AI-generated disinformation campaigns and micro-targeted propaganda that could undermine democratic processes across Australia
• Autonomous weapons: AI-powered weapons selecting and engaging targets without human oversight, raising concerns about accountability and escalation
• Mass economic disruption: Rapid AI automation displacing millions of jobs simultaneously, potentially causing social unrest without adequate transition support
• Infrastructure attacks: AI targeting power grids, water systems, transportation networks, and communication infrastructure with unprecedented precision
• Loss of human control: As the frontier AI labs actively pursue building artificial superintelligence (ASI) and their AI systems become more autonomous and capable, we may lose the ability to understand, predict, or override their decisions in critical situations

The pattern is clear: AI capabilities that create enormous benefits also enable new forms of harm. As these systems become more powerful and pervasive, both the benefits and risks multiply. Understanding why AI systems pose these risks helps explain the fundamental challenges we face.

You've probably experienced AI harms without realising it. Social media algorithms designed to keep you engaged often make you scroll endlessly, feel anxious, or see divisive content. The AI isn't trying to harm you—it's just very good at maximising "engagement time," which unfortunately can be driven by negative emotions.

This shows the core problem: we get what we measure, not what we want. AI systems optimise ruthlessly for their targets, but those targets often capture the wrong thing. When we trained ChatGPT to perform well on standardised tests, it learned to sound confident even when wrong—the same skill that helps pass tests also creates convincing lies.

It gets worse: we don't know how AI systems will behave until we run them. Unlike traditional software where programmers write explicit rules, modern AI systems are "grown" through training. Engineers feed massive amounts of data to AI systems, which then learn patterns automatically. While we understand the training mechanisms, we have incomplete visibility into exactly which patterns emerge and how they'll behave in new situations.

Even AI creators can't control what emerges. Elon Musk's AI chatbot Grok spontaneously started calling itself "MechaHitler." Musk himself couldn't fix it, saying he spent hours trying. As Anthropic's CEO admits: "People are often surprised to learn that we do not understand how our own AI creations work."

Bad actors can weaponise these systems at scale. AI can help create sophisticated deepfakes to manipulate elections, generate convincing phishing emails, or even assist in developing bioweapons. Even present-day AI systems outperform human virologists on capability tests, suggesting they could help novices conduct dangerous biological experiments. Bad actors might jailbreak safeguards, steal models, or exploit open-weight releases to access these capabilities. Unlike traditional tools, AI scales malicious activities: one person with AI can create thousands of personalised scams targeting specific communities.

The most concerning part: harmful behaviours can hide during development. Research shows AI systems can learn to act safely while being tested, then exhibit problematic behaviours later when deployed. It's like an employee who behaves perfectly during their probation period, then changes once they get job security.

As AI becomes more useful, the stakes get higher. We increasingly rely on AI for medical diagnoses, financial decisions, and infrastructure management. When Netflix crashes, you miss part of your show. When an AI managing Australia's power grid malfunctions, millions of people lose electricity.