AI risks: what can go wrong, what’s happened, and what can be done in Australia
From automated discrimination to catastrophic, civilisation-scale harm. AI is powerful enough that we only get the benefits if we handle the risks.
Below, we outline the AI risks we’re concerned about, what the evidence says, and what Australia could do about it.
What risks does AI pose?
Advanced AI can be deliberately misused, it can malfunction, and it can cause harm across whole systems. The International AI Safety Report groups AI risks this way. Below are the risks in each group. Select one to learn more.
Malicious use
Someone using these systems deliberately to cause harm.
- AI disinformation
- AI weapons & cyberattacks
- AI fraud & scams
- AI-generated toxic content
- Loss of consensus reality
- AI-generated false information (also appears under another category)
- AI security vulnerabilities (also appears under another category)
- Dangerous AI capabilities (also appears under another category)
Malfunctions
The systems failing, including in ways nobody intended or can explain.
- AI misalignment
- Dangerous AI capabilities (also appears under another category)
- Capability & robustness
- Transparency & interpretability
- Multi-agent AI risks
- Overreliance on AI
- AI-generated false information (also appears under another category)
- AI security vulnerabilities (also appears under another category)
- AI discrimination
- AI performing worse for some groups
Systemic risks
Harms that come from how widely these systems are adopted, rather than from any single failure.
Risks marked + appear under two categories: a model that states something false is a malfunction, and the same output deployed on purpose is malicious use. AI welfare and rights sits outside them, because the three are framed around harm to people and institutions.
Are these risks happening now?
Some AI risks are happening now. AI is already being used in scams, to make sexual deepfakes of real people, and to spread false information. Each risk’s page sets out the evidence, with sources.
For other risks, we’re seeing the warning signs. Between May and August 2025, Anthropic, OpenAI and Google DeepMind each released their newest model with extra safeguards, because their own testing could not rule out that it would help someone make a biological weapon. More capable models will be trusted with more consequential decisions and woven deeper into the economy and critical infrastructure, so when they fail or are misused the damage will be greater and harder to reverse.
How bad do experts think this could be?
The MIT AI Risk Initiative asked 272 experts how likely each risk is to become catastrophic by 2030 (more than 1 million deaths, or more than USD 100 billion in financial loss, or comparable civilisation-scale intangible damage).
If nothing changes, they put 18 of the 23 risks at one in ten or higher. The highest rated were dangerous AI capabilities, AI weapons & cyberattacks and power centralisation.
Chance each risk becomes catastrophic, as experts rated it
The solid part of each bar is what is left after governments act; the pale part is the difference between the two estimates.
- 7.2Dangerous AI capabilities21.5% as we are, 12.3 per cent if governments act
- 4.2AI weapons & cyberattacks21.0% as we are, 11.9 per cent if governments act
- 6.1Power centralisation18.0% as we are, 11.4 per cent if governments act
- 6.2AI and unemployment17.1% as we are, 10.9 per cent if governments act
- 6.6AI's environmental harm17.1% as we are, 11.8 per cent if governments act
- 6.4AI race dynamics16.6% as we are, 7.3 per cent if governments act
- 7.1AI misalignment16.6% as we are, 8.9 per cent if governments act
- 2.2AI security vulnerabilities15.7% as we are, 8.8 per cent if governments act
- 6.5AI governance failure14.4% as we are, 7.3 per cent if governments act
- 3.1AI-generated false information12.8% as we are, 6.8 per cent if governments act
- 3.2Loss of consensus reality12.6% as we are, 6.8 per cent if governments act
- 4.1AI disinformation12.6% as we are, 7.1 per cent if governments act
- 7.6Multi-agent AI risks12.2% as we are, 6.3 per cent if governments act
- 4.3AI fraud & scams11.5% as we are, 8.7 per cent if governments act
- 5.2Loss of human agency11.5% as we are, 6.7 per cent if governments act
- 2.1Loss of privacy11.2% as we are, 7.2 per cent if governments act
- 5.1Overreliance on AI10.7% as we are, 7.0 per cent if governments act
- 7.3Capability & robustness10.7% as we are, 7.0 per cent if governments act
- 1.2AI-generated toxic content9.7% as we are, 7.5 per cent if governments act
- 6.3Devaluation of human effort9.5% as we are, 6.0 per cent if governments act
- 1.3AI performing worse for some groups9.2% as we are, 5.4 per cent if governments act
- 1.1AI discrimination9.1% as we are, 5.5 per cent if governments act
- 7.4Transparency & interpretability7.5% as we are, 4.8 per cent if governments act
Left if governments actThe difference between the two estimates
What do other Australians think?
Most Australians want AI regulated, and for their leaders to rise to the occasion.
94%
Australians expect AI safety standards at least as strict as commercial aviation
2025 Survey Assessing Risks from Artificial Intelligence (SARA), University of Queensland
83%
Australians think AI regulation is falling behind the technology
2025 Survey Assessing Risks from Artificial Intelligence (SARA), University of Queensland
77%
Australians do not trust tech companies to ensure their AI is safe
2025 Survey Assessing Risks from Artificial Intelligence (SARA), University of Queensland
74%
Australians worry the government will not regulate AI enough
2025 Survey Assessing Risks from Artificial Intelligence (SARA), University of Queensland
What is Australia doing about it?
Laws
The Government has announced it is working on Australian Standards for AI and set up an Office of AI in the Department of the Prime Minister and Cabinet. The standards would set mandatory minimum requirements for large data centres, and conditions for frontier AI training in Australia, including how that training supports national security and safety. The Government intends to legislate them in early 2027 and there is currently a Joint Select Committee on Artificial Intelligence in progress.
Beyond the AI standards, the Government has signalled it is not pursuing a whole-of-economy AI Act. It previously consulted on mandatory guardrails for AI in high-risk settings, and that proposal was dropped.
Instead, the Government is asking existing regulators to manage AI risks in their domains. This means relying on existing privacy, consumer, online safety, anti-discrimination and critical-infrastructure laws, and plugging the gaps where needed. The Government has also published voluntary guidance for industry.
Capability
The Australian AI Safety Institute was announced in November 2025 with $29.9 million over four years, an average of about $7.5 million a year. It sits within the Department of Industry, Science and Resources, tests advanced AI models with technical partners, and is not a regulator.
What is not covered
The Australian Government does not require frontier AI companies to report on the capabilities of their models, or the risks they pose. It does not require them to report when something goes seriously wrong with their models. Other jurisdictions, like the EU, California and New York, have passed laws mandating incident and accident reporting.
Last updated .
What are people asking governments to do?
Experts and organisations have recommended ways to reduce these risks. Each one below links to where it was recommended. The ones readers have starred most come first.
- PartlyFund a world-leading AI Safety Institute
Called for by 23 organisations
Give Australia's AI Safety Institute the funding, expertise and powers to contribute in areas of AI safety that allies and AI… - Not in placeMandatory reporting of serious AI incidents
Called for by 10 organisations
Serious AI incidents and near misses should be reported to government, so regulators can see what is happening and act. - Not in placeMake the most powerful AI companies disclose their models' risks
Called for by 5 organisations
Companies building the most capable AI models should have to tell government and the public what their models can do and the… - Not in placeMandatory guardrails for AI in high-risk settings
Called for by 5 organisations
Binding rules for AI used in high-risk settings, covering testing, transparency and accountability. The Government consulted on… - Not in placePass a dedicated Australian AI law
Called for by 2 organisations
A single law setting definitions, standards and rules for AI across the economy, to support the reforms under way. - Not in placeWork with other middle powers on AI
Called for by 3 organisations
Australia should work with other middle powers to reduce AI risks and limit the concentration of power in a small number of… - Not in placePublish a national assessment of AI risks
Called for by 2 organisations
Government should assess how exposed Australia is to AI risks, including emerging risks and risks that cut across sectors, and… - Not in placeRequire DNA suppliers to screen orders for dangerous sequences
Called for by 2 organisations
Synthetic DNA imported into Australia should come only from suppliers that screen orders for dangerous sequences. The Biosecurity… - Not in placeMake AI developers responsible when their AI agents cause harm
Called for by 2 organisations
When AI agents cause harm, the law should be clear about who is responsible, and developers should not be able to shift that onto… - Not in placeTrack every government review of AI in one place
Called for by Global Shield Australia
One agency should track the government reviews and reforms that touch AI, and publish a register of them. - Not in placeParliament investigates the risks of the most powerful AI models
Called for by Global Shield Australia
Parliament's AI committee should examine the companies building the most powerful AI models, which safeguards work at the level…
What can you do?
Write to your representative
Decisions about all of this are made by members of parliament, most of whom hear from very few constituents about it. Use the tools below to find your representatives and draft a letter.
Write to your representative