What should we be afraid of with AI?

The question "is AI dangerous?" mixes 2 kinds of danger: harms that we observe and count, and scenarios that we cannot yet observe, the most serious of which is the loss of control of systems more capable than we are. The French, for their part, put a harm that is already here first: when surveyed by Ipsos for CESI in January 2025, 49% cite false information among the main risks of generative AI, 41% the replacement of jobs, and only 19% environmental impact.

The first 5 rows describe harms that are already happening, with different degrees of proof. The electricity of data centres is counted by the International Energy Agency; our analysis on aviation scales it. The fall in hiring of young people is measured without its cause being established, as detailed in our analysis on entry-level workers. The effects on learning rest on a few trials (our analysis), and the retention of conversations on the publishers' terms (our analysis). The last row is of a different nature: a forecast.

The broadest survey on the subject questioned, in 2023, the 2,778 researchers who had published in the main AI conferences. They were asked to assume that machines will one day do all tasks better and more cheaply than humans. In that case, their median probability of an "extremely bad" long-term outcome, such as human extinction, is 5 %, with a mean of 9%; 38% give at least 10%. Asked directly about extinction or a lasting loss of power for humanity, the question gives a median of 5% or 10% depending on the wording. In the same survey, 64% of respondents also give a non-zero probability to an extremely good outcome. The authors note that no proof understood by everyone makes such an outcome certain or implausible.

The international report on AI safety, chaired by Yoshua Bengio and published in February 2026 with more than 100 experts, describes the same uncertainty: opinions on loss of control vary widely, some judge these scenarios implausible, others likely enough to deserve attention given their seriousness. It adds that current systems do not have the capabilities to pose this risk, but are progressing in areas that lead to it, such as autonomous operation. In May 2023, a statement signed by researchers and industry leaders called for reducing the risk of extinction from AI to be a global priority, alongside pandemics and nuclear war.

The same survey asked which scenarios over the next 30 years deserve "substantial" or "extreme" concern. At the top come the spread of false information (86%) and the manipulation of opinion (79%), then, at 73% each, the use of AI by dangerous groups to make weapons such as modified viruses and its use by authoritarian regimes to control their population, and at 71% the worsening of economic inequality. The first 2 are harms that we already observe.

The best objection to our position is simple. A risk of a few per cent, if irreversible, weighs more than a certain but repairable harm: one does not recover from an extinction, whereas one can train beginners differently or decarbonise electricity. This is the reasoning that leads to funding defence against asteroids or pandemic preparedness, whose annual probability is low. It does not require believing in the worst, only not ruling it out.

We take it seriously, and we do not treat it as a priority for 3 reasons. The 5% is the median of individual opinions, which no observation can verify today; it changes with the wording of the question, from 5% to 10% in the median, and from one survey to another: the share of respondents giving at least 10% went from 48% in 2022 to 38% in 2023, on different samples. Present harms have levers that are being decided now, and that commit us for a long time: a connected data centre runs for years on the electricity allocated to it, and if hiring of beginners stays low, whatever the cause, years of learning the trade are missing. Finally, some of the tools serve against the 2: evaluating models before release, obliging publishers to disclose what they know. The European AI regulation has already prohibited, since 2 February 2025, harmful manipulation, social scoring and emotion recognition in the workplace and in education.