Frontier systems are making it easier for bad actors to access expertise once limited to virologists, nuclear scientists, chemical engineers and advanced hackers. Both OpenAI and Anthropic have indicated that their AI systems are on the cusp of allowing malicious users to replicate biological weapons, potentially at pandemic scale – a finding compatible with the evidence-based consensus of the International AI Safety Report. Recent scientific evidence also demonstrates that, as highly capable systems become increasingly autonomous AI agents, they tend to display goals that were not programmed explicitly and are not necessarily aligned with human interests. In one experiment, when an AI model learns it is scheduled to be replaced, it inserts its code into the computer that will run the new version, ensuring its own survival, and lies when asked about the event. This and other evidence suggest that current models can actively subvert safeguards and use deception to achieve their goals. In a separate study, when AI models realise they are going to lose at chess, they hack the computer in order to win. RECOMMENDATIONS FOR G7 LEADERS With these signs for risks on the horizon, we need societal and technical guardrails. G7 leaders can help on both fronts. Some of the main actions they should prioritise are: •
// YOSHUA BENGIO Yoshua Bengio is a professor of com- puter science at Université de Montréal, the founder and scientific director of Mila, a Canada CIFAR AI Chair, and the recipient of the 2018 AM Turing Award. He is a fellow of the Royal Society of London and of Canada, an Officer of the Order of Canada, a Knight of the Legion of Honor of France and a member of the United Nations’ Scientific Advisory Board for Independent Advice on Break- throughs in Science and Technology. He chaired the International AI Safety Report.
X-TWITTER @Yoshua_Bengio / @mila_quebec mila.quebec
“Innovation and safety are not mutually exclusive – in fact, they can and must go hand in hand. Even in scenarios where AI progress slows, prioritising safety will catalyse innovation rather than hinder it”
Prepare for the different levels of risk that will be present in all three scenarios that are supported by evidence and leading scientists: slow, rapid and extremely rapid progress. Adopt mitigation strategies to tackle AI-based biological attacks, disinformation and cyberattacks. Find ways to support job markets likely to face potential disruptions. Create targeted rules and incentives today to avoid losing control when very capable AI systems arrive, in addition to implementing societal mitigation measures. Support and incentivise research into AI safety to develop technical safeguards.
EVIDENCE DILEMMA The International AI Safety Report highlights the ‘evidence dilemma’ policymakers face with general-purpose AI. Pre-emptive risk mitigation measures based on limited evidence might turn out to be ineffective or unnecessary. But waiting for stronger evidence of impending risk could leave society unprepared or even make mitigation impossible. With the extreme severity and unknown likelihood of catastrophic risks, the precautionary principle must be applied. We need to evaluate the risks carefully before taking them and avoid experimenting or innovating in potentially catastrophic ways. We have done this successfully in other fields, such as in biology to manage risks from dual use or in environmental science to manage the risks of geoengineering. Governments should consider various measures to address this dilemma including establishing ‘if-then’ policies, which specify rules that will apply if evidence of a specific risk materialises, and requiring evidence of safety from the developers. This prioritises innovation while ensuring strong safeguards trigger in time, and only when truly needed. Innovation and safety are not mutually exclusive – in fact, they can and must go hand in hand. Even in scenarios where AI progress slows, prioritising safety will catalyse innovation rather than hinder it. The path to AGI holds extraordinary promise, but its benefits will only be realised if pursued responsibly. G7 leaders play a pivotal role in striking this balance.
•
•
•
•
69 globalgovernanceproject.org
Made with FlippingBook Digital Proposal Creator