Study Warns Highly Realistic AI-Generated Deepfakes Could Enhance ‘Electoral Interference’

This illustration photo shows a phone screen displaying a statement from the head of security policy at Meta with a fake video (R) of Ukrainian President Volodymyr Zelenskyy calling on his soldiers to lay down their weapons shown in the background, in Washington, on Jan. 30, 2023. (Olivier Douliery/AFP via Getty Images)

Highly realistic AI-generated deepfakes containing false allegations about political candidates could enhance “electoral interference,” according to a study.

Research by The Alan Turing Institute’s Centre for Emerging Technology and Security (CETaS) on Wednesday urged Ofcom and the Electoral Commission to address the use of AI in upcoming elections.

A number of politicians, including Prime Minister Rishi Sunak, Labour leader Sir Keir Starmer, and the Mayor of London Sadiq Khan have been the subjects of misleading AI content.

AI-Generated Voice Clones

CETaS, which conducts interdisciplinary research on emerging technology and national security policy, said that with the general election on July 4, there “is a very limited time to make significant changes to election security protections or electoral law.”

“Nevertheless, much can still be done to enhance short-term resilience against AI-based election threats,” it added.

In some cases, AI-generated voice clones can create false images or videos of political candidates making controversial statements or depicting candidates in contentious activities.

Other times, they can pretend to withdraw from the election race or endorse other candidates, or purportedly demonstrate ballot rigging.

It said that some of these threats may originate from hostile actors, while others from political parties themselves.

In more extreme scenarios, it warned that “customised AI malware” could result in voting systems being manipulated or misreporting votes. It said, though, that the UK will “be more insulated against these types of threats compared to countries such as the US, owing to the continued use of paper-based voting and human ballot counting.”

However, it said that upon close examination of AI-generated deepfakes, there has been no clear impact on election results.

In elections already held in Argentina, Indonesia, Poland, Slovakia, and Taiwan, political candidates expected to win from polling data still did so, even after the circulation of deepfakes that went viral.

Act Quickly

Despite this, CETaS called for more regulation and warned there is “no clear guidance.”

It said that the Electoral Commission should work with Ofcom and the Independent Press Standards Organisation to publish new guidance for media reporting on content which is either alleged or confirmed to be AI-generated.

It also wants to request voluntary agreements for political parties setting out how they should use AI for campaigning, and require AI-generated election material to be clearly marked as such.

Sam Stockwell, research associate at The Alan Turing Institute and the study’s lead author, said: “With a general election just weeks away, political parties are already in the midst of a busy campaigning period.

“Right now, there is no clear guidance or expectations for preventing AI being used to create false or misleading electoral information.

“That’s why it’s so important for regulators to act quickly before it’s too late.”

Alexander Babuta, director of CETaS, said: “While we shouldn’t overplay the idea that our elections are no longer secure, particularly as worldwide evidence demonstrates no clear evidence of a result being changed by AI, we nevertheless must use this moment to act and make our elections resilient to the threats we face.

“Regulators can do more to help the public distinguish fact from fiction and ensure voters don’t lose faith in the democratic process.”

Ransomware

The National Cyber Security Centre (NCSC) has said that the “shape-shifting rise of ransomware and extortion attacks” is a threat owing to its recent international rise.

It warned that Russia- and Iran-based actors had been conducting spear-phishing campaigns, or targeted attempts to steal sensitive information via email, against politicians, journalists, activists, and other groups.

Furthermore, in March, the NCSC assessed with “high certainty” that a Chinese state-affiliated group conducted online reconnaissance activities, such as collecting information, against the email accounts of UK parliamentarians that had been critical of China’s activities. The government reported that no parliamentary accounts were successfully compromised.

On May 23 a Parliament briefing paper exploring cyber security risks to elections said that there has been “little reported evidence of ransomware attacks directly affecting UK elections or democratic processes.”

PA Media contributed to this report.