ISKP: Use of Generative AI presenters to create newscasts

A group calling themselves “Khurasan TV” began circulating videos using artificial intelligence (AI) video generation to create newscasts of attacks claimed by Islamic State Khorasan Province (ISKP). The first of these videos appeared between 18 and 20 May 2024, and broadcasted the claim for the 17 May 2024 attack in Bamyan province, which killed three Spanish tourists. Since this time, at least eight videos have been created and shared by the group.

The video shared on instant messaging platform Teleguard (a Swiss made alternative to Telegram, which claims stronger privacy and encryption) uses an AI presenter in an environment mimicking a newsroom. A voice synthesis is used to narrate the claimed attack in Pashto.

Khurasan TV appears to be acting independently and has yet to publicly collaborate with other ISKP media outlets, such as the Al Azaim Media Foundation. Their content has not been shared extensively within propaganda channels, and appears to be isolated to either Teleguard or through reposts on X (formerly Twitter). 

The use of Teleguard is also notable for IS supporters; since 21 April 2024, AW has observed increased calls to switch to Teleguard from the group’s members on Rocketchat (a decentralised, open source instant messaging application). However, possibly due to the Teleguard’s limited features and nicheness, it has yet to be used extensively by groups sharing IS content. AW investigators have been unable to access Khurasan TV’s Teleguard channel as it is invite only, requiring a channel ID to access. AW sources confirmed that the AI videos originated on Teleguard.

Khurasan TV is not the only pro-IS group creating AI-generated newscasts. Another content producer, known as Hisad, has posted similar videos on Rocketchat since 26 March 2024. However, these videos were met with scrutiny by members of the group who believe AI generation to be haram. AW note that the hadith collection of Sahih Bukhari (the first hadith collection of the Six Books of Sunni Islam and one of the most valued books in Sunni Islam after the Quran) explicitly prohibits the making of images of living beings; AI-generated content could be considered doing exactly this.

AW research suggests that the pro-ISKP groups have used Virbo software, from a company called Wondershare, identifiable from the avatars. Virbo, which is free to download, generates a presenter that is animated using machine-learning. The user enters in a script for the avatar to narrate using voice synthesis. Although not all avatars appear to be present in the demo version of Virbo, AW have been able to match those used by Khurasan TV to ones offered on Virbo (shown below). Presenters used by Khurasan TV that aren’t present in the online demo can be found elsewhere online, with a Virbo watermark present.

Wondershare’s terms of service do not explicitly prohibit their software from being used to generate extremist content.

Accessibility of generating AI content

Advancements in AI technologies, such as Stable Diffusion, text generation, video generation and voice modulation have allowed violent extremist organisations to generate content, both for propaganda and recruitment. 

These advancements have allowed those with a proficiency in computing and semi-powerful computer systems to create AI-generated productions with little need for further advanced technical skills, where previously extensive knowledge and comprehensive computing hardware was required. Ongoing power increases of computer graphics cards and processing units have allowed everyday computer users to start working with entry-level AI technologies locally on their systems. 

Generation of imagery has become increasingly accessible through programmes. Although there has been an increase in online services offering AI video generation, they remain expensive and in many cases have strong censorship against use of services to generate pornography or illegal content. 

Voice generation has also become prevalent over the course of the last year, partly due to the popularity of a service called ElevenLabs.io, which allows users to generate any text to speech using a cloned voice of their choosing. To date, AW note ISKP-supporting platforms have only verifiably utilised the Virbo software. 

Reception of generative AI and voice synthesis among IS supporters

The use of AI in generating pro-IS content has been the subject of debate amongst the group’s supporters. The release of Hisad’s AI-generated videos to a pro-ISKP Rocketchat channel prompted users to comment on whether the use of Generative AI could be considered haram: firstly, due to the replication and animation of a human being; and secondly, due to the training data which would contain voices of the “murtadeen” used by the voice synthesis.

Hisad tried to appease users by blurring the faces of AI presenters, however, other users said that this was not enough. They claimed that any animation showing a human was haram. Users also referenced a previous incident regarding another media group, Al-Ansar, which had used human animation in a series released in March 2024, titled, “The daily life of Muawiyah and Salem.” After three episodes, the series was removed from IS archives. AW cannot confirm the reason for the removal.

Supporters also claimed AI voice generation to be haram, due to the voices used to train voice synthesis programs, including “voices of disbelievers,” as it is forbidden to use the voices of non-Muslims to recite the Quran or Islamic State news.

Following the discussion, Hisad was encouraged to use their own voice and animation of the Islamic State flag in the production of newscasts; however Hisad continued to post AI-generated newscasts. 

Conclusion 

The use of AI-animated presenters and voice synthesis appears to be controversial among IS supporters, and has yet to be embraced unilaterally. This supports AW’s hypothesis that Khurasan TV is acting independently of other media groups, and it is possible that its members are younger than other IS supporters. 

As of the time of writing, Khurasan TV’s latest video appears to have been released on 6 June 2024. Since then, the Teleguard channel may have been removed due to moderation, as the channel ID provided by AW sources is no longer functional. Despite increasing accessibility of tools to generate AI content for use as propaganda, further challenges such as content moderation and concerns regarding morality may limit the use of AI to generate pro-IS and pro-ISKP content. AW assesses that it is likely to be a transitory trend popular amongst younger supporters, but may not be utilised more broadly by established media groups.