The Role of Generative AI in Shaping Political Discourse on WhatsApp in India

This study examines the dissemination and characteristics of content generated through generative AI on WhatsApp in rural India. WhatsApp is the biggest end to end encrypted social network with over 2 billion monthly active users, including over 600 million in India. The data was collected from over 1,000 WhatsApp groups using an innovative, opt-in approach resulting in a diverse sample of 400 users from three villages in central and northern India. Our unique dataset includes over one million pieces of content, out of which we specifically focus on over 2,000 items flagged by WhatsApp as ‘forwarded many times’ between August and October 2023, indicative of their viral nature on the platform.

Our analysis of the content revealed that, despite the rarity of groups dedicated to political discussion, political matters predominated the viral content. A manual annotation was conducted to identify generative AI-generated content across text, images, audio, and video. Our findings suggest a minimal presence of generative AI creations, with fewer than a dozen instances, primarily images or videos, and no detectable generative AI-generated texts. The lack of generative AI text may be attributed to the nascent stage of Hindi-language AI content generation or the inherent difficulty in distinguishing AI-generated text.

The identified generative AI content fell into four main categories: deceptive material (e.g., futuristic infrastructure images misrepresented as current Indian projects), politically charged memes (e.g. a meme generated using generative AI mocking the non-violence struggle Mahatma Gandhi suggested for Palestine), erotic videos (containing a video of an AI generated Indian-looking avatar saying erotic stuff), and inspirational messages. Some examples of the different types of generative AI content found in our dataset can be found below. Despite the seemingly low prevalence of generative AI content in our sample, the potential for wider prevalence and impact remains uncertain due to the non-representative nature of the dataset.

The study posits that generative AI is not a predominant tool for electoral influence in India. The Bharatiya Janata Party’s (BJP, the right wing hindu majoritarian party currently in power) established infrastructure for content dissemination on WhatsApp overshadows the need for generative AI technologies. The true potency of generative AI lies not in content creation ease but in its amplification alongside existing political resources. The vast impact of social media is driven more by strategic dissemination than by straightforward content generation abetted by generative AI, as evidenced by the BJP's existing capabilities.

Nonetheless, understanding generative AI's influence remains critical, as it could provide other parties, especially new entrants, with a competitive edge. The role of encryption is particularly noteworthy, as it can obscure the spread of AI-generated content until potentially widespread dissemination.

This research underscores the necessity for a vigilant approach to emerging technologies in the information ecosystem, emphasizing the need for global and domestic strategies to address potential misuses in the public sphere. Our findings caution against suggestions, particularly by governments, to compromise or weaken encryption as a measure against the perceived threats posed by generative AI. They contribute to the dialogue on generative AI's role in shaping political narratives and the urgent need for preparedness against its misuse.

Contact Kiran Garimella (kiran.garimella@rutgers dotedu) if you have any questions.

Notes: production, consumption, distribution, persuation. what is the role of tech in each of them? generation is the easiest. distribution and persuation are the hardest. Does GenAI help in persuation? production and consumption are enabled to come together by the platforms. Especially on whatsapp because of no moderation, there is no way to disrupt the link between them. can we think of ways in which gen ai can be helpful? e.g. in creative ways?

The fact that we dont see it here doesnt mean its not a threat. It is of course. Like the many other things. - is this new? No. it was just not easy. - many phatic use cases too. e.g. to see cute pictures of gods, as children etc. - a word on sampling. This is viral content on whatsapp. if it is viral, it will be seen here. so if it was widely spread, it will be seen here. - how about personalization? will gen ai help in personalization? could we be missing that in our data? how can we be sure? - more examples here: https://www.boomlive.in/decode/how-ai-generated-images-took-centre-stage-in-telangana-elections-24076 - how about text? we really dont know how it is with text. - What is the unique capability that AI will bring? - read https://www.brookings.edu/articles/misunderstood-mechanics-how-ai-tiktok-and-the-liars-dividend-might-affect-the-2024-elections/ - compare it with the hype against misinfo or deepfakes - The increasing exposure to such synthetic media, even the benign ones would potentially lead to erosion of public trust in any media that is consumed. - particularly when combined with a lot of other normalized information from trusted sources... - Very important to separate out the ease of creation of artificial content and the impact it would have through events like misinformation. they are orthogonal. just the fact that they could intersect sometimes should not deter us from pursuing this direction. - for misinformation, think of this as a demand and supply problem. there is demand, and the supply can be scaled using this technique (gen ai) but also with other techniques. - social media removed limits for distribution, gen ai removed limits for creation - many of these are forwarded many times (e.g. example) - Talk about how deepfakes and AI content were used during the gaza war. e.g. read this article. -- 4 AI generated images and 2 deepfakes.