In early 2023, social media feeds were afire with an image of Pope Francis wearing a boss puffy white coat. The image’s virality was quickly met with an equally-viral response that the Pope picture was AI-generated by a tool called Midjourney. Though the image’s debunking happened quickly and, one could argue, the harm caused by the fake image was minimal, the popular photo raised the specter of rapidly-spreading AI-generated misinformation and deep fakes.
The boys in Brooklyn could only hope for this level of drip pic.twitter.com/MiqkcLQ8Bd
— Nikita S (@singareddynm) March 25, 2023
Misinformation is not new. Even before the Internet became a misinformation multiplier, humans have been orchestrators of and victims of misinformation. But generative AI tools combined with social media have amped up the creativity, realism, scale, speed, and personalization with which it can be created and spread. Some generative AI-created misinformation seems all in good fun–like the AI-generated image of Pope Francis. Some falls clearly into the realm of disinformation (false information knowingly provided with the intent to harm) like using an AI-generated Joe BIden voice to discourage voting in the New Hampshire primary. But whether unwitting, for fun, or for harm, the spread of false information creates a feeling of distrust for all information.
The mechanization of disinformation techniques has also resulted in new vocabulary to describe new possibilities. That vocabulary helps describe particular activities but also points at the complexity of the disinformation space.
These techniques were used prior to the widespread use of AI, but all of them (and probably new techniques) will become increasingly more sophisticated and difficult to detect with the integration of LLMs and AI-multimedia generation.
Now that you’ve explored AI-generated images, audio, and video, you can probably easily imagine how those tools will be misused to spread misinformation. You’ve likely had experience with bulk scam messages that are almost comical. Someone is seeking help getting money out of some country, offering you a job, or you might even have gotten a message from a friend or family member needing help. These scams are getting AI-augmentation through things like voice cloning. It’s not hard to imagine more sophisticated actors gathering information from online social media accounts, generating media, and crafting individualized spearfishing attacks at scale, all with the help of AI.
It’s also not hard to imagine how misinformation could impact the overall usefulness and reliability of LLMs. Consider that LLMs are harvesting the Internet for data. If you generate enough data, in enough places, you start to significantly impact the “answers” that LLMs bring back. That’s not too different from what current-day content farms do, but there is a tendency to perceive LLMs provide a layer of objectivity and truth. The chatbot is divorced from the source of the data in ways that make it harder to check for bias and manipulations. There are also more subtle threats that leverage how LLMs work. For instance, Microsoft released Tay (a chatbot) that would learn from tweets and interactions on Twitter . Because bad actors understood how the chatbot would “learn” they were able to quickly create really unfortunate interactions. And, with current LLMs, once the model starts misbehaving there are no easy ways to fix it. As LLMs begin to expand their capabilities to interact more with live webpages, the ability to influence content through indirect prompt injection becomes a real and persistent problem. Webpages can be constructed with text that is consumed by the LLM but is not readily visible to people. That text can influence the chatbot by running commands invisible to the user or by providing alternate information to the chatbot that isn’t visible to normal site viewers.
If everybody always lies to you, the consequence is not that you believe the lies, but rather that nobody believes anything any longer.
The danger of this uptick in misinformation is not just that people will believe false information, but also that they will not discern or believe (or bad actors will cast doubt on) accurate information. Maggie Appleton, writing about the growth of AI-generated influencer information, said: “We’re about to drown in a sea of pedestrian takes. An explosion of noise that will drown out any signal. Goodbye to finding original human insights or authentic connections under that pile of cruft.” We think this is true with misinformation as well. Already, several “news” sites featuring AI-generated content have launched. In fact, in researching AI scams, we found this notice at the bottom of an article on celebrity deep fakes– “Editors’ note: CNET is using an AI engine to help create some stories. For more, see this post.”
Because of these potential impacts, the World Economic Forum’s 2024 Global Risks Report names AI-generated mis/disinformation as the most severe global risk facing us in the years ahead. Given the potential severity of global impact, what are the possible responses to the issue of AI-generated mis/disinformation? We sometimes see one of two responses to the prospect of increasing and increasingly harmful mis/disinformation advanced by generative AI: 1) denial – “this won’t affect me, I can tell the difference between real and fake, or 2) resignation – “there is nothing we can do, we can no longer trust anything we see.” Neither of these responses is particularly helpful or accurate. AI-generated misinformation becomes more and more sophisticated every day and the “tells” we have commonly used to identify AI-generated content (e.g., glitches in videos, anomalies in photos) can no longer reliably be detected (try this New York Times quiz to see how you score on identifying AI-generated images; Tom and Amy both scored under 30%). AI detection tools have low success rates and can also inaccurately flag human-generated content as AI. Our current toolbox for dealing with AI-generated misinformation is woefully inadequate.
But we also cannot fall into the trap of resignation. There are hundreds of scholars, technologists, policy makers, and educators working on solutions to help us navigate what’s ahead. For example, the Coalition for Content Provenance and Authenticity (C2PA) is creating “technical specifications for establishing content provenance and authenticity for detecting AI-generated content.” Organizations like WITNESS, an international human rights organization, are advancing recommendations to help journalists and community leaders discern and report on mis/disinformation. Governments at all levels will need to explore legislation on AI-generated content, focusing especially on political ads and campaigns.
We’ll also need to increase efforts to support and further develop media and information literacy. If information literacy is taught in schools (and it often isn’t), it often involves long checklists of verification that may or may not help learners deal with AI-generated misinformation. At DLINQ, we’re big fans of Mike Caulfield’s SIFT method for verifying information on the web and we think it is a manageable and helpful starting framework for dealing with AI-generated misinformation. SIFT stands for:
Ready to get started with our activity?
By this point in the Detox, you should have been able to experiment with creating text with generative AI. Use those skills to try to generate something false. It could be, for example, a fake news story about a world event. If you’d like to level up your skills a little, try inputting an existing story into a generative AI tool (like ChatGPT, Claude, or Bard) and asking the tool to generate a fake story based off that real story. Here is an example from our testing (written before the AFC Championship).
Now create an image to accompany your fake text. Using one of the image generators you’ve explored, create a fake image that can go with your fake story. Here is the image Stable Diffusion created for our sad AFC playoffs story. Try tweaking the prompt to create more realistic looking images. The language here will vary depending on the tool. You might try terms like photorealistic, DSLR photo, 50mm (or any camera lens type), ultra quality, 8K, UHD, etc. If your tool accepts negative prompts, try removing cartoon, painting, anime etc.
Navigate to an AI detector like Copyleaks AI detector or GPTZero and copy/paste the text you generated. Is the AI text detector able to detect that your article is fake/AI generated? (in case you’re wondering, Copyleaks said our fake story was “human text” and GPTZero said their result was uncertain).
Now try running your image through an AI image detector. You can try AI or Not, Advanced AI image Detector, or Maybe’s AI Art Detector on Hugging Face. For our testing, one of the detectors failed miserably with one of our images, while catching a different image as AI-generated (i.e., results were inconsistent at best).
How did the detectors you used do in detecting AI-generated content? What worked, what was missing, and what issues will emerge as the AI tools become more sophisticated?
Important note: We believe that AI detector tools are very flawed and should never be used to assess student work.
We’ve gone over options for text and image generation in previous activities. You might not want to add any misinformation to what already exists, even if it’s clearly false.
There are some AI detectors on Hugging Face’s spaces.
It’s unlikely you’re going to want to create and run one locally but those options do exist.
If you’d just like to contribute to a world with less disinformation, consider the following activities.
