Using Social Annotation to Fact-Check Generative AI

As has now been widely discussed, generative AI is a productivity tool that can be used in various ways in our classrooms with a wide range of success and failure. Generative AI programs can be quite effective at various repetitive tasks or work such as summarizing, but can also “hallucinate”—that is, provide incorrect or misleading responses. Research shows that U.S. students are more suspicious of using generative AI than students in other nations and in some ways are falling behind.

Social Annotation, as we have previously discussed, is a pedagogical approach that research has shown can be a highly effective way to improve student learning outcomes, add transparency, and boost student engagement. It does this by asking students to collaboratively annotate digital texts by adding comments, questions, and tags. In our classrooms, this technique of engaging students through collaboratively annotating texts can be utilized to teach students about both the potential upsides and downsides of using generative AI. By annotating example generative AI outputs, such as texts produced by chatbots such as Copilot or ChatGPT, students can evaluate the benefits and risks of this technology in a hands-on, transparent and experiential way.

Hypothes.is (available as an etext to IU faculty and students) has done research and is providing support materials on how to use its social annotation program to help students learn to use generative AI applications such as ChatGPT by critiquing it as content experts, fact checkers, and writing editors. To use Hypothes.is, however, you have to have ordered it as an etext by the ordering deadline (for the upcoming 2024 Fall semester, March 24).

If you do not have access to Hypothes.is you can still implement teaching your students to use generative AI through using Social Annotation. For example, you can start by using MS Word or Google Docs. Simply use a prompt to generate text from a generative AI application such as Copilot, ChatGPT, or Gemini, and paste the result into a Word document. Then share that document with your students using OneDrive.

Your students can then add annotations to your Word file using the Review tab in Word to answer questions from the Hypothes.is suggestions above, such as:

• Identify examples of generative AI “hallucinating”; that is, giving incorrect or misleading information.
• Assess how generative AI introduces an argument. Does the introduction draw the reader in or is it too specific or too broad?
• Consider how a generative AI structures its response. What are its strengths and how could it be improved? Did it answer all parts of the prompt completely and if not, what was missed or left out?
• Analyze the voice and tone of a generative AI’s response. Does its tone and style make you want to keep reading? Is its voice critical, supportive, detached, gentle or something else? How so?

To find out more about how to try out a low-stakes social annotation assignment, please join me during my online workshop March 5th, 2024 at 3pm, Using Social Annotation to Fact-Check Generative AI. And, as always, if you have any immediate questions or concerns, feel free to contact us or request a consultation.