AI as an Exam Follow-Up Tool in General Chemistry

With the evolution of artificial intelligence (AI), I have turned to the chemical education literature for inspiration on how to embrace the dawn of AI in the general chemistry classroom.

I designed an AI-based exam review activity, that was inspired by the work in the literature. Through this activity, my students are prompted to reflect on their missed exam questions using a posted exam key and ChatGPT as their guide. Students are required to locate missed learning objectives in their notes, and then validate the answers provided by ChatGPT by using their notes and the textbook. Students were then instructed to write 1-2 complete sentences explaining their mistake on the exam and/or the mistake made by ChatGPT:

Follow the following steps for at least 6 questions where you lost points on Exam 1. A template and two examples are provided on the following sheet. If you lost points on fewer than 6 questions, then repeat this exercise for every question that you missed, even if the total is less than 6.

1. Locate in your notes or textbook where the question was covered.
   1. All questions were covered in class. If you cannot find the topic in your personal notes, you may consult a friend. If you still cannot find where the topic was covered, then use the textbook.
   2. “This wasn’t covered” = 0 points
   3. You may also reference in-class problem sets or ALEKS topics in place of class notes.
2. Ask your AI Source the exam question. Read the generative AI response. If the question requires a figure or graph, then you can copy and paste a screenshot of the graphic into the AI prompt.
3. Use your notes/problem sets/textbook to validate the response given by AI. State whether or not your AI source was correct.
   1. If your AI source was correct, proceed through steps 4-8.
   2. If you AI source was incorrect, copy or include a screenshot of AI’s incorrect response and write 1-2 complete sentences that explain why the AI-generated answer was incorrect. Did you make the same mistake as your AI source? If AI is incorrect, stop at this step.
4. Identify where you made your mistake. Write 1-2 complete sentences (not bullet points) explaining your mistake.
5. Tell your AI source to “Give me a similar question to try without the answer”. You must specify “without the answer” or the AI source will automatically generate the answer.
6. Use the space on your worksheet to test your understanding and attempt the AI generated question.
7. Tell you AI source to “Give me the correct answer”.
8. Repeat Steps 5-7 until you get the problem correct.

Reviews of the assignment from students have been mixed, ranging from “pointless” to “extremely helpful”. I feel that this assignment is beneficial in two ways. First, it forces students to reflect on their notes and realize that everything on the exam was, in fact, covered in class and in their notes. Second, it introduces students to productive uses of generative AI sources for academic use and forces them to reflect on the accuracy of said sources for the context of the course.

A full copy of the student handout (including the examples provided) can be seen below.