02
Verifying STEM AI Outputs: What You Really Need to Know
Your students are using AI — but do they know when it's wrong?
C
R
A
F
T
Learning Objectives
- Identify common LLM failure modes in code and content
- Apply the Check the Machine (CtM) 4-step protocol
- Conduct structured error hunts
- Design a CtM classroom activity for their subject/grade
- Distinguish productive AI use (learning) from unproductive (copying)
Key Activities
- Code error hunt (breakout: find bugs in AI-generated code snippets)
- Content error hunt (audit AI-generated science/math explanations)
- Check the Machine protocol deep dive (Task → Before → After → Takeaway)
- CtM template customization for participant's classroom
- AI policy committee exercise (draft verification-based classroom norms)
Talk:Do Ratio
26 min facilitator-led / 164 min participant activities (1:6.3)
Hands-on (86%)
Facilitator-led (14%)
Participant Takeaways
- CtM template (editable)
- Content Audit Checklist
- AI Error Gallery (annotated examples)
- Prompt library for verify-able outputs
Overview
Equip yourself with the “Check the Machine” (CtM) protocol and hands-on error-hunting skills for both AI-generated code and STEM content. This workshop focuses on academic integrity, error analysis, and critical thinking in STEM — addressing the real challenge of students using AI without verifying its outputs.
Reframe
“The problem isn’t that students USE AI — it’s that they don’t VERIFY AI” — This session reframes academic integrity concerns away from policing and toward teaching verification as a core skill.
The Check the Machine (CtM) Protocol
A reusable 4-step classroom verification framework from the CRAFT pedagogy (Fortify phase):
- Task — What you asked the AI to do
- Before — Your expectation or prior belief about the answer
- After — What the AI actually produced
- Takeaway — What the comparison reveals about the tool AND about your own understanding
Session Structure
| Block | Time | Type | Duration |
|---|---|---|---|
| Welcome + Icebreaker + Pre-Survey | 8:30 | Admin/Do | 10 min |
| CRAFT Orientation + Contextualize | 8:40 | Listen + Do | 15 min |
| Reframe (poll + breakout) | 8:55 | Listen + Do | 20 min |
| Break #1 | 9:15 | Break | 10 min |
| Assemble (I Do → We Do → You Do) | 9:25 | Listen(brief) + Do(extended) | 80 min |
| Break #2 | ~10:40 | Break | 10 min |
| Fortify (verification activity) | ~10:50 | Listen(brief) + Do(extended) | 25 min |
| Transfer (CRAFT debrief + lesson design) | ~11:15 | Listen(brief) + Do | 25 min |
| Resources + Post-Survey + Close | ~11:40 | Admin/Do | 20 min |
Your students don't need to be AI experts. They need to be AI skeptics. That's a skill you just learned to teach.