Answer integrity lab # AI quiz answer-key verification checklist An answer-key evidence matrix and reproducible response-matching test for finding unsupported keys, defensible distractors, rejected correct variants, unsafe wildcards, and stale source bindings. Published11 October 2026Reading time11 minEvidenceAnswer-key evidence matrix and response-matching acceptance testAuthorThe AI Quiz Lab Verify an AI-generated answer key by tracing every keyed answer and explanation to a named source unit, trying to prove each distractor from the same evidence, and testing every accepted open-response variant. Block publication when two closed answers are defensible, the key depends on an unstated assumption, or a materially correct response fails the matching rule. ## Key findings, verified 11 October 2026 - A displayed choice is not enough to prove its stored score: Moodle currently allows answer-level positive, partial, zero, and negative weights. - Moodle short answers can be case-sensitive and must match an accepted answer; wildcards can admit unintended phrases. - A fluent explanation can rationalize a wrong key, so key evidence and feedback need separate checks. - Closed questions need one clearly best answer under the assumptions stated in the stem. - Key verification does not establish source accuracy, fairness, accessibility, reliability, or validity. ## What is an answer-key audit? An answer-key audit is a reproducible comparison between the question, its permitted source evidence, the keyed response, distractors or accepted variants, scoring data, and feedback. It asks two independent questions: does the evidence support the key, and does the scoring engine recognize exactly the responses the author intends? ## Answer-key evidence matrix Item layerRequired recordPass conditionPublication blockerStemClaim, scope, qualifiers, and assumptionsThe task is answerable from permitted evidenceAn unstated assumption controls the answerKeyStable answer ID, text, and evidence unitDirect support exists for the complete answerNo support or the link points to different evidenceDistractorsReason each option is wrongNone is defensible under the stemTwo options can be correctExplanationEvidence-based rationaleExplains why the key wins and alternatives failMerely restates or rationalizes the keyWeightFull, partial, zero, or negative valueStored value matches the authored modelVisible key and stored weight disagreeAccepted variantsStrings, case, punctuation, locale, units, and toleranceAll materially correct variants passA correct spelling, unit, or format is rejectedVersionSource snapshot and item revisionThe decision can be reproducedSource or item changed without revalidation ## Reproducible answer-key test - Freeze versions. Record the item revision, source snapshot, and permitted evidence boundary. - Normalize the claim. Rewrite the stem with its scope, time, unit, locale, and assumptions made explicit. - Locate minimum evidence. Identify the smallest source unit that supports every part of the intended answer. - Challenge each option. Independently attempt to prove every distractor from the same source. - Classify the key. Mark it unique, ambiguous, unsupported, outdated, or dependent on outside knowledge. - Inspect stored rules. Check stable answer IDs, weights, shuffle bindings, and feedback—not display position alone. - Build response classes. For open answers, define accepted and rejected boundaries for case, whitespace, punctuation, spelling, synonyms, units, locale, and tolerance. - Run the oracle. Submit every class and compare stored response, score, and feedback with the expected result. - Retest transformations. Repeat after editing, shuffling, import/export, and source updates. - Require review. Use a second subject-matter reviewer for consequential decisions. ## Worked example: the technically correct distractor A report says a trial lasted “approximately eight weeks,” while its exact dates span 55 days. If the keyed option is eight weeks and a distractor is 55 days, both are defensible unless the stem asks for the report's stated approximation or defines the calculation rule. Repair the stem, remove the competing answer, or disclose the rounding convention; do not silently keep both. ## Short-answer matching example The intended response is “World Health Organization.” An exact matcher can reject “WHO,” while an overbroad wildcard can accept an unrelated phrase. Build explicit accepted variants plus negative controls, then test their order because Moodle stops at the first matching answer. For numeric responses, store the unit and tolerance separately from presentation formatting. ## Source-bounded key-verification prompt Audit the item below using only the supplied evidence units. Return the stem assumptions, intended key, direct evidence, every plausible competing answer, why each distractor is wrong, stored scoring rule, accepted variants, rejected boundary cases, and a verdict of unique, ambiguous, unsupported, outdated, or insufficient evidence. Do not repair the item silently; state the exact edit required. ## Publication blockers - The source does not directly support the key. - Two or more closed options are defensible. - The answer changes with an unstated date, unit, locale, or interpretation. - Stored answer IDs or weights do not match what the interface shows. - Feedback rationalizes the key without evidence. - A materially correct open response is rejected. - An incorrect response passes because a wildcard or tolerance is too broad. - The item or source changed after the last verification. ## What this checklist does not prove This checklist verifies key correctness and response matching for the reviewed items. It does not prove that the source is accurate or unbiased, nor does it establish content coverage, reliability, fairness, accessibility, security, or assessment validity. Regulated, medical, legal, safety-critical, and high-stakes uses require qualified human and governance review. ## Sources, verification date, and corrections This guide was verified on 11 October 2026 against Moodle's current [multiple-choice documentation](https://docs.moodle.org/en/Multiple_Choice_question_type), which documents answer weights and single- versus multiple-answer behavior; Moodle's [short-answer documentation](https://docs.moodle.org/en/Short-Answer_question_type), which documents exact matching, case sensitivity, answer order, and wildcards; and Cornell's [test-question guidance](https://teaching.cornell.edu/teaching-resources/assessment-evaluation/asking-good-test-questions), which recommends alignment and colleague review. The evidence matrix, ten-step test, examples, prompt, and blockers are original AI Quiz Lab evidence assets. Report a changed rule or reproducibility problem through the [corrections process](https://bestaiquizgenerator.com/corrections). ## Next checks Use the [distractor audit](https://bestaiquizgenerator.com/blog/ai-quiz-distractors) for option quality, the [feedback rubric](https://bestaiquizgenerator.com/blog/ai-quiz-feedback-rubric) for explanations, and the [learning-objective matrix](https://bestaiquizgenerator.com/blog/ai-quiz-learning-objective-matrix) for coverage. Use the [partial-credit test](https://bestaiquizgenerator.com/blog/ai-quiz-partial-credit-test) when an item awards more than full-or-zero credit. --- Canonical: https://bestaiquizgenerator.com/blog/ai-quiz-answer-key-checklist