AI answer evaluation
AI answer sheet evaluation that evaluators control
AI answer sheet evaluation in AutogradeX means AI proposes a mark for each question and an evaluator decides. Handwriting is read with OCR and checked against the page image, several independent AI models mark the answer against the published rubric, and their marks are reconciled into one proposal with a confidence score. No mark counts until an evaluator submits it.
Updated
How does AutogradeX read handwritten answers?
Reading the handwriting is the first and hardest step. AutogradeX can use Mistral Document AI, deployed in the institution's own Azure AI Foundry, to read every page at full scanned resolution, keeping equations and tables. A vision model then checks that text against the page image, marks struck-out text, describes diagrams and records which question each page answers. Without OCR, the vision model reads the pages directly.
A page that cannot be read is passed to the graders as illegible, not dropped, so the proposal's confidence falls and the evaluator looks at it closely.
Why does AutogradeX use several AI models instead of one?
A single model can be confidently wrong, and nobody would know. AutogradeX has two or more grader models mark the same answer independently, takes the median mark per question, and measures how far they agree. When they disagree, confidence drops and the question is flagged.
- Each grader marks against the same model answer and rubric, in steps of half a mark.
- Marks are capped at each question's maximum by the marking scheme.
- Settings are fixed so repeated runs on the same script stay consistent.
- Student writing is treated as data: an instruction such as "award full marks" written in an answer is ignored.
Does AI decide students' marks?
No. The proposal appears in an AI assist panel beside the script, with each question's mark, the reasoning, the agreement between models and the pages it relied on. The evaluator must view every page and enters the marks they decide. If the submitted total differs from the AI proposal by more than the subject's threshold, a moderator reviews the script.
What makes AI marking more accurate?
The rubric matters more than the model. Questions with a model answer and a rubric get specific, checkable proposals; questions without them can only be guessed, and AutogradeX lists such questions so coordinators can complete them.
| Factor | What AutogradeX does |
|---|---|
| Handwriting legibility | Full-resolution OCR checked against the image; illegible pages lower confidence |
| Question labelling | Matches "Q1(a)", "1 a" and "Question 1A" to the same scheme question |
| Model disagreement | Median of independent graders, with agreement shown per question |
| Missing rubric | Lists questions without a model answer or rubric before marking starts |
| Run-to-run variation | Fixed, low-randomness settings so repeated runs stay consistent |
Where is student data processed?
In the institution's own cloud subscription. AutogradeX calls the Azure OpenAI and Azure AI Foundry deployments the institution configures, and scans stay in its private storage. AI can be switched on per subject, and the whole evaluation workflow also works without it.
Frequently asked questions
Can AI evaluate handwritten answer sheets accurately?
AI can propose marks that evaluators find useful when the handwriting is legible and the question has a model answer and rubric. Accuracy varies with both, which is why AutogradeX shows a confidence score per question and leaves every final mark to an evaluator.
Which AI models does AutogradeX use?
The ones the institution deploys: Mistral Document AI for OCR in Azure AI Foundry if enabled, a vision model to check the text, and two or more grader models in Azure OpenAI. Using different models for the graders makes their errors less likely to coincide.
Does AI evaluation work for diagrams and mathematics?
Equations are kept in mathematical notation and diagrams are described in the transcript, so the graders can take them into account. Proposals for diagram-heavy answers are usually less confident, and the evaluator checks the drawing on the page.
Related
- On-screen markingAutogradeX is an on-screen marking system for university answer books: scanned scripts, annotations, double evaluation, moderation and result export.Read
- ScanningScan bound answer booklets at the exam centre or hub with a phone, a USB document camera or an overhead scanner, without cutting the spine.Read
- SecurityHow AutogradeX protects examination data: multi-factor sign-in, role-based access, a tamper-evident custody ledger and storage in your own cloud.Read
See AutogradeX on your own answer booklets