Practical Considerations on Handwriting Recognition

The full potential of AI-assisted grading requires text to be in a computer-readable format. While some features like answer shuffling can work with handwritten input, more advanced functionality demands digital text.

Optical Character Recognition (OCR) technology converts images of text into machine-encoded data, enabling automated processing. While OCR performs reliably with typed documents, handwritten content poses significant challenges due to varying writing styles. Handwriting Recognition addresses these challenges through two approaches: "offline" recognition, which processes static images of handwritten text, and "online" recognition, which utilizes pen movement data for improved accuracy.

Handwriting recognition proves particularly unreliable in exam environments for several reasons:

• Stress-induced poor handwriting quality
• Non-linear responses with corrections and annotations
• Complex formatting including arrows or "continued on next page" notes
• Individual word errors that can distort the meaning of entire answers

Even when successful word recognition is achieved, subsequent language and grammar corrections do not significantly improve the overall quality of the digitized text. Given these limitations, we initially decided to restrict our AI-assisted grading to assessments written in digital format. This approach ensured optimal performance and reliability of the grading system, avoiding the complications introduced by handwriting recognition technology.

Recent developments in Large Language Models (LLMs) show promising advances in handwriting recognition, particularly in academic settings. At ETH Zurich, Kortemeyer (2023) achieved significantly high accuracy com-pared to human graders using a workflow combining MathPix for text recognition and GPT-4 for assessment. However, even with these advances, challenges persist. Future improvements, especially in multimodal language models, are expected to further improve the quality of handwriting recognition.

Some examples of commercial handwriting recognition applied to exam text can be seen below:

Example of OCR applied to handwritten exam responses