It seems like with the current progress in ML models, doing OCR should be an easy task. After all, recognizing handwritten numbers was one of the prime benchmarks for image recognition (MNIST was released in 1994).
Yet, when I try to OCR any of my handwritten notes all I ever get is a jumbled mess of nonsense. Am I missing something, is my handwriting really that atrocious or is it the models?
Here’s a quick example, a random passage from a scientific article:
I tried EasyOCR, Tesseract, PPOCR and a few online tools. Only PPOCR was able to correctly identify the numbers and the words “J.” and “Chem.”. The rest is just a random mess of characters.
Edit: thank you all for shitting on my handwriting. That was not asked for, and also not helpful. That sample was intentionally “not nice” but is how I would write a note for myself. (You should see how my notes look like when I don’t need to read them again, lol)
chatGPT can transcribe it perfectly, and also works on a slightly larger sample. Deepseek works ok-ish but made some mistakes, and gemini is apparently not available in my country atm. I guess the context awareness is what makes those models better in transcription, and also why I can read it back without problems.
Maybe if your handwriting wasn’t so terrible, a machine could read it.
I just asked chatGPT to transcribe it and it said
The handwritten text in the image says:
“Dimer stabilization free energies were also determined from thermodynamic integration (TI, see methods), which provide a direct validation of the MM-GBSA results.”
J. Phys. Chem. B 2018, 122, 7038-7048
There was a post on HN recently about using LLMs for OCR. https://news.ycombinator.com/item?id=42952605
That’s perfect. Now I’m just wondering why chatGPT is apparently much better in OCR than a dedicated OCR model like EasyOCR or Tesseract.
Btw, Deepseek did a good job but not perfect. I also fed chatGPT a full page of notes and the transcription to markdown worked quite well, although not perfect. However, if I supply the same note as part of a larger pdf, it will refuse to transcribe it, stating that it’s unreadable.
Because LLMs can fill in gaps where the recognition fails.
Which can be problematic. If it makes a mistake and isn’t obviously wrong, that could go unnoticed.
100% agreed. But it doesn’t change the answer of why they are apparently better than OCR.
Yep
If I had to guess, I’d say it was the dot paper confusing the OCR reader. I suppose the LLM has some way to cancel out the dots and thereby gets a better scan of it.
Try gemini 2 it seems is pretty good at that as well
Huh that’s actually a better transcription than I was able to read of the handwriting, and I don’t exactly have good handwriting myself. I just couldn’t see any other reading of validation than ‘voliolation’.
I tried it a while ago out of desperation to read handwriting I couldn’t read at all, that’s why I thought to try it.
I’ve read that the USPS has amazing OCR for mail sorting. It is, of course, highly tuned for one particular data format.
also, banks and mobile check deposit. I’ve only ever seen it get it wrong once.
Try again on plain paper, or on lined/ruled paper. That dotted graph paper hurts my eyes and I’m pretty sure I’m mostly human…
I like dotted paper, the dots are less distracting than grids, lined paper sucks for sketches/etc. and with plain paper I’m missing guides. But I agree that on this particular one, the dots are a bit too prominent.
Are you trying to scan the text from paler with the dots? That is most likely making it even harder for the OCR to pick out the text.
You seriously need to work on your handwriting. I’m impressed OCR can make out anything at all from that.
This isn’t a OCR problem. This is a you problem. I’m human and I can only make out a few words.
Edit. Assuming it’s yours. Or is this from the scientific article? Regardless. Whoever wrote that needs to go back to third grade and redo their writing exercises.
I’m pretty good at reading terrible cursive, and this is my best attempt using the letters as written
Dime stabilization for enrjies were also determined from thermodynamih integsalion of the MM-GBSA results.
I think the first one in italics should be energies, but wouldn’t assume OCR would know the context to fill in the missing letters. Not sure what word that starts with thermo ends in an h or maybe a k. No idea on the one that starts with inte. I might have been able to determine those words if I was familar with the context, but OCR doesn’t work that way.
You took the time to spell your post correctly and use correct grammar.
I used to have very sloppy handwriting. I’ve come to realize that if you want other people to understand you, you do need to make an effort to be understandable.
Shortcuts in communication do not show superiority. Too many shortcuts devalue your communication, just like poor spelling and grammar would devalue your post.
I’m writing notes for myself and I can read them. When I’m writing for someone else (which rarely happens for handwritten notes) I take the time and effort to write nicer.
Also, I specifically didn’t write the example carefully because the use case for me would specifically be handwritten notes I made for myself.
So ideally there would be a way to train an AI on one’s own particular handwriting? (Not sarcasm or rudely)
Here’s what I got with Google Lens. Certainly some mistakes, but not “jumbled mess of nonsense.”
Dimes stabilization fire einiges were also delirmed. from thermodinamik integration (I), see methods), which provide a dimict, validation of the MM. GBSA results
J. Phys. Chem. B 2018, 122 7038-2048
That’s completely incomprehensible.
That IS a “jumbled mess of nonsense”!
