Elusive handwriting recognition

We live with the great promise of self-driving cars in five to ten years, with amazing cloud-based applications, virtual server computers, and a myriad of small apps that process, solve and answer virtually every question you may throw at them, short of the basic existential ones perhaps. So you’d think that in the realm of IT there is a software or hardware solution to every problem on the Earth.

Whereas indeed, fantastic solutions often exist for extremely complex, challenging matters, other apparently simple ones remain only partially addressed; quite surprisingly. Handwriting recognition (HR) is one of them. Today it remains to be perfected, especially when it comes to alphabets like Arabic.

HR goes through optical character recognition (OCR). First the text is scanned, then the “shapes” are analysed to be converted into digital characters, as if they were typed in a word or text processor of some kind. In most instances there is a need to OCR an existing already typed (i.e. not handwritten), paper-based document to convert it into a digital document for further editing.

With such documents OCR works like magic. From the amazing Omnipage by Nuance, definitely the world reference software system in this field, to free web-based OCR services, converting a typed hard-copy document you have into editable text is fast, affordable, easy, and achieves a output precision in the range of 95 to 99 per cent, depending on the quality of the printing, the font used, and other considerations. Unfortunately, recognising handwriting in another story.

Despite some progress accomplished since 1990, HR accuracy remains stalled in the 60 to 80 per cent range for mostly Latin-based alphabets, and falls below 60 per cent for Arabic. Such poor levels of accuracy make it not practical at all for actual use in the real world. The inherent complexity of the Arabic script, the variety of all the possible calligraphies, they all make it difficult if not impossible to meet the challenge, to this date at least. If HR accuracy is below 90 per cent you may as well retype the whole thing, it is faster and easier than correcting the faulty 10 per cent remaining.

Surprisingly the S-Note Android app that comes built in some of Samsung’s tablets and smarpthones is able to achieve HR with stunning accuracy, often higher than more complex Windows applications that claim to do the same, but still not good enough to make it a recognised standard that you can count on every day. Again, it does it well enough if you write English, French, German, Spanish, etc., not Arabic, and if you become familiar and skilled at using the company’s famous S-Pen. Although Samsung system claims Arabic HR, it is still experimental, by any standard.

Perhaps the reason for not achieving successful HR of languages like Arabic is not the technical difficulty per se, but the fact that no one in the industry wants to invest more time and effort on a task that may become useless in a few years. Given that we are typing much more than handwriting these days — and the best is yet to come, of course — who then needs HR?

Text is entered digitally in so many ways today, especially on mobile devices, that in the near future traditional handwriting is bound to become the exception not the norm. Just don’t tell schoolchildren yet.