Scribble handwriting recognition in iPadOS 14 is a more complex system than many might realize. Identifying letters and words goes beyond simply examining what someone has written. The iPad uses machine learning to recognize the writer’s intent through how they write, according to Craig Federighi, Apple’s SVP of software engineering.
Scribble depends on machine learning
To develop Scribble, Apple didn’t create an abstruse theory of handwriting and then try to apply it to the real world. Instead, it gave iPads and Apple Pencils to people around the globe and asked them to write. The tablets learned by watching people do it. That’s how machine learning works.<!– –>
In everyday use, Scribble continues to observe the process of writing, not just the final result. “If you understand the strokes and how the strokes went down, that can be used to disambiguate what was being written,” Federighi told Popular Mechanics.
The process of voice recognition for Siri proves so complex that much of the job is handled by centralized servers — it’s too hard for an iPhone by itself. But that’s not true of Scribble. For handwriting to be fast enough to be useful, the iPad has to do it all. “It’s gotta be happening in real time, right now, on the device that you’re holding,” Federighi said. “Which means that the computational power of the device has to be such that it can do that level of processing locally.”
More about Scribble
Scribble debuted in iPadOS 14. It can be used to enter text by writing on the screen, either print or cursive. iPad owners can enter text using a wide range of styli, not just the Apple Pencil. It works in most places a keyboard does. However, this system is so new that third-party applications are still adding support.
To learn more, read Cult of Mac’s guide: How to enter handwritten text with Scribble on iPadOS 14.