A real-time handwriting-to-LaTeX recognition model, built from small, modular networks: attention over
the strokes groups them into symbols, a CNN reads each symbol, and an attention model over symbols that builds the
tree structure (about 7M parameters in all), handling inputs and outputs of any length.
Try drawing:
Why I built this
I want to make real-time on device latex recognition broadly available, and to enable a new
generation of interactive apps for education, note taking, LLM interfaces and more.
The underlying
architecture isn't limited to math, and can be extended to any structured format - tables, diagrams, text.
Unlike sequence-based models, this architecture handles inputs and outputs of any length with a tiny fixed-size model.
This architecture is also unique in being inherently correctable and interactive, enabled by its tree-based modular structure.
For example, in selection mode, click on a symbol in the canvas and pick an alternative from the dropdown to correct it.