Despite their increasing capabilities, SW  digital editors (SW-DEs) are far from granting the user a sign-composition interface capable of challenging the simplicity of the pen-based handwriting approach.
At present, any SW-DE developed so far relies heavily on “Windows, Icons, Menus, Pointer” (WIMP) interfaces. Working with deaf people for many years, we observed that SW users are far more accurate, fast and comfortable when using the paper-pencil approach, than when dealing with the (more or less) complex interaction style of a SW-DE.
Observing the state of the art in SW-DEs, we laid the foundation for SWORD (SignWriting Oriented Resources for the Deaf), a project towards the full integration of deaf people in the digital society. SWift , a new SW-DE, was produced within SWORD, but it still carries most intrinsic shortcomings of its peers, including the strict dependency on WIMP interfaces.
Within SWORD, we also conceived a new generation of SW-DEs, able to partially overcome the concept of the WIMP interface and to move along the line of the so-called “natural interfaces” . The new tools are meant to relieve the user from burdens related to clicking, dragging, browsing on the User Interface (UI) during the SW production process, and to provide an interaction style which is as similar as possible to the handwriting approach.
Of course, since WIMP is currently the most common interface style, it cannot be totally left behind, because the interaction with most application is still firmly grounded on it. Nevertheless, our aim is to limit its use, or to dismiss it whenever possible, during the SW production process, which is the core part of any SW-DE. To achieve this goal, we are designing and implementing a SignWriting Optical Glyph Recognition (SW-OGR) engine (Fig. 1). Its aim is to perform the electronic conversion (recognition) of user-produced images containing handwritten (or printed) SW glyphs into machine-encoded (via ISWA) SW texts . Such texts can be stored using the same formats of the documents produced by SWift, and therefore allow the same statistical and linguistic analyses. In particular, introducing a SW-OGR engine within an existing SW editor, such as SWift, will allow the user to handwrite symbols on the composition area, rather than searching them among thousands other symbols. Each handwritten symbol will be processed and replaced with its standard ISWA version by SW-OGR.
Fig. 1: Screenshot taken from SW-OGR during the development phase (ongoing). The image shows a handwritten text (courtesy of ISTC-CNR) and the result
of the detection routine for the head symbols
The main difference between OCR and OGR is in the nature of SW: SW is composed by an extremely high number of symbols (about 38,000, compared with 26 letters of the Latin alphabet) in a multilinear/two-dimensional arrangement, without rigid rules for such composition and with a complex segmentation, particularly when handwritten. Due to the large amount of patterns to recognize, and consequently to the overwhelming training required, it is not feasible to exploit traditional pattern-matching or machine-learning approaches. Therefore we are adopting an approach based on geometric and spatial features of the glyphs.
2016 Presentation 64 Sign Wikipedias
ASL Presentation by
Adam Frost, Jason
Nesmith, Holly Sharer
& the CODA Brothers
Plus special video on
Tunisian Sign Wikipedia
by Mohamed Ali Balti