SmartSO: Chinese Character and Stroke Order Recognition with Smartwatch

Following the correct stroke order while writing Chinese characters composed of strokes plays an important role in handwriting efficiency and quality, especially for early education. Most existing systems use image processing techniques for character and stroke order recognition, which is sensitive to lighting conditions. In this paper, we present the design, implementation and evaluation of SmartSO, which utilizes the inertial sensors of an off-the-shelf smartwatch for Chinese character and stroke order recognition. SmartSo first identifies the Chinese character written by the user, based on which SmartSo decides whether the stroke order is written correctly to help improve users' writing behavior. The biggest challenge for stroke order recognition is that some Chinese characters have repeated strokes (strokes of the same type), e.g., "with two same horizontal strokes, and it is challenging to differentiate the writing order of such strokes given only the detected stroke composition (number and type of strokes). To mitigate this problem, we further analyze the hand movement between two adjacent strokes (referred to as direction motion) and propose a novel algorithm to recognize stroke order based on direction motion information. Finally, we build a fully functional prototype of SmartSO, and extensive experiments confirm its effectiveness and robustness.