KuchiNavi: lip-reading-based navigation app

Tatsuya Kanamaru; Takeshi Saitoh

doi:10.1117/12.3021118

25 March 2024 KuchiNavi: lip-reading-based navigation app

Tatsuya Kanamaru, Takeshi Saitoh

Proceedings Volume 13089, Fifteenth International Conference on Graphics and Image Processing (ICGIP 2023); 130891C (2024) https://doi.org/10.1117/12.3021118
Event: Fifteenth International Conference on Graphics and Image Processing (ICGIP 2023), 2023, Suzhou, China

Abstract

Lip-reading technology has the advantage that it can be used even in noisy environments and has been actively studied in recent years. In this paper, we develop a navigation application, "KuchiNavi," as a new application using lip-reading technology. The basic technology is word-level lip-reading technology, which utilizes an existing deep-learning model. However, we quantitatively evaluated lip-reading accuracy by selecting words for navigation, collecting utterance scenes independently, building an original dataset, and conducting recognition experiments. This paper, 101 Japanese words were selected, utterance scenes were collected from 15 people, and recognition experiments were conducted using the speakerindependent recognition task, the leave-one-person-out method. As a result, an average recognition rate of 88.2% was obtained. In addition, we developed an iOS app and conducted a demonstration in a car to confirm its effectiveness.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Tatsuya Kanamaru and Takeshi Saitoh "KuchiNavi: lip-reading-based navigation app", Proc. SPIE 13089, Fifteenth International Conference on Graphics and Image Processing (ICGIP 2023), 130891C (25 March 2024); https://doi.org/10.1117/12.3021118

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available