Sign Speak

Lisha Kurian; Anaj Pravin; Calvin Johnson; Abhishek Unnikrishnan; Aswin Sunil1

1

Publication Date: 2024/11/26

Abstract: The project is to enable people who are not versedin sign language or people from the deaf or hard- of-hearing community to communicate by using a system that translates their American Sign Language (ASL) gestures into text, which could then be converted into speech. Computer vision and machine learning algorithms allow the system to “read” the sign language as accurately as possible, and then translate into a native text. Text is transcribed to speech using Text-to-Speech (TTS) capabilities The proposed calibration can be applied to real-time applications serving purpose for accessible and decent spoken communication among different individuals with hearing loss which applies the natural co-articulation constraints in various social or professional environments.

Keywords: Component, Formatting, Style, Styling, Insert.

DOI: https://doi.org/10.38124/ijisrt/IJISRT24NOV518

PDF: https://ijirst.demo4.arinfotech.co/assets/upload/files/IJISRT24NOV518.pdf

REFERENCES

  1. Truong, V.N., Yang, C.K. and Tran, Q.V., 2016, October. A translator for American sign language to text and speech. In 2016 IEEE 5th Global Conference on Consumer Electronics (pp. 1-2). IEEE.
  2. Camgoz, N.C., Koller, O., Hadfield, S. and Bowden, R., 2020. Sign language transformers:Joint end-to-end sign language recognition and translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10023-10033).
  3. Amrutha, K. and Prabu, P., 2021, February. ML based sign language recognition system.In 2021 International Conference on Innovative Trends in Information Technology (ICITIIT) (pp. 1-6). IEEE.
  4. Chen, M., Tan, X., Li, B., Liu, Y., Qin, T., Zhao, S. and Liu, T.Y., 2021. Adaspeech: Adaptive text to speech for custom voice. arXiv preprint arXiv:2103.00993.
  5. Zhou, Z., Chen, K., Li, X., Zhang, S., Wu, Y., Zhou, Y., Meng, K., Sun, C., He, Q., Fan, W. and Fan, E., 2020. Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays. Nature Electronics, 3(9), pp.571-578.
  6. Cheng, K.L., Yang, Z., Chen, Q. and Tai, Y.W., 2020. Fully convolu- tional networks for continuous sign language recognition. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIV 16 (pp. 697-714).Springer Inter- national Publishing.
  7. Tan, X., Chen, J., Liu, H., Cong, J., Zhang, C., Liu, Y., Wang, X., Leng, Y., Yi, Y., He, L. and Zhao, S., 2024. Naturalspeech: End-to-end text-to-speech synthesis with human-level quality. IEEE Transactions on Pattern Analysis and Machine Intelligence.
  8. Tiku, K., Maloo, J., Ramesh, A. and Indra, R., 2020, July. Real-time conversion of sign language to text and speech. In 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA) (pp. 346-351). IEEE.
  9. Kim, T.H., Cho, S., Choi, S., Park, S. and Lee, S.Y., 2020, May. Emotional voice conversion using multitask learning with text-to-speech. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 7774-7778). IEEE.
  10. Kahlon, N.K. and Singh, W., 2023. Machine translation from text to sign language: a systematic review. Universal Access in the Information Society, 22(1), pp.1-35.
  11. Ren, Y., Hu, C., Tan, X., Qin, T., Zhao, S., Zhao, Z. and Liu, T.Y., 2020. Fastspeech 2: Fast and high-quality end-to-end text to speech. arXiv preprint arXiv:2006.04558.
  12. Makhmudov, F., Mukhiddinov, M., Abdusalomov, A., Avazov, K., Khamdamov, U. and Cho, Y.I., 2020. Improvement of the end-to-end scene text recognition method for “text-to-speech” conversion. International Journal of Wavelets, Multiresolution and Information Processing, 18(06), p.2050052.