July 28, 2025 \t \t \t \t \t \t \t \t \t \t \t <\/p>\n
One day, it might be possible to learn American Sign Language from anywhere \u2014 at home, in an office or at a party \u2014 as long as you\u2019ve got a pair of smart glasses, a watch or even a ring. <\/p>\n
This is the future Binghamton University computer scientist Yincheng Jin envisions in his recently awarded National Science Foundation CAREER grant of $599,588, aiming to design a daily ASL training platform using wearable technology and artificial intelligence.<\/p>\n
The CAREER award is the highest distinction the NSF offers to researchers it has tapped to become future leaders of their fields. As an assistant professor in the Thomas J. Watson College of Engineering and Applied Science\u2019s School of Computing, Jin focused his previous work on the area of wearables for ASL translation. Yet, he quickly discovered there was more to it: Many people, including the deaf and hard of hearing (DHH) community, don\u2019t understand ASL well enough in the first place. <\/p>\n
\u201cWhen I went in depth, I found that most DHH people aren\u2019t familiar with sign language. If they\u2019re not familiar, the translation is not that important,\u201d Jin said. \u201cI want to go further to help this community by helping them learn, so that\u2019s why I conducted this project.\u201d<\/p>\n
The reason why it\u2019s so difficult to grasp ASL, despite the prevalence of online learning resources, is not only that certain areas, such as rural regions, lack qualified teachers. It\u2019s also because those online lessons are just that \u2014 online. <\/p>\n
\u201cEven though websites or online materials increase, the limitation lies in that there is no one who helps [ASL learners] check whether their sign is correct or not,\u201d Jin said. \u201cIn that case, they are unable to practice by themselves and not able to improve, because of limited teachers, and are unable to teach themselves. This limits the ability of sign-language learners.\u201d <\/p>\n
Jin\u2019s latest research will be the first to combine wearable technology and advanced AI to provide the capability of ASL learning daily. <\/p>\n
\u201cWe proposed our method to use recent, advanced wearable technologies to sense or detect ASL motions, including facial and hand gestures,\u201d he said. \u201cAnd based on advanced large language models such as ChatGPT, we want to conduct a solution for learners by sensing gestures and providing corrective feedback.\u201d<\/p>\n
This way, anybody with access to such technology can learn ASL anywhere, while simultaneously receiving teacher-like feedback \u2014 even if no one is actually there to lead the class. <\/p>\n
\u201cI want to help all people who are interested in ASL or are required to learn ASL, which means that they just need to bring a wearable device \u2014 like smart glasses with three cameras \u2014 and they can practice ASL words and sentences and grammar,\u201d Jin said. \u201cAll of that, they can do it by themselves, whatever and wherever they want.\u201d <\/p>\n
To get to this point, Jin and his team must first gather data on all components of sign language for these devices and reconstruct those signs. His team will also use advanced machine learning algorithms to address the challenges of limited data, generalizability and differing scenarios. Students in Binghamton\u2019s ASL classes will have the opportunity to use Jin\u2019s AI-powered devices and train on ASL by themselves. <\/p>\n
\u201cWe combine the reconstructed gestures with online gestures and provide which part of your sign is incorrect,\u201d Jin said. \u201cLike a real teacher, we will use the large language model to give human-like feedback.\u201d <\/p>\n
On top of working with ASL learners and the DHH community in the Binghamton area, Jin will collaborate with ASL expert and educator Kenneth Dehaan. The results of this project can feed into the design of mobile apps that aid ASL learning and potentially never-before-seen smart devices, while generating robust datasets to be used in future courses on human-computer interaction.<\/p>\n
But while Jin\u2019s CAREER research is primarily focused on aiding ASL education, the principles of this technology apply to many other fields. He imagines his research might lend a hand to robot control and interactions, or even rehabilitation services. Take a gander through his previous work, and you\u2019ll similarly see the names of many creative solutions bridging people and computers already \u2014 ThermoTag, EchoWhisper, SonicASL. <\/p>\n
All of this, he said, is in the name of \u201cimproving human well-being, and to make humans live more intelligently.\u201d <\/p>\n","protected":false},"excerpt":{"rendered":"
July 28, 2025 \t \t \t \t \t \t \t \t \t \t \t One day, it might be possible to learn American Sign Language from anywhere \u2014 at home, in an office or at a party \u2014 as long as you\u2019ve got a pair of smart glasses, a watch or even a ring.
\n This is the future Binghamton University computer scientist Yincheng Jin envisions in his recently awarded National Science Foundation CAREER grant of $599,588, aiming to design a daily ASL training platform using wearable technology and artificial intelligence.
\n The CAREER award is the highest distinction the NSF offers to researchers it has tapped to become future leaders of their fields. As an assistant professor in the Thomas J. Watson College of Engineering and Applied Science\u2019s School of Computing, Jin focused his previous work on the area of wearables for ASL translation. Yet, he quickly discovered there was more to it: Many people, including the deaf and hard of hearing (DHH) community, don\u2019t understand ASL well enough in the first place.
\n \u201cWhen I went in ..<\/p>\n","protected":false},"author":3,"featured_media":15583,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25,28,2],"tags":[],"class_list":["post-14666","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","category-technology","category-tie-world"],"_links":{"self":[{"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/posts\/14666","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/comments?post=14666"}],"version-history":[{"count":2,"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/posts\/14666\/revisions"}],"predecessor-version":[{"id":15582,"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/posts\/14666\/revisions\/15582"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/media\/15583"}],"wp:attachment":[{"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/media?parent=14666"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/categories?post=14666"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/skybeaconnews.com\/index.php\/wp-json\/wp\/v2\/tags?post=14666"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}