College of New Jersey Professor and student using Google's Media Pipe to interpret sign language

In early March 2019, DAS Labs worked, in collaboration with a sign language professional associated with a prominent school for the deaf, on a prototype leveraging machine learning models to identify and teach users how to sign specific items, in their phone's field of view. This project had many challenges to overcome like storing the sign language demonstration videos, and deciding how large of a general object detection model to use on mobile, but was super interesting to collaborate on.

Recently, Kaitlyn Bonomo from the College of New Jersey reported a Computer Science professor and student duo doing the something similar but the user flow was the opposite to DAS Lab's 2019 project. Instead this project's goal is translate actual sign language, being used by people, into written English: https://news.tcnj.edu/2023/05/01/computer-science-asl-project/ .

With Google's Media Pipe, developers can harness some pretty sophisticated skeletal tracking, even extracting 3d feature points now, to estimate the position, pose and orientation of a person's hand to try to identify what letter or signed expression is being made. The challenges increase when trying to interpret a series of different expressions requiring more than a single pose, in other words a non-static, sweeping hand/arm gesture.

We at DAS Labs think this sort of ML work is just another intriguing example of what artificial intelligence can offer to increase accessibility and communication between people that speak different languages and/or have specific disabilities.

DAS Labs currently works with a venture backed mapping company whose sole focus is helping those with disabilities to navigate large indoor spaces, like hospitals, shopping centers, etc. using Augmented Reality.

Also, in the past, DAS built out a proof of concept for a visually impaired founder, Aziz Zeidieh of RannMax, using iPhone's LIDAR capability and object classification to navigate through public places, with object detection and obstacle avoidance - similar to how advanced models are used in autonomous driving vehicles.

"“Michael and his team took our idea for an app and turned it into a functional, accessible, and world class proof of concept. We could have gone with any one of thousands of iOS app developers, however, it was quality we wanted, and it was quality and 100% satisfaction we got when we chose to work with Dynamic Augmented Solutions.” -Aziz Zeidieh, Co-founder and CEO of RannMAX"

DAS Labs loves bringing to bear these and other augmented reality, computer vision and machine learning project experiences to provide innovators with meaningful solutions, deployable from mobile phones and AR headsets or glasses, that work in real world conditions.