Sign Language Translator

[Njordic]

Hey guys,

I'm looking for something that can translate sign language in real-time. If not someone with the know how to help build and code something like it. My Girlfriends sister is Deaf and finds it hard to get a job and to interact with people so I was wondering how hard this actually could be to build. All you would need is some finger sensors on a glove and maybe a Rasberry Pi to learn the gestures. I mean phones learn simple gestures so why can't we programme it to learn ASL or BSL to translate and a speaker the deaf person can wear. Is this possible to build? It cant be that expensive either, can it? 

Any Responses are welcome, thank you. 

[tikker]

It's a little tricky I'd assume, since sign language also has its dialects. This looks up your alley though:

https://www.newscientist.com/article/2133451-automatic-sign-language-translators-turn-signing-into-text/

http://www.kintrans.com

[Volbet]

It was tried about a year ago: https://www.newscientist.com/article/2140592-glove-turns-sign-language-into-text-for-real-time-translation/

 

Ultimatly, it's going to be very difficult to make such a thing. While ASL movements are pretty standardized, there's still going to be subtly within each person.

There's also dialects and slang within ASL. 

 

Hell, look at how difficult it is for computers to recognize handwritting via a direct touch input. Just imagine how difficult it would be for a computer to translate ASL via a camera or sensors. 

[Njordic]

Just now, tikker said:

It's a little tricky I'd assume, since sign language also has its dialects. This looks up your alley though:

https://www.newscientist.com/article/2133451-automatic-sign-language-translators-turn-signing-into-text/

http://www.kintrans.com

Yeah, It'll be tricky but it's almost like every language and Google Translate has shown it's possible. I know Google has literally the biggest team behind any Linguistics software of it's kind but even if it's the trouble of such dialects businesses could still make money from selling each one separately. If you could use Googles API for Google Translate isn't there also a strong possibility that the actually translating to whatever language won't be hard because all you would need to do is decipher what the sign the person is doing is saying and simultaneously put it through Google Translator so it reads it out? 

[Njordic]

Surely with AI or Computer Learning software, it could detect the difference, or am I over simplifying this? 

[tikker]

47 minutes ago, Njordic said:

Yeah, It'll be tricky but it's almost like every language and Google Translate has shown it's possible. I know Google has literally the biggest team behind any Linguistics software of it's kind but even if it's the trouble of such dialects businesses could still make money from selling each one separately. If you could use Googles API for Google Translate isn't there also a strong possibility that the actually translating to whatever language won't be hard because all you would need to do is decipher what the sign the person is doing is saying and simultaneously put it through Google Translator so it reads it out? 

The product I linked literally does that.

 

The thing with these kind of things is that it works best with a large team behind it if you go the machine learning route. It needs to be trained, have a large database etc. It's no coincidence that Google Translate is one of the (if not the) best around while simultaneously having a dedicated team and lots of resources.

[Njordic]

1 hour ago, tikker said:

The product I linked literally does that.

 

The thing with these kind of things is that it works best with a large team behind it if you go the machine learning route. It needs to be trained, have a large database etc. It's no coincidence that Google Translate is one of the (if not the) best around while simultaneously having a dedicated team and lots of resources.

That's awesome tech but that uses a camera as the sensor so using it as a personal device is impossible, it's crazy to think no one's done this as a finger sensor. Thanks guys for the help

[tikker]

3 hours ago, Njordic said:

That's awesome tech but that uses a camera as the sensor so using it as a personal device is impossible, it's crazy to think no one's done this as a finger sensor. Thanks guys for the help

Ah you were specifically looking for a glove type thing. Well then @Volbet's article looks to be the closest to that so far. It's not that nobody has done it, more that it may not be ready for production yet.

 

Here's a video of such a glove: https://www.youtube.com/watch?v=l01sdzJHCCM