December #1
In this short month of December, I have only researched a little more, into technology and into companies.
In the study, The RWTH-PHOENIX-Weather Database of German Sign Language, that my engineering mentor helped me find, I learned that their current technology mainly uses video recognition and to locate, detect, and recognize sign language movements. The included pictures illustrate the details how they spot distinguishable points on people's faces, hands, joints, etc. to decide what movement it is. Facial expression as part of sign language expression is included besides hand movements.
Another key point I realized is that, all the detection are conducted from an outside point of view. However, I've been envisioning my translator to be worn by deaf people themselves, so that the sign language detection needs to be done from the other direction, deaf people's points of view. This is going to be a big challenge that I will consider now.
On Kitware, Inc.'s website, the company that my on-campus business mentor recommended to me, I figured out that they have cooperated with many open-source platforms on software development or prototyping. They also provide services of counseling and training to start-up companies, in data analysis, software computing, and more.
After coming back from winter break in January, I will start focusing on the business part with Ms. Bernard and decide what parts I need to dig in first, and I will consider reaching out to Kitware and other companies of this kind for help accordingly.
December 14, 2017
December 1, 2017
Interview
November #2
Ms. Tena Tyler's lecture about how to conduct interviews was really helpful. I learned that we as interviewers should keep good manners, do research in advance, ask good questions, and pay attention to context. The note-taking tips that she mentioned were also crucial, including reserving some margin space, writing down my observations, developing my own short-hands. I also remember that I should keep listening after closing my notebook in order to not leave out anything.
With all these advice, I had an interview with my off-campus mentor, Guangliang Zhao.
Dr. Zhao is an engineer at General Electrics (Global Research Center in Niskayuna). He mainly works on the research and development of image recognition and its application in company products. With the background of signature project, he guided me to look into various existing technologies that have implications of sign language movement recognition through camera. He elaborated on the limitation of using image recognition for detection, so we shifted to consider video recognition.
When I asked "to what degree is video recognition currently developed?", he showed me a research online https://www-i6.informatik.rwth-aachen.de/~forster/database-rwth-phoenix.php which showcases a database used for sign language interpretations. I furthered asked what some next steps that I could take on are. He answered that, we would definitely continue researching into accessible and helpful technologies, and it might be a good idea to talk with my business mentor (Ms. Bernard on campus) to clarify what information from the technology side is needed for constructing my business plan. In this way, we can prioritize discussing those required info, and it can be helpful to set a clear plan/path.
Ms. Tena Tyler's lecture about how to conduct interviews was really helpful. I learned that we as interviewers should keep good manners, do research in advance, ask good questions, and pay attention to context. The note-taking tips that she mentioned were also crucial, including reserving some margin space, writing down my observations, developing my own short-hands. I also remember that I should keep listening after closing my notebook in order to not leave out anything.
With all these advice, I had an interview with my off-campus mentor, Guangliang Zhao.
Dr. Zhao is an engineer at General Electrics (Global Research Center in Niskayuna). He mainly works on the research and development of image recognition and its application in company products. With the background of signature project, he guided me to look into various existing technologies that have implications of sign language movement recognition through camera. He elaborated on the limitation of using image recognition for detection, so we shifted to consider video recognition.
When I asked "to what degree is video recognition currently developed?", he showed me a research online https://www-i6.informatik.rwth-aachen.de/~forster/database-rwth-phoenix.php which showcases a database used for sign language interpretations. I furthered asked what some next steps that I could take on are. He answered that, we would definitely continue researching into accessible and helpful technologies, and it might be a good idea to talk with my business mentor (Ms. Bernard on campus) to clarify what information from the technology side is needed for constructing my business plan. In this way, we can prioritize discussing those required info, and it can be helpful to set a clear plan/path.
Meeting with Ms. Bernard
November #1
On Nov. 16, I had a meeting with my on-campus mentor, Ms. Bernard from the Business Office, to introduce my project and to discuss the future steps.
I explained that my signature this year is a continuation of my business model competition project last year, and now I'm specifically focused on developing the preliminary translation system (software part) of the sign language translator concept that I had from last year. I will work closely with Ms. Bernard to build a formal business plan on the improved idea with a technological basis of the software system.
We agreed that it was a good idea that I started off my year (during Sept. and Oct.) working on the technology side and exploring what technologies/equipment I may need.
Ms. Bernard recommended a local company called Kitware (https://www.kitware.com). They offer help on projects, counseling on research development, and collaboration with start-up companies. I will research into Kitware and see if there is any opportunity that I can later contact them for advice.
Ms. Bernard and I planned to start working on the business part of my signature after winter break in January. So before then, I will continue figuring out the technology part of my off-campus mentor, a GE engineer.
On Nov. 16, I had a meeting with my on-campus mentor, Ms. Bernard from the Business Office, to introduce my project and to discuss the future steps.
I explained that my signature this year is a continuation of my business model competition project last year, and now I'm specifically focused on developing the preliminary translation system (software part) of the sign language translator concept that I had from last year. I will work closely with Ms. Bernard to build a formal business plan on the improved idea with a technological basis of the software system.
We agreed that it was a good idea that I started off my year (during Sept. and Oct.) working on the technology side and exploring what technologies/equipment I may need.
Ms. Bernard recommended a local company called Kitware (https://www.kitware.com). They offer help on projects, counseling on research development, and collaboration with start-up companies. I will research into Kitware and see if there is any opportunity that I can later contact them for advice.
Ms. Bernard and I planned to start working on the business part of my signature after winter break in January. So before then, I will continue figuring out the technology part of my off-campus mentor, a GE engineer.
November 3, 2017
Next-step Plan
October #2
Special equipment for data processing: Open source computing
Special equipment for data processing: Open source computing
- Intel Realsense https://software.intel.com/en-us/realsense-sdk-windows-eol
- Sense Time co. https://www.sensetime.com/videoMonitor/
- (need more research and engineering help)
Business Plan:
- Will start drafting
- Will discuss with Ms. Bernard
- Will make a list of companies that I wish to contact later
Research--Technologies
October #1
In October, I mainly working on researching into existing technologies that have implications for sign language movement detection (through camera).
Image Recognition:
Video Recognition:
Human Pose Estimation
(Articulated) Hand Tracking
In October, I mainly working on researching into existing technologies that have implications for sign language movement detection (through camera).
Image Recognition:
- Basically mature nowadays. My mentor has the technology himself.
- Photos taken by normal camera (e.g. on smart phones) can be used
- Able to process thousands of data/images now
- After collecting data, the processing is easy to be operated on common computational models
- can add fish-eye lens (universal 3-in-1 clip-on camera lens) --> wider angle
Video Recognition:
- currently allows only tens of videos
- requires depth camera to collect data (e.g. Kinect)
- requires professional computers to process data **(depth analysis? need more research)
Human Pose Estimation
- requires depth camera to collect data (e.g. Kinect)
- requires professional computers to process data
- need more research
e.g. Realtime Multi-Person 2D Human Pose Estimation: https://youtu.be/pW6nZXeWlGM
(Articulated) Hand Tracking
- requires depth camera to collect data (e.g. Kinect)
- requires professional computers to process data
- need more research
- e.g. https://youtu.be/A-xXrMpOHyc
 |
| Handpose: Articulated Hand Tracking |
 |
| Kinect 3D Hand Tracking |
October 13, 2017
Timeline
September #2
September:
- set goals for the project
- contact mentors
October:
- research more on sign language recognition method (e.g. Video recognition, Hand tracking, Human pose estimation...)
- brainstorm about the business plan
November:
- figure out what resources are needed for the proposed method
- consult mentors
- start writing the business plan
December:
- collect sample data for the recognition method
January:
- continue writing the business plan
- process sign language data
February:
- Finish the business plan
- run the recognition model on computer
March:
- start contacting technology companies
- refine the model
April:
- present the business case to companies
- finish the preliminary prototype
May:
- conclude the experience of contacting companies
- signature presentation
About Me
September #1
Hello!
My name is Nina, and I'm a senior in high school this year.
The STEM signature project I'm working on is mainly about a Sign Language translator (system). The goals are: (1)build a business case for the sign language translator concept to present to technology companies, and (2)create the translation system prototype.
This is a continuation of my "I to I" business plan project, prepared for New York State High School Business Model Competition in May 2017. In a group of four, we observed the urgent need of a Sign Language translator to help deaf people with their communication with hearing people. We then developed the idea of translator smart glasses that diminish the barrier between deaf and hearing people by connecting them Eye to Eye.
My group won the 2nd place in the competition, and we decided to carry on this project in a more technical way. Since it is foreseen to be super challenging for me to take care of all professionally technical parts, I will work on constructing an elaborated business plan in preparation for future contact with technology companies, and create a preliminary system as the supplementary demonstration.
My interests in business, physics, and engineering have led me to this project. It's really cool to conduct a project through which I turn an idea into actions in real life, and I connect myself with different groups of people, including deaf people, sign language teachers, and engineers, outside of school.
Hello!
My name is Nina, and I'm a senior in high school this year.
The STEM signature project I'm working on is mainly about a Sign Language translator (system). The goals are: (1)build a business case for the sign language translator concept to present to technology companies, and (2)create the translation system prototype.
This is a continuation of my "I to I" business plan project, prepared for New York State High School Business Model Competition in May 2017. In a group of four, we observed the urgent need of a Sign Language translator to help deaf people with their communication with hearing people. We then developed the idea of translator smart glasses that diminish the barrier between deaf and hearing people by connecting them Eye to Eye.
My group won the 2nd place in the competition, and we decided to carry on this project in a more technical way. Since it is foreseen to be super challenging for me to take care of all professionally technical parts, I will work on constructing an elaborated business plan in preparation for future contact with technology companies, and create a preliminary system as the supplementary demonstration.
My interests in business, physics, and engineering have led me to this project. It's really cool to conduct a project through which I turn an idea into actions in real life, and I connect myself with different groups of people, including deaf people, sign language teachers, and engineers, outside of school.
 |
| "I to I" group icon |
 |
| Sign Language "A-Z" |
 |
| "I to I" smart glasses_preliminary design |
 |
| AR technology |
Subscribe to:
Posts (Atom)