Textual Information Access for the Visually Impaired Ali Zandifar and Changjiang Yang, Ramani Duraiswami,  Larry Davis and Nail Gumerov

• Introduction
• Preliminary Prototype and Hardware
• Research Issues
  
  See other PIRL Publications

Perceptive Interface and Reality Lab

Introduction

• Approximately 1 million legally blind and 5 million visually impaired people live in the US. Similar proportions elsewhere
• Wearable computer based prostheses could help them access visual information in the environment.
• A significant portion of the available information in the world is textual
  - Product and pharmaceutical labels
  - Grocery and department store aisle signs
  - Street signs, ATM instructions etc.
• Goal: 
  - Develop a personal reader using off-the-shelf OCR and text-to-speech, wearable computers and cameras.
  - Developing necessary computer vision algorithms.
   

TOP

Preliminary Prototype and Hardware 

• Develop a personal reader that can be used to read magazines and newspapers.
• Direct it to the low vision population and use user input to detect text
  

• Color firewire camera with controllable zoom, focus, iris and saturation mounted on a light helmet
• Computer Vision and image processing unit
• Output and User interface( a speech synthesis module and/or display unit) 
  

 
TOP

Research Issues

• Identify test bearing region in the image
• Unwarp text bearing region into front-to-parallel rectangles to provide input to Optical Character Recognition (OCR)
• Develop dictionaries and strategies OCR on data in a specialized format (warnings, nutritional labels) that have a known format and dictionary
• Stabilization of the video image be stabilized to compensate for motion or jittering by a moving person
• Super-resolution and mosaicking of an image sequence 
• Real-time Camera Control
• Develop a user-interface suitable for the visually impaired
  
  (bold – currently active research areas)

See individual Links
- Unwarping Text on Curved Surfaces
- Mosaicking
- Super-Resolution Enhancement

TOP

Unwarping Text on Curved Surfaces 

• Using differential geometry based constraints to unwarp the image .
• On the warped surface:
      - Curves have the same length
      - Enclosed curves enclose the same area
• Use constraints and regularity of text spacing to unwarp text.

Mosaicking

• Images captured by a camera are low resolution.
• Capturing pieces of a page at high zoom and then stitching the pieces together (“mosaicking”) is a solution
• To determine the scan protocol:
      - Get the rough estimate of Camera zoom with respect to readable font size OCR
      - Zoom to the region of interest and auto-focus to remove degradation 

TOP

Super-Resolution Enhancement

In super resolution multiple low resolution images (that are slightly displaced) are combined to achieve a high resolution image.

• Subpixel registration of images by phase correlation method.
  

• Automated super-resolution of image sequences using back-projection and MAP estimator.