Image Based Interactive Device for Effective coMmunication (IBIDEM)

TIDE Project 1038


Objectives

The main objective of IBIDEM is to develop a videophone useful for lip reading by hearing-impaired people as well as provide capabilities for remote monitoring based on a novel type of space-variant visual sensor and using standard telephone lines. Long-distance communication for both social and practical purposes is becoming an increasingly important factor in every-day life. Hearing-impairment does, however, prevent many people from using normal voice telephones for obviuos reasons. A solution to this problem for the hearing-impaired is the use of videophones. Currently available videophones working on standard telephone lines (PSTN) do, however, not meet the dynamic requirements necessary for lip reading. The spatial resolution is also too small. In order to facilitate lip reading, signing, and finger spelling, IBIDEM will develop a videophone based on a novel type of visual sensor matching the resolution of the human retina in both the spatial and temporal domains (a retina-like or space-variant sensor). Members of the IBIDEM consortium already hold a patent in Europe and the US for a prototype of such an imaging device. The geometry of the visual sensor, similarly to the human retina, has a high resolution in the central part and a degrading resolution in the peripheral visual field, as shown in the figure. This solution results in a reduction of the number of pixels of the acquired image (allowing a higher transmission rate on standard telephone lines) without degrading the perceptual appearance of the image, as can be seen from figure.
Picture Here
The figure shows in the background the layout of a space-variant sensor having a resolution of 128 photoreceptor on each of the 64 eccentricity; the total amount of pixel is then 8192. The circular image represents the output of such a sensor during "finger spelling" (this is the image that will be shown) while the rectangular image on the right shows (enlarged) the log-polar representation of the image (this is the image that is being received/transmitted).

A second objective of IBIDEM is the use of the same equipment for remote monitoring of health status. The system can be used for obtaining information about the status of a client in the form of images and could be extended to include various physiological parameters like heart rate, blood pressure etc. The IBIDEM project will construct a videophone using a camera with the retinal sensor, a motorized system for moving the point of view of the camera as well as a LCD to display the transmitted images. This videophone will be a high-quality, low-cost aid for both the hearing-impaired as well as being useful for remote monitoring. The videophone will be designed with active participation of members of the deaf and hard-of-hearing community, and will be demonstrated by inter-personal communication between two speakers one or both with hearing disabilities as well as in a situation with remotely monitoring health conditions using audio and visual information.


Click here to view a sequence of a "finger spelling" experiment.

This sequence has been obtained using a simulated sensor of about 8,000 pixels. The sequence shows the images as they could appear on the screen of the videophone (sorry for the greenish appearance, in reality will be B/W).

Contact Person

Dr. Fabrizio Ferrari	tel. +39-10-3620102
UNITEK Consortium	fax. +39-10-314873
via Pisa 12/1		e-mail: ferrari@aitek.it
16147 Genova - Italy

Consortium

Participants		Role (Nation)	Principal Investigators
UNITEK Consortium	Coordinator (I)	Jan Nielsen, Andrea Mannucci
IMEC			Partner (B)	Belgium	Lou Hermans, Bart Dierickx
Thomson-CSF		Partner (FR)	Eric Auffret, Christophe Guerin
Knossos Technologies    Partner (GR)	Andreas Manganas, Antonis Argyros
DIST - Univ. of Genoa	Partner (I)	Giulio Sandini, Paolo Questa
Scuola Sup. S. Anna	Partner (I)	Paolo Dario, Alessandra Sassi
Instituut voor Doven	Associate (NL)	Han Frowein, Wendy Woelders
Scienzia Machinale Srl	Subcon. (I)	Luca Bosio
A.F.A. Centro R.E.U.L.	Subcon. (I)	Silvana Baroni, Claudio Rossi