[CEATEC] NTT Docomo Prototypes Head-mounted Smartphone

Oct 4, 2012
Tetsuo Nozawa, Nikkei Electronics
The "Hands-free Video Phone"
The "Hands-free Video Phone"
[Click to enlarge image]
Two of the five cameras are used to take images of the user's eyes.
Two of the five cameras are used to take images of the user's eyes.
[Click to enlarge image]
On the rear side, there is a camera that takes an image of the background.
On the rear side, there is a camera that takes an image of the background.
[Click to enlarge image]
An image (including CG) of the user was used in a demonstration of a video call.
An image (including CG) of the user was used in a demonstration of a video call.
[Click to enlarge image]
Biosensors are expected to be embedded in the inside of the frame.
Biosensors are expected to be embedded in the inside of the frame.
[Click to enlarge image]
The "AR Shopping" function superimposing a video explaining the bag
The "AR Shopping" function superimposing a video explaining the bag
[Click to enlarge image]

NTT Docomo Inc announced a concept model of its next-generation smartphone called "Hands-free Video Phone" and exhibited a prototype having some actual functions at CEATEC JAPAN 2012.

At the show, the company demonstrated the video phone function of the prototype.

The Hands-free Video Phone is a headset being developed for making a video call while it is worn on the head like glasses. NTT Docomo plans to realize functions such as (1) a video phone function that uses AR (augmented reality)-based technologies, (2) the "Virtual Office" function, which enables hand gesture-based keyboard inputs, (3) the "Vital Monitoring" function, which monitors the user's health with sensors embedded in the headset and (4) the "AR Shopping" function, which displays, for example, a video explaining a product.

AR-based video phone

The prototype has the first function and part of the second function. As for the first function, the headset is equipped with five small cameras using a fisheye lens, etc. Two of the five cameras are used to take images of the user's eyes and their surroundings. And the user's face is reproduced by combining those images with the user's front image that is taken beforehand, CGs, etc. Furthermore, an image taken by the camera mounted on the back of the headset is superimposed on the user's image as a background image.

With this method, the user can transmit his/her image and the image of the background without holding a camera or placing a camera-equipped personal computer in front. At this point, however, it is not possible to see the image of the person on the other end of the line because the prototype is not equipped with a display.

Virtual Office

With the Virtual Office function, NTT Docomo aims to detect hand movements with two camera embedded in the lower part of the headset and use them for inputting characters. However, at this point, the prototyped headset can recognize hand movements only with a low accuracy, the company said.

Vital Monitoring

The Vital Monitoring and AR Shopping functions have not been added to the headset yet. For the former, NTT Docomo plans to mount a minute thermometer, blood-pressure meter, pulse monitor, biogas sensor that checks body odors, etc and use them on a daily basis for keeping the user in good shape.

AR Shopping

The latter function displays a video explaining a product, etc on the headset's screen with AR-based technologies. And the user can select a product and pay money with voice operations using the voice-recognition function of the phone. Also, NTT Docomo is considering adding a dial key that adjusts the number of AR images displayed on the headset's screen when the number becomes too large.

NTT Docomo has not announced when it will commercialize the headset. But other companies including Google Inc have been earnestly developing AR-based head-mounted displays (HMDs). So, it is possible that such a smartphone will be commercialized in the near future.