TV Observes Viewers, Checks Their Interests in Programs

May 27, 2011
Tetsuo Nozawa, Nikkei Electronics
A demonstration of the "UTAN (user technology assisted navigation) TV viewing interface." Under the TV, there is a camera that can take pictures of viewers. And, on the table, there is a tablet PC that displays a variety of information.
A demonstration of the "UTAN (user technology assisted navigation) TV viewing interface." Under the TV, there is a camera that can take pictures of viewers. And, on the table, there is a tablet PC that displays a variety of information.
[Click to enlarge image]
The conditions of the viewers are estimated by combining data on their faces, facial expressions and motions.
The conditions of the viewers are estimated by combining data on their faces, facial expressions and motions.
[Click to enlarge image]

NHK Science & Technology Research Laboratories (STRL) announced the "UTAN (user technology assisted navigation) TV viewing interface" at Open House 2011, which opens for the public from May 26 to 29, 2011.

With the interface, a TV observes its viewers and provides information related to the TV program in accordance with the way they are watching it.

The UTAN takes a picture of a viewer with a camera mounted on a TV and estimates the viewer's degrees of interest, concentration, etc from circumstances such as who is watching the TV with how much interest. Then, information suited for the circumstances is sent to a tablet PC, etc near the viewer as, for example, "recommended information." When there are multiple viewers, it is possible to determine the interest, etc of each of them.

This estimation is conducted based on the viewer's face, facial expression and motions.

"If the viewer's expression does not change for a certain period of time even though he keeps watching TV, he is estimated to be concentrating on the program," NHK STRL said.

At this point, the recognition of facial expression is conducted by measuring differences from its normal state. And it does not determine whether the viewer is smiling or getting angry though it is technically possible, NHK STRL said.