New Technology Interactively Renders 3D Space in Real Time

May 27, 2011
Tetsuo Nozawa, Nikkei Electronics
A demonstration of "Interactive IP Rendering"
A demonstration of "Interactive IP Rendering"
[Click to enlarge image]

NHK Science & Technology Research Laboratories (STRL) demonstrated a technology that renders real-looking 3D animation video in real time in accordance with user operation.

The demonstration of the technology, "Interactive IP Rendering," took place at Open House 2011, which opens for the public from May 26 to 29, 2011.

Specifically, when a display on which 3D video based on the IP (integral photography) method was being displayed was tilted sideways, characters in the display slid depending on the tilt angle. Because the characters were drawn in 3D, it looked as if they were living in the display.

"At first, we were planning to let characters come out of the display when it is tilted forward," NHK STRL said. "But, because it was little difficult to see, it ended up in this version."

The IP method here means the technology that is the base of the "Integral 3D TV," which NHK STRL is now developing. While existing 3D TVs display two types of images for the right and left eyes, the IP method simultaneously displays images seen from all directions in principle.

3D video of the IP method is different from normal 3D video having multiple viewpoints. The IP method is also called ray reproduction method, and it emits light beams in a way that a real object seems to exist. Therefore, if the IP method can be realized in an ideal way, the "contradiction between adjustment and congestion" should not occur.

The contradiction means this: While the right and left eyes decide the angle of the line of sight in accordance with an image popping out of a 3D display or an image deep inside the display, the eyes are focused on the surface of the display. The inconsistency of the two depths confuses brain, causing, for example, "3D sickness." In the ideal IP method, this problem does not occur because the right and left eyes are focused not on the surface of a display but on an image.

Video data of the IP method can be created by shooting a real object with a camcorder having a lens array. However, to create computer graphics or animation video, this method cannot be used because there is no real object to shoot. Therefore, it is necessary to reconstruct images seen from various angles in a six-dimensional (normally five- to nine-dimensional) data space called "ray space."

This time, NHK STRL developed a technology to create a ray space of animation video and a technology to create it in real time in accordance with user operation, realizing the demonstration.

動画 Characters in a 3D space slide depending on tilt angle

You need Windows Media Player to play this video
Click the play button