Researchers at the University of Washington have developed a new technique for creating 3d cg animation from a single photo. They’re calling it “ Photo Wake-Up ” and the demo video above gives a clear idea of what can be accomplished with the technique. The Photo Wake-Up research was supported by Facebook, Google, Huawei, and a Reality Lab Huawei Fellowship.
The technique is demonstrated by using a variety of single-image inputs, including photographs, realistic illustrations, cartoon drawings, and abstracted human forms. The figures are animated in various ways, such as walking toward the screen, running, sitting, and jumping.
In a paper that details the process (download PDF ), the researchers —
Chung-Yi Weng, Brian Curless, and Ira Kemelmacher-Shlizerman — explain that a lot of the technical development required to do this already exists. Those existing techniques include segmenting a person from an image, 2d skeleton estimating, and fitting a morphable, posable 3d model into the still image. The UofW researchers have built on that previous research by attempting to have the animated figures conform more closely to the 2d silhouettes and to create more natural looking characters.
The key technical advance of Photo Wake-Up is creating an animatable 3d model that matches the silhouette of a single photo and handles self-occlusion. “Rather than deforming the 3d mesh from the first stage – a difficult problem for intricate regions such as fingers and for scenarios like abstract artwork – we map the problem to 2d, perform a silhouette-aligning warp in image space, and then lift the result back into 3d,” the researchers explained. “This 2d warping approach works well for handling complex silhouettes. Further, by introducing label maps that delineate the boundaries between body parts, we extend our method to handle certain self-occlusions.”
The First Attempt at a VR Experience – The Sensorama. In the 1950s, a cinematographer by the name of Morton Heilig came up with a unique concept he later developed, known as the Sensorama. Featuring an arcade-style theater cabinet, the sensorama was aimed at stimulating a person’s senses. It featured a stereoscopic 3D display, fans and smell generators, stereo speakers, as well as a vibrating chair. The idea of the Sensorama was to fully immerse a person into a film-like experience. Heilig also went on to create as much as six short movies for his device.