Computational Materials and Digital Fabrication
Designing mathematical models that accurately represent and predict physical properties of real world materials is a central component in computer graphics as well as mechanical engineering and materials science. The problem is challenging because real world materials exhibit extraordinary variety and complexity. We have approached this problem by constructing acquisition devices to measure properties of real world materials and then deriving accurate data-driven models from these measurements. In particular, we have developed data-driven representations for texture, reflectance, spatially-varying reflectance, time-varying reflectance, subsurface scattering, and elastic material behavior.
We also focus on the reverse process – conversion from virtual to physical materials and objects using different fabrication methods. Our research goes beyond traditional, single material 3D printing by providing tools for fabricating complex, multi-material objects with desired visual and physical material properties. We have been developing a complete process and software/hardware framework that allows moving from abstract computer models to their physical counterparts efficiently and accurately. In the process, we have been addressing the following fundamental challenges: (1) developing representations and corresponding user-interfaces for designing complex, multi-material objects; (2) accurate and efficient simulation methods that can interactively predict properties and behavior of multi-material designs without physically generating it; (3) scalable and efficient architectures that convert multi-material models to inputs for 3D printers; (4) designing modular, high-resolution 3D printers that allow manufacturing multi-material composites made from a wide range of materials.
Computational Photography and Displays
The emerging field of computational photography and displays adds general computation capabilities and generalized optics to digital cameras and displays in order to obtain a superior imaging, viewing, and interaction. The main challenge is how to combine the hardware, the associated algorithms, and representations for images and video. In this area, we have been developing application-specific systems that demonstrate exceptional capabilities by blending custom hardware and novel algorithms. In particular, we have been working on three-dimensional TV, which has a significant potential impact and is expected to be the next big step in digital communications. We have proposed the first complete 3D TV system that allows for scalable real-time acquisition, transmission, and 3D display of dynamic scenes. We have been working on the fundamental algorithms for 3D TV – antialiasing, stereoscopic perception, and multi-view expansion. In the field of computational photography, we have designed imaging systems and associated algorithms for completely automated scene segmentation (e.g., alpha matting) and systems for changing aperture and focus in post-production.
Virtual Humans and Robotics
One of the most difficult computer graphics challenges is creating digital humans that are indistinguishable from real ones. This process has applications in movies, games, medicine, cosmetics, computer vision, biometrics, and virtual reality. The problem is challenging because we are incredibly sensitive to the subtleties of human appearance and motion. In order to solve the problem, we have been appling data-driven methods to capture and analyze human appearance, shape, motion, and contact with external forces. All these components are essential for creating realistic virtual humans. We have developed algorithms for computing image-based and polyhedral visual hulls to capture human shape and appearance in real-time. We have improved these systems by using high-quality templates or multi-view normal maps. In order to capture motions outside of a special studio, we have designed wearable systems that combine miniature ultrasonic and inertial sensors. In parallel to investigating full-body capture methods, we have been exploring methods for face acquisition.
We have been applying the developed models to recreate more realistic robots and animatronic characters.