Research
With an impact reaching across the entire spectrum of the Rensselaer experience, the research focus at EMPAC will provide students, researchers, artists, and audiences with opportunities for leading-edge science and engineering, performance technologies and the arts across disciplines. The spaces and infrastructure are equally enabled for research that links sensory perception and human expression with computational processes.
Scholar/Researcher in Residence Program
EMPAC aims to create an environment of fertile creation, cross- pollination, and intellectual stimulation. Visiting scholars and researchers will participate in the formation of an intellectual community in scientific and engineering disciplines that may also engage perceptual and artistic knowledge and practice. EMPAC will also be a platform for research activities in areas such as augmented reality, virtual reality, scientific visualization, audification, haptics, human/machine interfaces and interaction, auralization, and multi-modal modeling in large-scale, fully media-integrated environments.
Selected Research Projects:
A Robust Distributed Intelligent System for Telematic Applications
2008-2010
Jonas Braasch, Pauline Oliveros (Co-PIs); Doug Van Nort, Research Specialist; Luke Noonan, undergraduate student assistant; Kyle McDonald, graduate student assistant
Complex communication for co-located performers within telepresence applications across networks is still impaired compared to performances that take place in one physical location. This impairment must be significantly reduced to allow the broader community to participate in complex communication scenarios. To achieve this goal, an avatar in the form of a musical conductor with forms of artificial intelligence will coordinate between co-located musicians. Improvised Contemporary Live Music of a larger ensemble, serving as a test bed, is arguably one of the most complex scenarios one could think of, because it requires engaged communication between individuals within a multiple-source sound field that also has to be considered as a whole. The results are expected to inspire solutions for other communication tasks. The avatar system will actively coordinate co-located improvisation ensembles in a creative way. To achieve this goal, Computational Auditory Scene Analysis (CASA) systems, to allow robust feature recognition, and Evolutionary algorithms, for the creative component, will be combined, to form the first model of its kind. The research results are expected to be significant by themselves and are not bound to telematic applications. With regard to the latter, the proposed system will have a clear advantage over a human musician/conductor, while intelligent algorithms are clearly lacking behind human performance in most other applications, especially when it comes to creativity.
Immersive Architectural Daylighting Design Experience
2009
Yu Sheng, Theodore C. Yapo, Christopher Young, and Barbara Cutler
We present an application of interactive global illumination and spatially augmented reality to architectural daylight modeling that allows designers to explore alternative designs and new technologies for improving the sustainability of their buildings. Images of a model in the real world, captured by a camera above the scene, are processed to construct a virtual 3D model. To achieve interactive rendering rates, we use a hybrid rendering technique, leveraging radiosity to simulate the inter-reflectance between diffuse patches and shadow volumes to generate per-pixel direct illumination. The rendered images are then projected on the real model by four calibrated projectors to help users study the daylighting illumination. The virtual heliodon is a physical design environment in which multiple designers, a designer and a client, or a teacher and students can gather to experience animated visualizations of the natural illumination within a proposed design by controlling the time of day, season, and climate. Furthermore, participants may interactively redesign the geometry and materials of the space by manipulating physical design elements and see the updated lighting simulation.
Our daylighting design tool is implemented in 3 systems. First, it is a desktop software daylighting tool for widespread use in architectural design. Second, it is a small-scale, low-cost virtual heliodon, appropriate to be installed in the conference room of architectural design practice for regular use. Third, it is a full-scale, immersive design environment in which participants can gather to experience animated visualizations of the natural illumination within a proposed design. Our project will also leverage the computational resources at the new Computational Center for Nanotechnology Innovations (CCNI).
Studio 2 Visual Tracking System
2009
Rich Radke, Dept. of ECSE, Rensselaer Polytechnic Institute
We will design and install a semi-permanent overhead visual tracking system for EMPAC’s Studio 2, enabling real-time multi-person tracking for both artistic installations and research applications. This information would be available to any end-users (artists or researchers) for the purposes of designing interactive visualizations, auralizations, or artistic installations. We believe that a general multi-person tracker would be a valuable infrastructure addition to EMPAC that would save time for users of the space and promote interaction between computer vision researchers and artists.
Modeling the Transmissive Ceiling Canopy
2004
Principal Researchers: Paul Henderson, Ioana Pieleanu Faculty Supervisor: Dr. Rendell Torres, Contributing Faculty: Dr. Mendel Kleiner, Johannes Goebel, Contributor: Kirkegaard Associates
The project team was commissioned to construct a detailed computational model of the EMPAC concert hall ceiling canopy design as specified by acoustician Kirkegaard Associates. Both the acousticians and campus officials determined that a study should be performed using auralization techniques to both aurally verify the feasibility of the design and to select a proper fabric material for the canopy.
Links:
Scholar/Researcher in Residence Program @ EMPAC
Research @ Rensselaer: Experimental Media and the Arts