EMPAC aims to create an environment of fertile creation, cross-pollination, and intellectual stimulation. Visiting scholars and researchers will participate in the formation of an intellectual community in scientific and engineering disciplines that may also engage perceptual and artistic knowledge and practice. EMPAC will also be a platform for research activities in areas such as augmented reality, virtual reality, scientific visualization, audification, haptics, human/machine interfaces and interaction, auralization, and multi-modal modeling in large-scale, fully media-integrated environments.

Photo: MANTA—Part of Smart Geometry 2013 in Studio 1—Goodman.


The Sound of EMPAC
Zackery Belanger

This essay is a report on the acoustic environment of the Experimental Media and Performing Arts Center, as told by a member of the design team who spent four years occupying and experiencing the building subsequent to its opening in 2008. EMPAC's materials, spaces, and profound silence are described for those who are interested in the design and creative use of spaces for sound.

Room Acoustics and Background Noise at EMPAC
Zackery Belanger

The Curtis R. Priem Experimental Media and Performing Arts Center (EMPAC) at Rensselaer Polytechnic Institute is among the highest calibre performing arts facilities in the world. All stages of its design and construction were executed with the intent of high performance in room acoustics, background noise, and sound isolation. Each of the four primary venues at EMPAC - the Concert Hall, Theater, Goodman Studio 1, and Studio 2 were designed with layout, volume, distribution of surfaces, and variable acoustic systems that allow them to comprise a large continuum of acoustic possibility.

This report contains room acoustic and background noise measurements performed in 2011. Its purpose is to inform artists and researchers who will use EMPAC for their work. For room acoustics the usual frequency range of testing was extended to higher and lower frequencies using a supplemental high-frequency omni source and low-frequency subwoofer, respectively. For background noise a special low-noise microphone was used. An equipment list is available at the end of this report.

The background noise of all spaces at EMPAC are very low, and exceptionally low in the four main venues. Additionally, the reverberation times in the Concert Hall and the two studios can be adjusted in many ways, as the detailed measurements in the report show.

Download Report »

Wave Field Synthesis
Todd Vos

The EMPAC Wave Field Synthesis Array is a 496-channel discrete loudspeaker array designed to take advantage of the added sound-stage dimensionality made available through the use of wave field synthesis (WFS). EMPAC embarked on the design and construction of this high-density loudspeaker array with the goal of increasing the high-frequency resolution of current, traditional, horizontal WFS systems while creating a modular, easily deployable, and accessible solution for research, composition, and presentation.

The Campfire
Eric Ameres, Gordon Clement

Human-computer interaction has focused predominantly on ways in which a single user interacts with a single computer, yet we live in a world of real-time one-to-one, one-to-many, and many-to-many relationships. As not only computers, but also the operations and problems that they are tasked with become more complex and interconnected, multi-human, multi-computer real-time interactions need to be integrated into physical and virtual interface design.

The Campfire is a novel multi-user, collaborative, immersive computing interface. Unlike its HCI predecessors, rather than an environment or a window into an application, The Campfire is itself a spatial object that forms a focal point for presentation, data-exploration, and collaboration. It is an immersive interface that achieves its immersion by focusing individual and group attention rather than overwhelming the user.

Physically, The Campfire is a desk-height panoramic screen and floor projection that users gather around and look into. While users engage with the system, they do not lose sight of one another. This is a very important difference between The Campfire and virtually all other computing interfaces. Although it occupies space between users, it does not occlude the view among and the natural discourse between them. There are no artificial or even virtual barriers between users. They can see one another and develop shared focus and attention in ways impossible with other interfaces.

Funded Summer Research Project in Scientific Data Exploration
Jonas Braasch, Johannes Goebel

A summer research project in Scientific Data Exploration will be hosted by the Center for Cognition, Communication and Culture (CCC), the Experimental Media and Performing Arts Center (EMPAC), and the Institute for Data Exploration and Analysis (IDEA) with support from the Office for Research. For this program we are seeking applications for funded summer positions from undergraduate and graduate students who are interested to participate.

The goal of the summer program is to explore new ways of visualizing, sonifying, and interacting with big data in large-scale virtual reality systems. We will select 3 to 4 projects to work on with groups of 3-4 students each, including a current EMPAC/CCI project on visualizing buffer exchange processes in Rensselaer’s IBM super computer for debugging purposes. The group will work with the visualization software Field for data visualization and Max/MSP for data sonification. We are looking for applicants with a background in programming, design, electronic music, and computer/human user interfaces for this interdisciplinary project.

Participating students will receive a summer stipend at institute standard rates for undergraduate and graduate students. Regular group meetings and a closing demonstration will be part of the program.

The summer program will begin in May 19 through August 15, 2014 and kick off with a workshop in the programming language and data visualization software Field led by Field-inventor Marc Downie. The summer program will take place at the CCC and EMPAC. This summer research project is a continuation of data visualization research initialized by EMPAC, for which an NSF grant was received by the three groups to lay the foundation for data visualization. In the context of IDEA, this summer program will begin to create a user group among students that is focused on applications to navigate big-data exploration. It will be the first program for Rensselaer’s new Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab). The CRAIVE-Lab is a large-scale virtual environment with seamless 14-ft-high panorama projection around a floor area of 40x32 sq. ft. and a 128-channel loudspeaker system for data sonification.

Participants are expected to be available for the introductory Field workshop May 19–23 and additionally 11 weeks attendance within the period May 26 through August 15. Interested students should apply to Jonas Braasch ( by April 20, 2014. Applications a brief statement of interests as well as the student’s resume (including the student’s name and department, and a list of skill sets).

Decisions will be made by April 24, 2014, based on both merit and relevance to the proposed research. Student applicants will be informed via email.

CCC – Center for Cognition, Communication, and Culture
IDEA – Institute for Data Exploration and Applications
EMPAC – The Curtis R. Priem Experimental Media and Performing Arts Center

The Next Acoustic Architecture
Zackery Belanger, Johannes Goebel

Innovations in acoustic design are sometimes hidden, misunderstood, or lost entirely once construction ends and design teams move on. Zackery Belanger worked for acoustician Kirkegaard Associates from 2002 to 2010, witnessing the EMPAC project through small design roles and large analysis processes. During this time he identified a few key innovations in the design of the Concert Hall and became convinced of their potential to advance the field.

For this research residency, Belanger will utilize the Concert Hall, numerical simulations, and established architectural design software to explore one possible future for architectural acoustics. In this future, our understanding of the relationships between enclosure geometry and sound will be an integral component of the design process. Surfaces of absorption, diffusion, transmission, and reflection will no longer occupy discrete categories, but instead lie on a continuous spectrum of geometric possibility, and the traditional use of the term “acoustic treatment” will become archaic.

A Robust Distributed Intelligent System for Telematic Applications
Jonas Braasch, Pauline Oliveros, Doug Van Nort, Kyle McDonald, Luke Noonan

Complex communication for co-located performers within telepresence applications across networks is still impaired compared to performances that take place in one physical location. This impairment must be significantly reduced to allow the broader community to participate in complex communication scenarios. To achieve this goal, an avatar in the form of a musical conductor with forms of artificial intelligence will coordinate between co-located musicians. Improvised Contemporary Live Music of a larger ensemble, serving as a test bed, is arguably one of the most complex scenarios one could think of, because it requires engaged communication between individuals within a multiple-source sound field that also has to be considered as a whole. The results are expected to inspire solutions for other communication tasks. The avatar system will actively coordinate co-located improvisation ensembles in a creative way. To achieve this goal, Computational Auditory Scene Analysis (CASA) systems, to allow robust feature recognition, and Evolutionary algorithms, for the creative component, will be combined, to form the first model of its kind. The research results are expected to be significant by themselves and are not bound to telematic applications. With regard to the latter, the proposed system will have a clear advantage over a human musician/conductor, while intelligent algorithms are clearly lacking behind human performance in most other applications, especially when it comes to creativity.

Immersive Architectural Daylighting Design Experience
Yu Sheng, Theodore C. Yapo, Christopher Young, Barbara Cutler

We present an application of interactive global illumination and spatially augmented reality to architectural daylight modeling that allows designers to explore alternative designs and new technologies for improving the sustainability of their buildings. Images of a model in the real world, captured by a camera above the scene, are processed to construct a virtual 3D model. To achieve interactive rendering rates, we use a hybrid rendering technique, leveraging radiosity to simulate the inter-reflectance between diffuse patches and shadow volumes to generate per-pixel direct illumination. The rendered images are then projected on the real model by four calibrated projectors to help users study the daylighting illumination. The virtual heliodon is a physical design environment in which multiple designers, a designer and a client, or a teacher and students can gather to experience animated visualizations of the natural illumination within a proposed design by controlling the time of day, season, and climate. Furthermore, participants may interactively redesign the geometry and materials of the space by manipulating physical design elements and see the updated lighting simulation.

Our daylighting design tool is implemented in 3 systems. First, it is a desktop software daylighting tool for widespread use in architectural design. Second, it is a small-scale, low-cost virtual heliodon, appropriate to be installed in the conference room of architectural design practice for regular use. Third, it is a full-scale, immersive design environment in which participants can gather to experience animated visualizations of the natural illumination within a proposed design. Our project will also leverage the computational resources at the new Computational Center for Nanotechnology Innovations (CCNI).

Studio 2 Visual Tracking System
Rich Radke

We will design and install a semi-permanent overhead visual tracking system for EMPAC’s Studio 2, enabling real-time multi-person tracking for both artistic installations and research applications. This information would be available to any end-users (artists or researchers) for the purposes of designing interactive visualizations, auralizations, or artistic installations. We believe that a general multi-person tracker would be a valuable infrastructure addition to EMPAC that would save time for users of the space and promote interaction between computer vision researchers and artists.

Usable Content in an Interactive World

In a project that can affect experimental media as well as a wide variety of other fields, Rensselaer’s Department of Language, Literature, and Communication has been awarded a major research grant from the Society for Technical Communication. In this project, Rensselaer will develop standards for analyzing, designing, and testing the usability of tech-mediated communications — ranging from graphics, to Web sites and Web gallery interfaces, to wikis and distance education environments.

The team is studying “post-documents,” the electronic interfaces that have become increasingly visible as we move toward a paperless society, relying less heavily on traditional, physical documents for informative and educational purposes. The first goal is to identify what factors make post-documents usable. Following their findings, the group will produce and test a “Post-Document Toolkit,” made up of a set of broad characteristics that make electronic content usable and an associated set of metrics for measuring post-document usability.

Accurate Recordings of Sounds From Multiple Source

Jonas Braasch, assistant professor of architecture, and his group are developing a new recording system that correctly tracks and reproduces multiple sound-sources and incorporates them into 3-D sound projection systems. This new approach has the potential to significantly improve and refine current sound systems, both for audio and video experiences. Braasch, who received this year’s Lothar Cremer Award from the German Acoustical Society, the highest scientific honor given by the society, investigates various aspects of spatial hearing in complex scenarios, among other subjects.

Adding the Sense of Touch to Virtual Environments

Suvranu De, assistant professor of mechanical, aerospace, and nuclear engineering, leads a team that is combining the sense of touch with 3-D computer models of organs to create a new approach to training surgeons, a virtual simulator that allows surgeons to touch, feel, and manipulate computer-generated 3-D tissues and organs. De and his collaborators are now turning that expertise in computer haptics (sense of touch) and virtual environments to a new project that could profoundly impact experimental arts presentations.

They are developing the foundations for a robust and reliable shared virtual environment in which interactive haptic feedback is added to existing visual and auditory feedback. The project will concentrate on developing methods to overcome current network limitations such as latency and packet loss and then study how humans use the ability to communicate touch over long distances when interacting with other humans or machines.

New Technology for World-Class Acoustics

Ning Xiang, chair of Rensselaer’s graduate program in architectural acoustics, is using advanced signal theory concepts to develop a multiple sound-source measurement system to better predict the acoustic characteristics of complex rooms such as concert halls. The 1,200-seat main EMPAC performance hall will be the first room to be so characterized. Xiang’s “multiple acoustics measurement technique” greatly streamlines the process of measuring acoustics, gathering the acoustical information of any given space in minutes.

His method also can be used to quickly identify and correct acoustic mistakes within an existing performance space or to verify design goals in new building construction. Xiang’s approach enables all speakers in an area to emit a sound to designated receivers simultaneously. To interpret the data of each sound as it travels from speaker to receiver, Xiang’s group has created a new deconvolution algorithm that is loaded onto a computer with data acquisition and digital-to-analog capabilities. The system measures and graphs how well sounds move within the space.

Modeling the Transmissive Ceiling Canopy
Paul Henderson, Ioana Pieleanu, Dr. Rendell Torres, Dr. Mendel Kleiner, Johannes Goebel, Kirkegaard Associates

The project team was commissioned to construct a detailed computational model of the EMPAC concert hall ceiling canopy design as specified by acoustician Kirkegaard Associates. Both the acousticians and campus officials determined that a study should be performed using auralization techniques to both aurally verify the feasibility of the design and to select a proper fabric material for the canopy.