Research

MANTA

With an impact reaching across the entire spectrum of the Rensselaer experience, the research focus at EMPAC provides students, researchers, artists, and audiences with opportunities for leading-edge science and engineering, performance technologies and the arts across disciplines. The spaces and infrastructure are equally enabled for research that links sensory perception and human expression with computational processes.

Scholar/Researcher-in-Residence Program

EMPAC aims to create an environment of fertile creation, cross-pollination, and intellectual stimulation. Visiting scholars and researchers will participate in the formation of an intellectual community in scientific and engineering disciplines that may also engage perceptual and artistic knowledge and practice. EMPAC will also be a platform for research activities in areas such as augmented reality, virtual reality, scientific visualization, audification, haptics, human/machine interfaces and interaction, auralization, and multi-modal modeling in large-scale, fully media-integrated environments.

Photo: MANTA—Part of Smart Geometry 2013 in Studio 1—Goodman.
Funded Summer Research Project in Scientific Data Exploration
2014
Jonas Braasch, Johannes Goebel

Funded Summer Research Project in Scientific Data Exploration for undergraduate and graduate students

A summer research project in Scientific Data Exploration will be hosted by the Center for Cognition, Communication and Culture (CCC), the Experimental Media and Performing Arts Center (EMPAC), and the Institute for Data Exploration and Analysis (IDEA) with support from the Office for Research. For this program we are seeking applications for funded summer positions from undergraduate and graduate students who are interested to participate.

The goal of the summer program is to explore new ways of visualizing, sonifying, and interacting with big data in large-scale virtual reality systems. We will select 3 to 4 projects to work on with groups of 3-4 students each, including a current EMPAC/CCI project on visualizing buffer exchange processes in Rensselaer’s IBM super computer for debugging purposes. The group will work with the visualization software Field for data visualization and Max/MSP for data sonification. We are looking for applicants with a background in programming, design, electronic music, and computer/human user interfaces for this interdisciplinary project.

Participating students will receive a summer stipend at institute standard rates for undergraduate and graduate students. Regular group meetings and a closing demonstration will be part of the program.

The summer program will begin in May 19 through August 15, 2014 and kick off with a workshop in the programming language and data visualization software Field led by Field-inventor Marc Downie. The summer program will take place at the CCC and EMPAC. This summer research project is a continuation of data visualization research initialized by EMPAC, for which an NSF grant was received by the three groups to lay the foundation for data visualization. In the context of IDEA, this summer program will begin to create a user group among students that is focused on applications to navigate big-data exploration. It will be the first program for Rensselaer’s new Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVE-Lab). The CRAIVE-Lab is a large-scale virtual environment with seamless 14-ft-high panorama projection around a floor area of 40x32 sq. ft. and a 128-channel loudspeaker system for data sonification.

Participants are expected to be available for the introductory Field workshop May 19–23 and additionally 11 weeks attendance within the period May 26 through August 15. Interested students should apply to Jonas Braasch (braasj@rpi.edu) by April 20, 2014. Applications a brief statement of interests as well as the student’s resume (including the student’s name and department, and a list of skill sets).

Decisions will be made by April 24, 2014, based on both merit and relevance to the proposed research. Student applicants will be informed via email.

CCC – Center for Cognition, Communication, and Culture
IDEA – Institute for Data Exploration and Applications
EMPAC – The Curtis R. Priem Experimental Media and Performing Arts Center

The Next Acoustic Architecture
2013
Zackery Belanger, Johannes Goebel

Innovations in acoustic design are sometimes hidden, misunderstood, or lost entirely once construction ends and design teams move on. Zackery Belanger worked for acoustician Kirkegaard Associates from 2002 to 2010, witnessing the EMPAC project through small design roles and large analysis processes. During this time he identified a few key innovations in the design of the Concert Hall and became convinced of their potential to advance the field.

For this research residency, Belanger will utilize the Concert Hall, numerical simulations, and established architectural design software to explore one possible future for architectural acoustics. In this future, our understanding of the relationships between enclosure geometry and sound will be an integral component of the design process. Surfaces of absorption, diffusion, transmission, and reflection will no longer occupy discrete categories, but instead lie on a continuous spectrum of geometric possibility, and the traditional use of the term “acoustic treatment” will become archaic.

A Robust Distributed Intelligent System for Telematic Applications
2012
Jonas Braasch, Pauline Oliveros, Doug Van Nort, Kyle McDonald, Luke Noonan

Complex communication for co-located performers within telepresence applications across networks is still impaired compared to performances that take place in one physical location. This impairment must be significantly reduced to allow the broader community to participate in complex communication scenarios. To achieve this goal, an avatar in the form of a musical conductor with forms of artificial intelligence will coordinate between co-located musicians. Improvised Contemporary Live Music of a larger ensemble, serving as a test bed, is arguably one of the most complex scenarios one could think of, because it requires engaged communication between individuals within a multiple-source sound field that also has to be considered as a whole. The results are expected to inspire solutions for other communication tasks. The avatar system will actively coordinate co-located improvisation ensembles in a creative way. To achieve this goal, Computational Auditory Scene Analysis (CASA) systems, to allow robust feature recognition, and Evolutionary algorithms, for the creative component, will be combined, to form the first model of its kind. The research results are expected to be significant by themselves and are not bound to telematic applications. With regard to the latter, the proposed system will have a clear advantage over a human musician/conductor, while intelligent algorithms are clearly lacking behind human performance in most other applications, especially when it comes to creativity.

Immersive Architectural Daylighting Design Experience
20092011
Yu Sheng, Theodore C. Yapo, Christopher Young, Barbara Cutler

We present an application of interactive global illumination and spatially augmented reality to architectural daylight modeling that allows designers to explore alternative designs and new technologies for improving the sustainability of their buildings. Images of a model in the real world, captured by a camera above the scene, are processed to construct a virtual 3D model. To achieve interactive rendering rates, we use a hybrid rendering technique, leveraging radiosity to simulate the inter-reflectance between diffuse patches and shadow volumes to generate per-pixel direct illumination. The rendered images are then projected on the real model by four calibrated projectors to help users study the daylighting illumination. The virtual heliodon is a physical design environment in which multiple designers, a designer and a client, or a teacher and students can gather to experience animated visualizations of the natural illumination within a proposed design by controlling the time of day, season, and climate. Furthermore, participants may interactively redesign the geometry and materials of the space by manipulating physical design elements and see the updated lighting simulation.

Our daylighting design tool is implemented in 3 systems. First, it is a desktop software daylighting tool for widespread use in architectural design. Second, it is a small-scale, low-cost virtual heliodon, appropriate to be installed in the conference room of architectural design practice for regular use. Third, it is a full-scale, immersive design environment in which participants can gather to experience animated visualizations of the natural illumination within a proposed design. Our project will also leverage the computational resources at the new Computational Center for Nanotechnology Innovations (CCNI).

Studio 2 Visual Tracking System
2009
Rich Radke

We will design and install a semi-permanent overhead visual tracking system for EMPAC’s Studio 2, enabling real-time multi-person tracking for both artistic installations and research applications. This information would be available to any end-users (artists or researchers) for the purposes of designing interactive visualizations, auralizations, or artistic installations. We believe that a general multi-person tracker would be a valuable infrastructure addition to EMPAC that would save time for users of the space and promote interaction between computer vision researchers and artists.

Usable Content in an Interactive World
2006

In a project that can affect experimental media as well as a wide variety of other fields, Rensselaer’s Department of Language, Literature, and Communication has been awarded a major research grant from the Society for Technical Communication. In this project, Rensselaer will develop standards for analyzing, designing, and testing the usability of tech-mediated communications — ranging from graphics, to Web sites and Web gallery interfaces, to wikis and distance education environments.

The team is studying “post-documents,” the electronic interfaces that have become increasingly visible as we move toward a paperless society, relying less heavily on traditional, physical documents for informative and educational purposes. The first goal is to identify what factors make post-documents usable. Following their findings, the group will produce and test a “Post-Document Toolkit,” made up of a set of broad characteristics that make electronic content usable and an associated set of metrics for measuring post-document usability.

Accurate Recordings of Sounds From Multiple Source
2006

Jonas Braasch, assistant professor of architecture, and his group are developing a new recording system that correctly tracks and reproduces multiple sound-sources and incorporates them into 3-D sound projection systems. This new approach has the potential to significantly improve and refine current sound systems, both for audio and video experiences. Braasch, who received this year’s Lothar Cremer Award from the German Acoustical Society, the highest scientific honor given by the society, investigates various aspects of spatial hearing in complex scenarios, among other subjects.

Adding the Sense of Touch to Virtual Environments
2006

Suvranu De, assistant professor of mechanical, aerospace, and nuclear engineering, leads a team that is combining the sense of touch with 3-D computer models of organs to create a new approach to training surgeons, a virtual simulator that allows surgeons to touch, feel, and manipulate computer-generated 3-D tissues and organs. De and his collaborators are now turning that expertise in computer haptics (sense of touch) and virtual environments to a new project that could profoundly impact experimental arts presentations.

They are developing the foundations for a robust and reliable shared virtual environment in which interactive haptic feedback is added to existing visual and auditory feedback. The project will concentrate on developing methods to overcome current network limitations such as latency and packet loss and then study how humans use the ability to communicate touch over long distances when interacting with other humans or machines.

New Technology for World-Class Acoustics
2006

Ning Xiang, chair of Rensselaer’s graduate program in architectural acoustics, is using advanced signal theory concepts to develop a multiple sound-source measurement system to better predict the acoustic characteristics of complex rooms such as concert halls. The 1,200-seat main EMPAC performance hall will be the first room to be so characterized. Xiang’s “multiple acoustics measurement technique” greatly streamlines the process of measuring acoustics, gathering the acoustical information of any given space in minutes.

His method also can be used to quickly identify and correct acoustic mistakes within an existing performance space or to verify design goals in new building construction. Xiang’s approach enables all speakers in an area to emit a sound to designated receivers simultaneously. To interpret the data of each sound as it travels from speaker to receiver, Xiang’s group has created a new deconvolution algorithm that is loaded onto a computer with data acquisition and digital-to-analog capabilities. The system measures and graphs how well sounds move within the space.

Modeling the Transmissive Ceiling Canopy
2004
Paul Henderson, Ioana Pieleanu, Dr. Rendell Torres, Dr. Mendel Kleiner, Johannes Goebel, Kirkegaard Associates

The project team was commissioned to construct a detailed computational model of the EMPAC concert hall ceiling canopy design as specified by acoustician Kirkegaard Associates. Both the acousticians and campus officials determined that a study should be performed using auralization techniques to both aurally verify the feasibility of the design and to select a proper fabric material for the canopy.