May 2007
Socializing Cyberinfrastructure: Networking the Humanities, Arts, and Social Sciences
Ruzena Bajcsy University of California, Berkeley
Klara Nahrstedt, University of Illinois, Urbana-Champaign
Lisa Wymore, University of California, Berkeley
Katherine Mezur, Mills College

3. Selected TI Challenges and Solutions

To enable real-time interactive dance choreography in tele-immersive environments, multiple challenges need to be overcome. Some of the challenges have been solved, others are still in progress.

One of the major challenges is to accomplish 3D real-time video reconstruction. The 3D reconstruction algorithms are still lacking behind the 2D video capabilities in real-time performance due to their algorithmic complexity as well as inefficient implementation. A second major challenge is the robust calibration of 3D cameras under different light conditions, mobility and other environmental changes that happen in TI sites.

The current solutions in the 3D real-time reconstruction space currently rely on an image-based trinocular stereo algorithm7 to calculate the depth information for each pixel. The output from one 3D camera cluster corresponds to one stream of 3D frames containing both color and depth information. All cameras are synchronized via hotwires to take shots simultaneously. They are calibrated with a self-calibration algorithm by Svoboda et al.8 Figure 3 shows some of the challenges and solutions/results in the 3D vision space.

Figure 3

Figure 3. Challenges and current solutions (results) with respect to calibration and 3D real-time reconstruction.

One of the major challenges in TI networking is the bandwidth management and appropriate semantic networking protocols to provide Quality of Service (QoS) on top of the best effort heterogeneous Gigabit Ethernet/Internet 2 networks.

The current networking solution relies on our semantic-aware, session data protocol within the service middleware layer that runs on top of the TCP/IP protocol stack. It executes 3D video stream compression as well as end-to-end multi-tier streaming using bandwidth estimation, context-aware bandwidth allocation and rate adaptation. The service middleware layer provides the control plane functionalities for the bandwidth management, such as the multi-stream coordination and adaptation algorithms, depending on the end-to-end Internet 2 bandwidth availability and user view selections. Furthermore, the control functions include display setup with selection of scene views and screen layouts at the viewer site. Figure 4 shows some of the networking challenges and solutions/results in the semantic networking space.

Figure 4

Figure 4. End-to-End throughput result using our TI session layer protocol.

Pages: 1 2 3 4 5

Reference this article
Bajcsy, R., Nahrstedt, K., Wymore, L., Mezur, K. "Tele-Immersive Environments for Geographically Distributed Interaction and Communication," CTWatch Quarterly, Volume 3, Number 2, May 2007. http://www.ctwatch.org/quarterly/articles/2007/05/tele-immersive-environments/

Any opinions expressed on this site belong to their respective authors and are not necessarily shared by the sponsoring institutions or the National Science Foundation (NSF).

Any trademarks or trade names, registered or otherwise, that appear on this site are the property of their respective owners and, unless noted, do not represent endorsement by the editors, publishers, sponsoring institutions, the National Science Foundation, or any other member of the CTWatch team.

No guarantee is granted by CTWatch that information appearing in articles published by the Quarterly or appearing in the Blog is complete or accurate. Information on this site is not intended for commercial purposes.