PhD-BPD Dissertation Defense Presentation: Guanzhou Ji

Title: Capturing, Editing, and Relighting Indoor Scenes from a Single Panorama

Name: Guangzhou Ji, PhD candidate in Building Performance & Diagnostics (BPD)

Date: Thursday, January 30, 2025
Time: 1:00-3:00pm ET
Location: CFA 102 & Zoom

Advisory Committee:

Azadeh Omidfar Sawyer (Co-Chair), PhD, LEED AP
Assistant Professor, School of Architecture
Carnegie Mellon University

Srinivasa Narasimhan (Co-Chair)
U.A. and Helen Whitaker Professor of Robotics, Robotics Institute
Carnegie Mellon University

Susan Finger, PhD
Professor, Civil and Environmental Engineering
Associate Dean, Integrative Design Arts & Technology (IDeATe)
Carnegie Mellon University

Sing Bing Kang, PhD

Abstract:
Virtual staging has been widely used to showcase indoor spaces. However, architectural design requires the integration of disparate software tools and significant manual input, which limits its scalability across various spaces. Recent computer vision applications have started using a single photograph to understand 3D scenes. This thesis introduces a novel inverse rendering pipeline that automatically transforms an indoor panorama into a new virtual panorama within its outdoor context. The first contribution is an indoor-outdoor HDR photography method. In each scenario, an indoor panorama is captured along with its corresponding outdoor image to understand indoor spatially-varying light. This effort has resulted in a new HDR dataset that includes 278 scenes with photometric calibration. The second contribution is a full inverse rendering pipeline. Conventional data-driven approaches face limitations in relighting indoor scenes. In this work, the 3D floor layout, material properties, and illumination conditions are directly estimated from the captured photographs, and a virtual model is constructed for realistic scene editing and relighting. The third contribution is a light-to-heat estimation method. Beyond the pixel, the absolute light level for virtual indoor scenes is computed to guide indoor activities. Meanwhile, transient heat simulation is implemented to estimate the heat generated by natural light on indoor surfaces. This technique is applied to various indoor scenes and provides applications such as furniture removal and insertion, interior design, and virtual home tours, which promote sustainable building design and real estate development. The project webpage for this work can be seen here. 

The thesis document can be accessed here.