PhD-BPD Dissertation Defense Presentation: Tiancheng Zhao
Title: An Intelligence in Residence: Bringing Sensing, Reasoning, and Conversation Into the Built Environment Through an IEQ Robot
Name: Tiancheng Zhao, Ph.D. candidate in Building Performance and Diagnostics (PhD-BPD)
Date: Friday, October 9, 2026
Time: 9:00-11:00am ET
Location: Cohon University Center, Room: Class of 1987 (2nd Floor) & Zoom
Dissertation Committee:
Erica Cochran Hameen, Ph.D. (Chair)
Associate Professor
School of Architecture
Carnegie Mellon University
Bhiksha Ramakrishnan, Ph.D.
Professor
Language Technologies Institute
Carnegie Mellon University
Deva Kannan Ramanan, Ph.D.
Professor
Robotics Institute
Carnegie Mellon University
Abstract:
A room can appear comfortable in its measurements yet feel uncomfortable to the people using it. Making sense of that difference requires bringing together environmental readings, visual observations, and occupants' experiences at specific places and times. The challenge is to turn these different forms of evidence into guidance that people can act on.
This thesis introduces a unified, multimodal AI framework that fundamentally rethinks how Indoor Environmental Quality (IEQ) is measured, analyzed, and improved within buildings. At its core, the system integrates three traditionally separate modalities — environmental sensor streams, visual context from onboard cameras, and natural language feedback from occupants — into a coherent representation of occupant comfort and indoor environmental performance. By leveraging advancements in Large Language Models (LLMs) and Vision Foundation Models (VFMs), the system elevates IEQ assessment from a labor-intensive, episodic activity to a continuous, autonomous, and expert-level analytic process.
This dissertation provides four main deliverables: 1) A synthetic IEQ supervised fine-tuning (SFT) dataset that includes synthetic IEQ inputs and reference answers. These inputs and responses are utilized to form input-response pairs within the dataset. 2) Utilization of the synthetic IEQ dataset to fine-tune open source Multimodal Large Language Models (MLLMs) to generate recommendations based on the analysis of photograph descriptions, sensor readings, and occupant questions. 3) Verbal question-and-response exchanges between occupants and the robot — audio to text, and then text to audio — connecting occupants to AI-generated recommendations for building operation, maintenance, and retrofit decisions that affect IEQ. 4) Integration of environmental sensing, visual observations, and robot localization to examine how IEQ conditions vary across a building and over time. The resulting smart multi-sensor IEQ platform displays IEQ metrics in a unified, interactive dashboard, allowing users to inspect readings and compare conditions across rooms and repeated measurement runs.
The impact of this thesis is significant. It enhances traditional IEQ monitoring performed by human technicians and expert consultants by adding an autonomous agent in the form of a robot, capable of high-resolution sensing, rich contextual interpretation, and personalized recommendation generation. The robot dramatically increases spatial and temporal coverage, reduces operational cost associated with Post Occupancy Evaluation (POE) assessments, and democratizes expert-level IEQ guidance. More broadly, the work demonstrates how robotics, LLMs and VFMs can be integrated into a coherent computational architecture that bridges the gap between physical sensing, semantic context, and human comfort. This framework lays the foundation for intelligent indoor environments that proactively maintain health, comfort, and sustainability in ways that were previously impractical or impossible through manual assessment alone.