Building Foundation Models – Potentials, Challenges and Research Directions for Using LLM and LVM in AEC

Abstract: Foundation Models like Large Language Models (LLM) and Large Vision Models (LVM) show huge potential in automating processes in building design, construction, and operation. However, there are concerns about their capability to create coherent, usable, sustainable, and safe architecture. This paper analyzes the challenges and potentials of LLM and LVM in the AEC sector. We discuss a collection of relevant AEC modalities like CAD models, semantic graphs, and time series and how they could integrate into the current landscape of models. From the analysis we derive research directions toward the development of domain specific “Building Foundation Models”.