Paper Conference

Proceedings of Building Simulation 2021: 17th Conference of IBPSA


Evaluating different metrics for inter-model comparison of urban-scale building energy simulation time series

Hicham Johra 1, Michael Mans 2, Konstantin Filonenko 3, Ina De Jaeger 4, Dirk Saelens 4, Torben Tvedebrink 1
1 Aalborg University, Denmark
2 RWTH Aachen University, Germany
3 University of Southern Denmark, Denmark
4 KU Leuven, Belgium

Abstract: This article discusses different statistical methods to compare the predictions (in the form of time series) of urban-scale building energy models. It focuses on intermodel comparison without empirical measurements as a reference case. It is thus suggested to build a reference time series as a point-to-point mean average of all the models to be compared (ensemble methods). Different comparison metrics found in the literature are then reviewed and tested on simulation data from a modelling common exercise (DESTEST project). Most of those metrics behave similarly. Finally, a Python-based time series comparison tool is presented. It uses three simple metrics to compare building models: NMBE, CVRMSE of hourly data and CVRMSE of daily amplitudes.
Keywords: Numerical model validation, numerical model comparison, inter-model comparison, comparison metrics
Pages: 1556 - 1563