Conference Paper

Proceedings of eSim 2024: 13th Conference of IBPSA-Canada

     

A multi-armed bandit reinforcement learning method for occupant thermal preference learning

Hussein Elehwany, Mohamed Ouf, Burak Gunay, Nunzio Cotrufo, Jean-Simon Venne, Junfeng Wen
Carleton University

Abstract: Many commercial buildings depend on fixed thermostat setpoints, which increases energy consumption and may compromise occupants’ comfort. In recent years, many occupant-centric control (OCC) methods have been introduced that can learn occupants’ thermal preferences and adjust thermostat setpoints accordingly. This study uses reinforcement learning (RL) to develop a smart learning thermostat. The developed algorithm utilizes multi-armed bandit (MAB) RL method leveraging feedback from unsolicited occupant interactions with thermostats and energy use to derive the optimal setpoints. The algorithm aims to balance between the goals of saving energy and enhancing occupant comfort. An occupant behaviour model is developed to simulate the stochastic behaviour of occupants with thermostats in an EnergyPlus simulation environment. The integration of the MAB algorithm with the simulation model was achieved using the Python API in EnergyPlus. The MAB algorithm has a simpler problem formulation than other RL methods as it does not require state variables. This enables the algorithm to learn faster. The MAB algorithm also requires less computational power as it does not require the training of any artificial neural networks (ANN).
Keywords: reinforcement learning, preference learning, occupant-centric controls, smart thermostats, multi-armed bandit
Paper: