Reinforcement learning-based thermal comfort control for vehicle cabins

    Student thesis: Doctoral ThesisDoctor of Philosophy


    Vehicle Heating, Ventilation and Air Conditioning (HVAC) systems aim to ensure that passengers are thermally comfortable. However, thermal comfort is influenced by a large number of environmental variables and, furthermore, thermal preferences can vary greatly between individuals due to physiological, behavioural and cultural factors. This means that, generally, occupants need to adjust the nominally “comfortable” HVAC settings in order to achieve and maintain thermal comfort. This thesis established that, in order to develop efficient HVAC control algorithms, there is a need to i) sense a range of variables beyond air temperature, and ii) adopt learning-based techniques to take into account user preferences.

    This thesis proposes a novel reinforcement learning-based HVAC controller combined with virtual sensing to enable energy-efficient, comfort-oriented, high-level HVAC control.

    Towards this goal, the thesis first explores which of the thermal comfort models presented in the literature is the most suitable for real-time use in an HVAC system. The evaluation is based on data gathered from experimental trials with human subjects conducted over a wide range of conditions. Nils¬son’s equivalent temperature-based model is shown to provide the highest correlation scores with the subjective occupant comfort data. Furthermore, Nilsson’s model has the advantage of estimating local (not only overall) thermal sensation and requiring only two input parameters—the clothing index and equivalent temperature.

    Although equivalent temperature is shown to be necessary for estimating thermal comfort, it cannot feasibly be measured in real-time in a manufactured vehicle. Therefore, this thesis introduces a novel concept, called Virtual Thermal Comfort Sensing (VTCS), a method that estimates occupant body part equivalent temperatures from a minimalistic set of inexpensive cabin environmental sensors. Implement¬ing VTCS within a vehicle cabin consists of two stages. First, using a mutual information-based approach, the set of cabin environmental sensors that correlate well with the body part equivalent temperatures is selected. Second, Multiple Linear Regression (MLR) is applied to infer the occupant body part equival¬ent temperatures from the selected cabin environmental sensors. MLR was selected as the most suitable learning method out of seven different approaches, based on the estimation accuracy provided and the processing time required for the estimation. The VTCS approach was evaluated using empirical data and provided an average Root Mean Square Error (RMSE) of 1.41 oC across environmental conditions characterised by cabin interior temperature rates of change of up to 6 oC per minute, ambient temperature differences up to 6 oC per experimental trial, varying ambient wind, solar load and precipitation.

    Finally, this thesis proposes an innovative reinforcement learning-based controller that phrases the control specification in terms of an overall objective function based on the thermal comfort of the pas¬sengers and the energy consumption. The performance metric used in evaluating the controller was the reward—a measure quantifying how thermally comfortable the occupant is and the amount of energy consumed. The proposed controller was evaluated through simulation within both single-zone and multi-zone scenarios and it exceeded the performance of a basic controller and a fuzzy logic-based controller by a factor of 2.15 and 1.7, respectively. These results translate into an average of 29.05% energy saving over 200 testing scenarios when compared to the fuzzy logic-based controller, while thermal comfort was achieved and maintained successfully.

    The combination of the VTCS and the reinforcement learning-based controller sets the benchmark for future HVAC systems aimed at delivering true occupant comfort. Furthermore, the techniques described in this thesis can be transferred to a wide variety of applications such as skin temperature-driven control or HVAC control in buildings.
    Date of Award2015
    Original languageEnglish
    Awarding Institution
    • Coventry University
    SupervisorJames Brusey (Supervisor) & Elena Gaura (Supervisor)

    Cite this