A framework for ex ante evaluation of energy savings campaigns

Caroline Wilson, Stuart Graeme

Research output: Chapter in Book/Report/Conference proceedingConference proceeding

Abstract

Awareness campaigns, education and training programmes, label schemes and smart metering are all initiatives based on the principle that more and better information will encourage consumers to conserve energy use. Initiatives of this type can realise efficiency savings of up to 30%, and are likely to remain politically popular while preferred by the public to legislation or fines. While widespread internationally, such programmes can have mixed performance, with savings often not reaching potential. A major challenge when attempting to drive behaviour using communication is that what makes information persuasive is contextual, according to the situation and the interests, cultural expectations and needs of the audience. Further, across Europe such behaviour change initiatives do not currently have a track record of quality evaluation (EEA, 2013). Ex-ante evaluation would provide organisations with much-needed guidance on whether such initiatives would be likely to be successful, or indeed identify the risk that they might be counter-productive. The Elaboration Likelihood Model (Petty & Cacioppo, 1986) offers a potential framework for assessing this type of strategy ahead of implementation. This paper reports on the use of the framework to assess a proposal to engage building users with a new feedback mechanism. The buildings in this research are a sub-set of those in the EU-funded SMARTSPACES project to deliver more efficient energy use in public buildings. The goal of the intervention in these sub-set buildings is to build participatory engagement rather than issue top-down information about what building users should do. Quantitative findings show that the proposed intervention contains information which is credible and believable and associated with raised intentions adopt energy saving behaviour. Qualitative feedback from focus groups provides insights into the aspects of the intervention which fail to motivate. Other learning for the SMARTSPACES project was that even though the goal may be to avoid a top down authority-led intervention, and initiative based on participatory principles still needs guidance for the invitee on how to participate/respond. A key finding was that an ͚invitation to participate͛, without prompts as to what action to take, risks leaving audiences overwhelmed, which is associated with a sense of powerlessness (Ereaut & Segnit, 2006; Fisher, 2013; FUTERRA, 2005). Another key learning is that further support, such as cultural priming, and possibly greater shared knowledge on both sides, is needed before two-way communication is realistic in these situations. These insights demonstrate how using communication theory to frame an ex ante evaluation has potential general use as a mechanism to improve the effectiveness of information-led initiatives to encourage adoption of new behaviours.
Original languageEnglish
Title of host publicationEuropean Evaluation Society Biennial Conference:
Subtitle of host publicationEvaluation for an Equitable Society
Publication statusPublished - 2 Oct 2014
Event11th European Evaluation Society Biennial Conference: Evaluation for an Equitable Society - Dublin, Ireland
Duration: 1 Oct 20143 Oct 2014

Conference

Conference11th European Evaluation Society Biennial Conference
Abbreviated titleEES
CountryIreland
CityDublin
Period1/10/143/10/14

Fingerprint

energy saving
campaign
evaluation
building
savings
EEA
saving behavior
energy
communication behavior
communication theory
learning
training program
EU
legislation
efficiency
communication
performance
education
Group

Cite this

Wilson, C., & Graeme, S. (2014). A framework for ex ante evaluation of energy savings campaigns. In European Evaluation Society Biennial Conference: : Evaluation for an Equitable Society

A framework for ex ante evaluation of energy savings campaigns. / Wilson, Caroline; Graeme, Stuart.

European Evaluation Society Biennial Conference: : Evaluation for an Equitable Society. 2014.

Research output: Chapter in Book/Report/Conference proceedingConference proceeding

Wilson, C & Graeme, S 2014, A framework for ex ante evaluation of energy savings campaigns. in European Evaluation Society Biennial Conference: : Evaluation for an Equitable Society. 11th European Evaluation Society Biennial Conference, Dublin, Ireland, 1/10/14.
Wilson C, Graeme S. A framework for ex ante evaluation of energy savings campaigns. In European Evaluation Society Biennial Conference: : Evaluation for an Equitable Society. 2014
Wilson, Caroline ; Graeme, Stuart. / A framework for ex ante evaluation of energy savings campaigns. European Evaluation Society Biennial Conference: : Evaluation for an Equitable Society. 2014.
@inproceedings{b4b8cb76af5b42e5997ea8164c5a33d3,
title = "A framework for ex ante evaluation of energy savings campaigns",
abstract = "Awareness campaigns, education and training programmes, label schemes and smart metering are all initiatives based on the principle that more and better information will encourage consumers to conserve energy use. Initiatives of this type can realise efficiency savings of up to 30{\%}, and are likely to remain politically popular while preferred by the public to legislation or fines. While widespread internationally, such programmes can have mixed performance, with savings often not reaching potential. A major challenge when attempting to drive behaviour using communication is that what makes information persuasive is contextual, according to the situation and the interests, cultural expectations and needs of the audience. Further, across Europe such behaviour change initiatives do not currently have a track record of quality evaluation (EEA, 2013). Ex-ante evaluation would provide organisations with much-needed guidance on whether such initiatives would be likely to be successful, or indeed identify the risk that they might be counter-productive. The Elaboration Likelihood Model (Petty & Cacioppo, 1986) offers a potential framework for assessing this type of strategy ahead of implementation. This paper reports on the use of the framework to assess a proposal to engage building users with a new feedback mechanism. The buildings in this research are a sub-set of those in the EU-funded SMARTSPACES project to deliver more efficient energy use in public buildings. The goal of the intervention in these sub-set buildings is to build participatory engagement rather than issue top-down information about what building users should do. Quantitative findings show that the proposed intervention contains information which is credible and believable and associated with raised intentions adopt energy saving behaviour. Qualitative feedback from focus groups provides insights into the aspects of the intervention which fail to motivate. Other learning for the SMARTSPACES project was that even though the goal may be to avoid a top down authority-led intervention, and initiative based on participatory principles still needs guidance for the invitee on how to participate/respond. A key finding was that an ͚invitation to participate͛, without prompts as to what action to take, risks leaving audiences overwhelmed, which is associated with a sense of powerlessness (Ereaut & Segnit, 2006; Fisher, 2013; FUTERRA, 2005). Another key learning is that further support, such as cultural priming, and possibly greater shared knowledge on both sides, is needed before two-way communication is realistic in these situations. These insights demonstrate how using communication theory to frame an ex ante evaluation has potential general use as a mechanism to improve the effectiveness of information-led initiatives to encourage adoption of new behaviours.",
author = "Caroline Wilson and Stuart Graeme",
year = "2014",
month = "10",
day = "2",
language = "English",
booktitle = "European Evaluation Society Biennial Conference:",

}

TY - GEN

T1 - A framework for ex ante evaluation of energy savings campaigns

AU - Wilson, Caroline

AU - Graeme, Stuart

PY - 2014/10/2

Y1 - 2014/10/2

N2 - Awareness campaigns, education and training programmes, label schemes and smart metering are all initiatives based on the principle that more and better information will encourage consumers to conserve energy use. Initiatives of this type can realise efficiency savings of up to 30%, and are likely to remain politically popular while preferred by the public to legislation or fines. While widespread internationally, such programmes can have mixed performance, with savings often not reaching potential. A major challenge when attempting to drive behaviour using communication is that what makes information persuasive is contextual, according to the situation and the interests, cultural expectations and needs of the audience. Further, across Europe such behaviour change initiatives do not currently have a track record of quality evaluation (EEA, 2013). Ex-ante evaluation would provide organisations with much-needed guidance on whether such initiatives would be likely to be successful, or indeed identify the risk that they might be counter-productive. The Elaboration Likelihood Model (Petty & Cacioppo, 1986) offers a potential framework for assessing this type of strategy ahead of implementation. This paper reports on the use of the framework to assess a proposal to engage building users with a new feedback mechanism. The buildings in this research are a sub-set of those in the EU-funded SMARTSPACES project to deliver more efficient energy use in public buildings. The goal of the intervention in these sub-set buildings is to build participatory engagement rather than issue top-down information about what building users should do. Quantitative findings show that the proposed intervention contains information which is credible and believable and associated with raised intentions adopt energy saving behaviour. Qualitative feedback from focus groups provides insights into the aspects of the intervention which fail to motivate. Other learning for the SMARTSPACES project was that even though the goal may be to avoid a top down authority-led intervention, and initiative based on participatory principles still needs guidance for the invitee on how to participate/respond. A key finding was that an ͚invitation to participate͛, without prompts as to what action to take, risks leaving audiences overwhelmed, which is associated with a sense of powerlessness (Ereaut & Segnit, 2006; Fisher, 2013; FUTERRA, 2005). Another key learning is that further support, such as cultural priming, and possibly greater shared knowledge on both sides, is needed before two-way communication is realistic in these situations. These insights demonstrate how using communication theory to frame an ex ante evaluation has potential general use as a mechanism to improve the effectiveness of information-led initiatives to encourage adoption of new behaviours.

AB - Awareness campaigns, education and training programmes, label schemes and smart metering are all initiatives based on the principle that more and better information will encourage consumers to conserve energy use. Initiatives of this type can realise efficiency savings of up to 30%, and are likely to remain politically popular while preferred by the public to legislation or fines. While widespread internationally, such programmes can have mixed performance, with savings often not reaching potential. A major challenge when attempting to drive behaviour using communication is that what makes information persuasive is contextual, according to the situation and the interests, cultural expectations and needs of the audience. Further, across Europe such behaviour change initiatives do not currently have a track record of quality evaluation (EEA, 2013). Ex-ante evaluation would provide organisations with much-needed guidance on whether such initiatives would be likely to be successful, or indeed identify the risk that they might be counter-productive. The Elaboration Likelihood Model (Petty & Cacioppo, 1986) offers a potential framework for assessing this type of strategy ahead of implementation. This paper reports on the use of the framework to assess a proposal to engage building users with a new feedback mechanism. The buildings in this research are a sub-set of those in the EU-funded SMARTSPACES project to deliver more efficient energy use in public buildings. The goal of the intervention in these sub-set buildings is to build participatory engagement rather than issue top-down information about what building users should do. Quantitative findings show that the proposed intervention contains information which is credible and believable and associated with raised intentions adopt energy saving behaviour. Qualitative feedback from focus groups provides insights into the aspects of the intervention which fail to motivate. Other learning for the SMARTSPACES project was that even though the goal may be to avoid a top down authority-led intervention, and initiative based on participatory principles still needs guidance for the invitee on how to participate/respond. A key finding was that an ͚invitation to participate͛, without prompts as to what action to take, risks leaving audiences overwhelmed, which is associated with a sense of powerlessness (Ereaut & Segnit, 2006; Fisher, 2013; FUTERRA, 2005). Another key learning is that further support, such as cultural priming, and possibly greater shared knowledge on both sides, is needed before two-way communication is realistic in these situations. These insights demonstrate how using communication theory to frame an ex ante evaluation has potential general use as a mechanism to improve the effectiveness of information-led initiatives to encourage adoption of new behaviours.

M3 - Conference proceeding

BT - European Evaluation Society Biennial Conference:

ER -