Measuring Engagement in eHealth and mHealth Behavior Change Interventions: Viewpoint of Methodologies

Camille E Short, Ann DeSmet, Catherine Woods, Susan L Williams, Carol Maher, Anouk Middelweerd, Andre Matthias Müller, Petra A Wark, Corneel Vandelanotte, Louise Poppe, Melanie D Hingle, Rik Crutzen

    Research output: Contribution to journalArticle

    15 Citations (Scopus)
    9 Downloads (Pure)

    Abstract

    Engagement in electronic health (eHealth) and mobile health (mHealth) behavior change interventions is thought to be important for intervention effectiveness, though what constitutes engagement and how it enhances efficacy has been somewhat unclear in the literature. Recently published detailed definitions and conceptual models of engagement have helped to build consensus around a definition of engagement and improve our understanding of how engagement may influence effectiveness. This work has helped to establish a clearer research agenda. However, to test the hypotheses generated by the conceptual modules, we need to know how to measure engagement in a valid and reliable way. The aim of this viewpoint is to provide an overview of engagement measurement options that can be employed in eHealth and mHealth behavior change intervention evaluations, discuss methodological considerations, and provide direction for future research. To identify measures, we used snowball sampling, starting from systematic reviews of engagement research as well as those utilized in studies known to the authors. A wide range of methods to measure engagement were identified, including qualitative measures, self-report questionnaires, ecological momentary assessments, system usage data, sensor data, social media data, and psychophysiological measures. Each measurement method is appraised and examples are provided to illustrate possible use in eHealth and mHealth behavior change research. Recommendations for future research are provided, based on the limitations of current methods and the heavy reliance on system usage data as the sole assessment of engagement. The validation and adoption of a wider range of engagement measurements and their thoughtful application to the study of engagement are encouraged.

    Original languageEnglish
    Article numbere292
    JournalJournal of Medical Internet Research
    Volume20
    Issue number11
    DOIs
    Publication statusPublished - 16 Nov 2018

    Fingerprint

    Telemedicine
    Health Behavior
    Information Systems
    Health
    Research
    Social Media
    Self Report
    Consensus

    Bibliographical note

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.

    Copyright © and Moral Rights are retained by the author(s) and/ or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This item cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder(s). The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders.

    Keywords

    • Telemedicine
    • Internet
    • Health Promotion
    • Evaluation studies
    • Treatment adherence and compliance
    • Outcome and process assessment (health care)

    Cite this

    Short, C. E., DeSmet, A., Woods, C., Williams, S. L., Maher, C., Middelweerd, A., ... Crutzen, R. (2018). Measuring Engagement in eHealth and mHealth Behavior Change Interventions: Viewpoint of Methodologies. Journal of Medical Internet Research, 20(11), [e292]. https://doi.org/10.2196/jmir.9397

    Measuring Engagement in eHealth and mHealth Behavior Change Interventions : Viewpoint of Methodologies. / Short, Camille E; DeSmet, Ann; Woods, Catherine; Williams, Susan L; Maher, Carol; Middelweerd, Anouk; Müller, Andre Matthias; Wark, Petra A; Vandelanotte, Corneel; Poppe, Louise; Hingle, Melanie D; Crutzen, Rik.

    In: Journal of Medical Internet Research, Vol. 20, No. 11, e292, 16.11.2018.

    Research output: Contribution to journalArticle

    Short, CE, DeSmet, A, Woods, C, Williams, SL, Maher, C, Middelweerd, A, Müller, AM, Wark, PA, Vandelanotte, C, Poppe, L, Hingle, MD & Crutzen, R 2018, 'Measuring Engagement in eHealth and mHealth Behavior Change Interventions: Viewpoint of Methodologies' Journal of Medical Internet Research, vol. 20, no. 11, e292. https://doi.org/10.2196/jmir.9397
    Short, Camille E ; DeSmet, Ann ; Woods, Catherine ; Williams, Susan L ; Maher, Carol ; Middelweerd, Anouk ; Müller, Andre Matthias ; Wark, Petra A ; Vandelanotte, Corneel ; Poppe, Louise ; Hingle, Melanie D ; Crutzen, Rik. / Measuring Engagement in eHealth and mHealth Behavior Change Interventions : Viewpoint of Methodologies. In: Journal of Medical Internet Research. 2018 ; Vol. 20, No. 11.
    @article{fc0c0cba547c4fc5b5329fc381ae543a,
    title = "Measuring Engagement in eHealth and mHealth Behavior Change Interventions: Viewpoint of Methodologies",
    abstract = "Engagement in electronic health (eHealth) and mobile health (mHealth) behavior change interventions is thought to be important for intervention effectiveness, though what constitutes engagement and how it enhances efficacy has been somewhat unclear in the literature. Recently published detailed definitions and conceptual models of engagement have helped to build consensus around a definition of engagement and improve our understanding of how engagement may influence effectiveness. This work has helped to establish a clearer research agenda. However, to test the hypotheses generated by the conceptual modules, we need to know how to measure engagement in a valid and reliable way. The aim of this viewpoint is to provide an overview of engagement measurement options that can be employed in eHealth and mHealth behavior change intervention evaluations, discuss methodological considerations, and provide direction for future research. To identify measures, we used snowball sampling, starting from systematic reviews of engagement research as well as those utilized in studies known to the authors. A wide range of methods to measure engagement were identified, including qualitative measures, self-report questionnaires, ecological momentary assessments, system usage data, sensor data, social media data, and psychophysiological measures. Each measurement method is appraised and examples are provided to illustrate possible use in eHealth and mHealth behavior change research. Recommendations for future research are provided, based on the limitations of current methods and the heavy reliance on system usage data as the sole assessment of engagement. The validation and adoption of a wider range of engagement measurements and their thoughtful application to the study of engagement are encouraged.",
    keywords = "Telemedicine, Internet, Health Promotion, Evaluation studies, Treatment adherence and compliance, Outcome and process assessment (health care)",
    author = "Short, {Camille E} and Ann DeSmet and Catherine Woods and Williams, {Susan L} and Carol Maher and Anouk Middelweerd and M{\"u}ller, {Andre Matthias} and Wark, {Petra A} and Corneel Vandelanotte and Louise Poppe and Hingle, {Melanie D} and Rik Crutzen",
    note = "This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included. Copyright {\circledC} and Moral Rights are retained by the author(s) and/ or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This item cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder(s). The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders.",
    year = "2018",
    month = "11",
    day = "16",
    doi = "10.2196/jmir.9397",
    language = "English",
    volume = "20",
    journal = "Journal of Medical Internet Research",
    issn = "1439-4456",
    publisher = "Journal of Medical Internet Research",
    number = "11",

    }

    TY - JOUR

    T1 - Measuring Engagement in eHealth and mHealth Behavior Change Interventions

    T2 - Viewpoint of Methodologies

    AU - Short, Camille E

    AU - DeSmet, Ann

    AU - Woods, Catherine

    AU - Williams, Susan L

    AU - Maher, Carol

    AU - Middelweerd, Anouk

    AU - Müller, Andre Matthias

    AU - Wark, Petra A

    AU - Vandelanotte, Corneel

    AU - Poppe, Louise

    AU - Hingle, Melanie D

    AU - Crutzen, Rik

    N1 - This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included. Copyright © and Moral Rights are retained by the author(s) and/ or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This item cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder(s). The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders.

    PY - 2018/11/16

    Y1 - 2018/11/16

    N2 - Engagement in electronic health (eHealth) and mobile health (mHealth) behavior change interventions is thought to be important for intervention effectiveness, though what constitutes engagement and how it enhances efficacy has been somewhat unclear in the literature. Recently published detailed definitions and conceptual models of engagement have helped to build consensus around a definition of engagement and improve our understanding of how engagement may influence effectiveness. This work has helped to establish a clearer research agenda. However, to test the hypotheses generated by the conceptual modules, we need to know how to measure engagement in a valid and reliable way. The aim of this viewpoint is to provide an overview of engagement measurement options that can be employed in eHealth and mHealth behavior change intervention evaluations, discuss methodological considerations, and provide direction for future research. To identify measures, we used snowball sampling, starting from systematic reviews of engagement research as well as those utilized in studies known to the authors. A wide range of methods to measure engagement were identified, including qualitative measures, self-report questionnaires, ecological momentary assessments, system usage data, sensor data, social media data, and psychophysiological measures. Each measurement method is appraised and examples are provided to illustrate possible use in eHealth and mHealth behavior change research. Recommendations for future research are provided, based on the limitations of current methods and the heavy reliance on system usage data as the sole assessment of engagement. The validation and adoption of a wider range of engagement measurements and their thoughtful application to the study of engagement are encouraged.

    AB - Engagement in electronic health (eHealth) and mobile health (mHealth) behavior change interventions is thought to be important for intervention effectiveness, though what constitutes engagement and how it enhances efficacy has been somewhat unclear in the literature. Recently published detailed definitions and conceptual models of engagement have helped to build consensus around a definition of engagement and improve our understanding of how engagement may influence effectiveness. This work has helped to establish a clearer research agenda. However, to test the hypotheses generated by the conceptual modules, we need to know how to measure engagement in a valid and reliable way. The aim of this viewpoint is to provide an overview of engagement measurement options that can be employed in eHealth and mHealth behavior change intervention evaluations, discuss methodological considerations, and provide direction for future research. To identify measures, we used snowball sampling, starting from systematic reviews of engagement research as well as those utilized in studies known to the authors. A wide range of methods to measure engagement were identified, including qualitative measures, self-report questionnaires, ecological momentary assessments, system usage data, sensor data, social media data, and psychophysiological measures. Each measurement method is appraised and examples are provided to illustrate possible use in eHealth and mHealth behavior change research. Recommendations for future research are provided, based on the limitations of current methods and the heavy reliance on system usage data as the sole assessment of engagement. The validation and adoption of a wider range of engagement measurements and their thoughtful application to the study of engagement are encouraged.

    KW - Telemedicine

    KW - Internet

    KW - Health Promotion

    KW - Evaluation studies

    KW - Treatment adherence and compliance

    KW - Outcome and process assessment (health care)

    U2 - 10.2196/jmir.9397

    DO - 10.2196/jmir.9397

    M3 - Article

    VL - 20

    JO - Journal of Medical Internet Research

    JF - Journal of Medical Internet Research

    SN - 1439-4456

    IS - 11

    M1 - e292

    ER -