Discussion
Mental health apps are marketed to employers to promote the mental wellbeing of individual employees, and thereby benefit the company overall. The supporting evidence for these claims alludes to an association between use of the app and increased productivity, relying on inference as opposed to scientific method.
For example, the resource “Head-space for Work” ( https://www.headspace.com/work) attempts to build a logical chain between reduced employee stress and improved organizational monetary outcomes. The resource cites occupational health studies, which note an association between stress and monetary losses, as well as articles that identify an association between mindfulness and a reduction in stress. Finally, Headspace for Work” highlights the importance of mindfulness and its effects on teamwork, creativity, and cognition.
Most of the app-based literature makes inferences based on numerous limitations of study design; the most glaring of which is a failure to assess human functioning. For instance, does the measured outcome have a meaningful effect on the patient's affect and behavior? Or does it simply represent a reduction on a psychological scale? One could argue that trait-based variables such as stress, well-being, and happiness are less detrimental to function compared to mental health disorders. Loss of functioning or impairment in work/play is a key criterion for the diagnosis of depression and anxiety disorders based on the Diagnostic and Statistical Manual of Mental Disorders, 5th edition.12 As such, apps that claim significant success in preventing or treating psychological disorders may have a higher likelihood of improving functional outcomes.
However, this does not mean that a reduction on a scale of anxiety or depression is necessarily clinically (let alone functionally/occupationally) relevant. It is possible, for instance, that a given change on a symptomatic scale is not large enough to have a plausible clinical effect. In other words, if a mental health app is meant to decrease depressive symptoms, does a given reduction in the Patient Health Questionnaire-9 (PHQ-9) score then result in a significant and relevant effect on occupational or enterprise outcomes?13 For example, does a lower score on the Perceived Stress Scale (PSS-10) cause a clinically meaningful reduction in stress to improve productivity in the work environment (Work Productivity and Impairment Questionnaire)?
Strength of evidence for claims that apps improve occupational outcomes depends on statistically significant direct effects demonstrated in experimental trials. These trials should use validated physical health, mental health, and/or psychosocial scales that are either clinically relevant and/or demonstrate a direct effect on occupational outcomes (via similarly well validated and clinically relevant scales). The ideal study to validate the claims of a workplace-based mental health app intervention would be a randomized controlled trial conducted among employees from businesses of varying sizes and industries. Outcome measures should include validated and clinically relevant scales of mental health (anxiety, depression), psychological wellbeing, and occupational metrics. Based on our review of the literature, this study does not yet exist.
In this article, we identified 12 studies published between 2014 and 2019 (Table 1) from the primary literature that met our inclusion criteria. Five studies were designed with a single-group and seven studies were randomized controlled trials. Certain studies directly measured changes in neuropsychiatric outcomes (ie, depression, anxiety) among the target population, which we have labeled “direct.” Other studies measured changes in related outcomes, including physical health, well-being, and productivity, which we have labeled as “indirect.” Of note, several studies included both “direct” and “indirect” outcomes measures (Table 1). We considered these studies in the context of the “direct” or “indirect” outcomes they reported.
Among the “direct” cohort of studies,14–17 target outcomes focused on reduction of depression and anxiety in employees. All four of these studies demonstrated that use of a mental health app caused a significant reduction in scores on psychometric scales for depression and/or anxiety among workers. The findings of Bostock et al.14 and Birney et al.15 are most relevant and applicable given their large sample size, randomized controlled design, and broad targeted population of employees. Of note, Birney et al.15 is the only study included in our review where the recruited employees demonstrated mild to moderate depression (per PHQ-9 rating scale) at baseline. The other studies either included healthy employees14,17 or those who self-reported mental health symptoms.16
A peculiar finding from the Birney et al.15 study was that among those workers randomized to the cognitive-behavioral therapy app, employees who were recruited through an EAP demonstrated a more robust reduction in depressive symptoms compared to the non-EAP workers. This raises the question of whether app-based interventions in the workplace are generally more effective when they are introduced and supported by face-to-face mental health programs.
Although there are changes in scores on depressive scales across these four studies, deeper evaluation is necessary to determine clinical relevance. Birney et al.15 report a mean difference between app use and no app use of 2.6 on the PHQ-9.15 Similarly, Deady et al.17 found statistical significance with a reduction in PHQ-9 scores of 2.3. Bostock et al.14 used a less common clinical scale (the Hospital Anxiety and Depressive Scale), which ranges from 0 to 21. The statistically significant change in those employees using the app was 1.69 in anxiety scores and 1.45 in depressive scores.14 Lappalainen et al.16 used the original Beck Depression Inventory and noted a reduction in depression scores by 8.46 in their intervention arm. However, the control arm also showed a reduction in scores by 4.16 Clinical relevance seems unlikely given the total score is out of 63.
The “indirect” outcome measures that were most studied were “reduction in stress”16–23 followed by improvement in psychological well-being.14,17,20,22,23 Most of these studies found that people assigned to the app-based intervention group reported significant improvements in stress and psychological well-being with respect to the controls. Other “indirect” outcomes studied include mindfulness24 and general well-being.19
Only one study included an active control group, in which patients were guided through self-observation and self-direction programming that spanned the intervention duration.20 The authors reported that the intervention group showed significant improvement in general well-being (World Health Organization-Five Well-Being Scale), work-related well-being, perceived stress (PSS), work-related stress, and general stress compared to the active control group. Another study examined a mental health app targeted specifically at middle managers versus general employees.18 Heber et al.19 was one of the few studies included in our analysis that reported long-term follow up data. At the 6-month follow-up, the intervention group continued to have significantly lower PSS-10 scores compared to the wait-list control group.19
None of the studies we identified evaluated the effects of app usage in the workplace on broad, organization-level outcomes including company productivity and sales growth. One study, however, noted an “indirect” effect of mental health apps on individual organizational outcomes in the workplace including a reduction in absenteeism and an increase in work performance.17 Another study noted statistically significant secondary outcomes in job control and job social support when employees used a mental health app for 8 weeks.14 But two other articles showed no significant improvement in worker engagement22 or in the sense of social community at work,23 respectively.