Assessing How Violent Threats from Students Toward Schools Are Affected Before and After a School Shooting

Methods

Our Institutional Review Board (Lifespan, Providence, RI) approved this retrospective study. We reviewed the records for all children who presented to Hasbro Children's Hospital Psychiatry Emergency Service or Access Center at Bradley Hospital in the 6 weeks before and 6 weeks after the Parkland shooting on February 14, 2018. Criteria for inclusion was threat of mass-casualty school violence recorded in the encounter history of present illness. Threat of mass-casualty school violence was defined as any verbal or written threat portending future grievous bodily harm or death to two or more people at school or during a school-sanctioned event. The data were stored in REDCap (a web application for building and managing online surveys and databases created by Vanderbilt University) and included basic demographic information, social background information, and the presence or absence of a list of characteristics we hypothesized could mitigate or elevate risk. Demographic data were analyzed with descriptive statistics.

A generalized linear model (logistic) was used to model the proportion of patients reporting school violence by time (before and after the Parkland shooting). A generalized linear model (negative binomial) was used to model the number of days a patient was admitted by level of time. To understand differences over time by level of specific threat, the number of days admitted was also modeled by level of time and specific threat. We included an interaction term in the model to allow for differences over time by level of specific threat. This analysis was repeated for level of gun access. Changes in clinical decision-making about patient disposition (ie, rate of hospital admission, partial program discharge, and outpatient discharge) were also modeled (logistic) by time and level of specific threat. Classic “sandwich” estimation was used to adjust for any model misspecification. Familywise alpha was maintained at 0.05 using the Holm adjustment for multiple comparisons. Adjusted P values are reported unless otherwise stated. All statistical models were run using Proc Glimmix (SAS: version 9.2; SAS Institute Inc, Cary, NC), allowing for modeling of generalized linear models, as well as deriving P values for model fixed effects or estimated mean comparisons.

Results

We identified 42 records of patients who met inclusion criteria: 35 (83.3%) of the patients were male and 7 (16.7%) were female. The mean age at presentation was 12.89 ± 3.59 years. Thirteen of the 603 patients who presented for emergency psychiatric care in the 6 weeks preceding the Parkland shooting met inclusion criteria. In the 6 weeks after the Parkland shooting, 29 of the 609 patients who presented for emergency psychiatric care met inclusion criteria. The proportion of patients reporting school violence was higher after the Parkland shooting than before: mean estimate 0.046, 95% confidence interval (CI) of 0.033–0.066 versus 0.021 (95% CI, 0.012–0.036), respectively (P = .0164) (Figure 1).

Figure 1. Proportion of patients reporting to psychiatric emergency services with threats of school violence before and after the Parkland, Florida shooting. Blue and red dots represent mean estimates and bars represent 95% confidence intervals.

Patient Characteristics

Almost one-half of all patients (n = 20, 48.8%) were referred to the hospital by their school; the rest were referred by police (n = 12, 29.3%), family (n = 6, 14.6%), or outpatient mental health provider (n = 1, 2.4%). Twenty-two (52.4%) patients were taking psychotropic medications at presentation, 25 (59.5%) were actively being seen by a mental health provider, and 16 (38.1%) had a previous psychiatric hospitalization. Additionally, 11 (26.2%) patients referenced previous school shootings during their psychiatric evaluation (Table 1).

Table 1: Selected Patient Characteristics (N = 42)

Admission Length

The estimated mean number of admission days was longer after the Parkland shooting than before (26.4 [95% CI, 13.1–53.4] vs 9.2 [95% CI, 5.8–14.6] days, respectively; P = 0.0164) (Figure 2). One patient had a much higher number of admitted days (n = 150) than others after the Parkland shooting. After reanalysis omitting this high-leverage point, admission days were still longer after the Parkland shooting than before (17.6 [95% CI, 12.9–24.1] vs 9.2 [95% CI, 5.7–14.6] days, respectively; P = 0.0254).

Figure 2. Number of days admitted to the hospital for patients presenting to psychiatric emergency services before and after the Parkland, Florida shooting. Blue and red dots represent mean estimates and bars represent 95% confidence intervals. Gray circles represent days admitted for individual patients.

Threat Level

Days admitted for patients with a specific plan before and after the Parkland shooting were 9.2 (95% CI, 4.8–17.8) and 41.4 (95% CI, 11.5–19) days, respectively. This difference was significant (P = .0138, adjusted P = .0607) (Figure 3). This difference was maintained when omitting the patient who had a 150-day admission (P = .0481). Days admitted for patients with a vague threat before and after the Parkland shooting were 9 (95% CI, 5.5–14.8) and 14.8 (11.5–19), respectively (this did not reach significance; P = .0772, adjusted P = 0.1749) (Figure 3). Before the Parkland shooting, the days admitted for patients were not significantly different between those with a specific plan and those with a vague threat (P = .9448). After the Parkland shooting, the days admitted were significantly longer for those who had a specific threat than those with a vague threat (P = .0411, adjusted P = .1661) (Figure 3). This difference was not maintained when the high-leverage point was removed (P = .185).

Figure 3. Number of days admitted to the hospital for specific and vague threats for before and after the Parkland, FL shooting. Blue dots represent the mean estimate for vague threats and red dots represent the mean estimates for specific threats. Bars represent 95% confidence intervals. Gray circles represent days admitted for individual patients.

Gun Access

Level of time and level of gun access were found to affect the length of hospitalization (P = .0009 and .0150, respectively). Patients who were reported to have gun access, no-known gun access, and undocumented gun access had 22 [95% CI, 13–37.3], 11.8 [95% CI, 7.2–19.1], and 9.5 [95% CI, 8.2–11] admitted days, respectively (Figure 4). Patients who had documented gun access had a higher number of admitted days than patients with undocumented gun access (P = .0049, adjusted P = .0128). Those with reported gun access did not have statistically different admitted days than the no-known access group (P = .0811, adjusted P = .1820) (Figure 4). Admitted days were not found to be statistically different between the no-known access group and the undocumented access group (P = .3821). The number of admitted days was not found to vary by level of gun access across time points (no significant interaction between time and gun access, P = .8645).

Figure 4. Number of days admitted to the hospital before and after the Parkland, FL shooting by level of gun access. Blue dots represent the mean estimate for patients with gun access, red dots represent the mean estimates for patients with no gun access, and green dots represent patients with undocumented gun access. Bars represent 95% confidence intervals. Gray circles represent individual patients admitted days.

Disposition

Time and level of specific threat were not found to affect the rate of hospital admission (P = .5891 and P = .1395, respectively). Time and level of specific threat were also not found to affect rate of outpatient referral (P = .820 and P =.587, respectively). Time and level of specific threat were found to affect rate of partial program referral (P < .0001 and P < .0001, respectively; interaction P < .0001). For patients with a specific plan, the rate of partial program admission was higher after the Parkland shooting compared to before (0.18 [95% CI, 0.04–0.52] compared to 0 [95% CI, 0-0]; P < .0001). The rate of partial program admission for patients that had specific threats before the Parkland shooting was lower than all other subgroups (P < .0001) (Figure 5).

Figure 5. (A) Rate of hospital admission, (B) rate of outpatient referral, and (C) rate of partial program admission by level of threat for before and after the Parkland, FL shooting.

Discussion

The principal aim of this study was to determine how the rates of hospital admission and duration of stay were affected proximal to a school shooting in youth who present for psychiatric evaluation after threatening school violence, and to discover if these patients had any discerning characteristics that could inform future decision-making during psychiatric evaluations. Our results show that rate and proportion of threats of school violence causing psychiatric emergency service visits and the mean length of hospital stays increased in the 6 weeks after the Parkland shooting relative to the 6 weeks before. Surprisingly, there was not a statistical difference in mean number of days admitted between patients with gun access and patients with no gun access.

The increase in incidence and proportion of youth presenting to emergency services for psychiatric evaluation after making a threat of school violence in the weeks after the Parkland shooting (n = 29, 4.6%) compared to the weeks before (n = 13, 2.1%) is consistent with the literature. These results parallel the findings of Kostinsky et al.,⁹ who established that the number of threats of school violence escalated shortly after the Columbine school shooting. The results are also in accordance with the contagion effect observed in several different types of widely publicized tragedies.^10–13 The observed increase in proportion of threats of school violence in youth presenting for emergency psychiatric evaluation after the Parkland shooting and other school shootings likely stem from some combination of competing effects: (1) adolescents who become inspired by attention and media coverage of school shootings; (2) adolescents who are already troubled and begin expressing themselves differently through imitation without intention of actual harm; and (3) hypervigilance of parents and teachers, resulting in increased reporting and referral to emergency services. Although these effects provide a logical basis for increased incidence and proportion of youth presenting to emergency services for threats of school violence after the Parkland shooting, the data and analysis presented herein do not provide evidence in favor of a conclusion of one specific effect over any other.

The mean length of hospitalization also demonstrated a statistically significant increase for youth presenting to emergency services after threats of school violence in the 6 weeks after the Parkland shooting (26.4 days) than the 6 weeks before (9.2 days). Even omitting the high-leverage point (a patient with a 150-day admission in the post-Parkland group), the mean length of hospitalization still significantly increased after the Parkland shooting (mean of 17.6 days vs 9.2 days). The extended mean length of hospitalization can possibly be explained by a heightened risk for imminent violence. Davoren et al.¹⁴ found that higher assessed scores on “immediacy of risk of serious violence” was correlated to longer lengths of stay for patients in forensic psychiatric hospitals. Another explanation suggests that physicians exercised more caution because of the high-profile event. This caution is likely borne out of the physician's sense of duty to protect the public, the perceived likelihood of imitation, and overcompensation due to media scrutiny directed at physicians and the health care system for failing to properly identify and treat patients presenting significant risk to society.¹⁵

Patients were further divided into subgroups of those with a specific plan for their threat of school violence and those with a vague plan, and the mean number of days admitted to the hospital were calculated. We hypothesized that patients presenting with a specific plan for enacting school violence would constitute a more credible threat, which would manifest in longer admissions to the hospital. However, the difference between the number of days admitted for the specific plan and vague plan subgroups did not rise to the level of significance. This suggests that specific threats were not necessarily deemed more credible than vague threats prior to the Parkland shooting. Nevertheless, patients with specific plans did exhibit a significantly higher mean length of stay in the wake of the Parkland shooting relative to patients with vague plans. Those with specific plans after the Parkland shooting experienced a significantly higher number of days admitted relative to those with specific plans before the Parkland shooting. Once again, this trend may possibly be attributed to physicians giving more heed to threats after the Parkland shooting or that the threats are considered more credible, signifying the youth have been inspired by the Parkland shooting in some way. The data do not provide evidence that elucidates the causal mechanism of this trend, and it is possible that both explanations contribute to the results. Those with vague plans before and after the Parkland shooting did not demonstrate a significant difference in length of stay, suggesting that the Parkland shooting did not influence the perceived credibility of vague threats nor the manner in which physicians responded to vague threats.

To further quantify the perceived threat level, we measured admission length as a function of gun ownership. The mean number of days admitted was not statistically different after the Parkland shooting relative to before in any group with gun access. Gun owners experienced longer average hospitalization than people who did not own guns, although this difference was not statistically significant. Gun owners did experience a significantly higher number of admitted days relative to those whose access to guns was undocumented. One explanation is that physicians perceived patients with immediate access to guns to be at higher risk for carrying out a threat, and so were more cautious to avoid premature discharge. In a study by Kaufman et al.,¹⁶ suicide rates were higher in homes with guns, states with higher levels of gun ownership, and states with fewer restrictions and gun laws, as immediate gun access makes it easier to act on impulsive thoughts.

There was no detectable difference between the rate of hospital admissions before and after the Parkland shooting in patients that made specific threats, nor between specific threat and vague threat subgroups. Additionally, there were no detectable differences for rates of referral to outpatient treatment between the two subgroups nor within each subgroup before and after the Parkland shooting. However, the rate of admission for partial hospitalization was significantly higher after the Parkland shooting relative to before for patients with specific plans, because zero patients with specific plans of school violence were referred to partial hospitalization programs before the Parkland shooting. These findings support our hypothesis that there would be an increased rate of inpatient hospitalization in the weeks after the Parkland shooting. An increase in rates of admission to partial hospitalization programs without a corresponding increase in in-patient hospitalization suggests that the physicians believe those patients do not pose an imminent threat to themselves or others.¹⁷

Study Limitations

This study does possess some limitations. Given its retrospective nature, we were unable to control information gathered in the medical record, especially undocumented gun ownership data. We also could not account for potential confounders, such as prior media exposure to violence, death, and suffering separate from the Parkland shooting. Other potential confounders are academic and social stressors. The Parkland shooting occurred in February, marking the end of the first academic semester and start of a new semester for grade schools. Future research could explore these confounding variables by conducting a longitudinal design throughout the academic year.

Our results were also limited by the small sample size of patients presenting to psychiatric emergency services and disposition data. Therefore, the absence of a detectable difference in categorical disposition data does not substantiate that such a difference does not exist—rather, it demonstrates the importance of enhancing data collection techniques after incidences of mass violence to facilitate future research. Furthermore, our dataset is susceptible to sampling bias. Our findings gleaned from two Rhode Island hospitals are not necessarily representative of the US population, so extrapolation would require additional corroborating evidence. Finally, the cross-sectional design of our study investigates associations rather than causality. Future research with larger sample sizes and longitudinal design should aim to replicate these findings.

Conclusion

This study found that youth who presented to psychiatric emergency services surrounding the Parkland shooting reported threats of school violence in higher proportions after the tragedy. Additionally, making highly specific threats, as well as ownership of a gun, were correlated with longer hospital admissions. This study was limited by the small sample size of emergency service data from Hasbro Children's and Bradley Hospitals, as well as insufficient reporting of medical records. The data could not provide explanations into causal mechanisms of the increased proportion of threats of school violence nor longer length of hospital stays. Despite these limitations, our findings demonstrate the importance of conducting future research into topics surrounding the fallout from acts of public mass violence by identifying risk factors in patients threatening school violence to improve assessment and management of such threats.