Pediatric Annals

Evidence on Office-Based Interventions to Improve Childhood Immunization Delivery

Shannon L Udovic, MD, MS; Tracy A Lieu, MD, MPH

Abstract

Physician's offices, private clinics, and health maintenance organizations (HMOs) are the usual source of care for more than 85% of all US children.1 Thus, office-based interventions are a potentially powerful means of improving US immunization coverage. National standards based on expert consensus2 include several recommendations for office-based interventions: sending reminder or recall messages, reducing missed opportunities by administering simultaneous vaccine doses and using all visits to screen and immunize, and performing audits to assess immunization coverage levels.

The challenge for clinicians and administrators in office settings is that there are many possible interventions, and resources are often limited. Choosing die most promising interventions should ideally be based on scientific evidence about effectiveness and cost-effectiveness. This review identifies office-based interventions that have been shown to improve the delivery of immunizations to children and also describes evidence on cost-effectiveness.

METHODS

A literature search of English-language publications on interventions to improve the delivery of immunizations to children was conducted. The Medline database was searched from 1966 to the present. The keyword immunization identified a list of 13,097 articles, which was then narrowed using a variety of keywords including intervention (n=186, of which 21 abstracts were reviewed), incentive (n=ll articles, of which 3 abstracts were reviewed), computer reminder system (n=73, of which 14 abstracts were reviewed), and provider reminder (n=23, of which 4 abstracts were reviewed). The reference lists of relevant articles were reviewed. Senior researchers in the field were asked to identify additional relevant studies. To classify studies based on scientific quality and to classify the strength of our recommendations, we used a modified version of scales from the U.S. Preventive Services Task Force (Table I).3

Table

Randomized controlled trials give strong evidence that postcards, letters, personal and automated telephone calls all increase the rates of appointments attended by children and parents compared with no intervention.14'18 One study suggested that by reducing missed appointments, mailed postcards saved $7-50 for every dollar invested.15 There are few studies that compare the effectiveness of the various possible strategies relative to one another. Studies suggest that there is little difference in effectiveness between letters and postcards16 or between postcards and telephone messages.19

Missed Opportunities and Provider Prompting

A missed opportunity is defined as any office visit at which an immunization is due for, but is not administered to, an age-eligible child. Missed opportunities are common in a variety of settings, including suburban office practices, hospital-based clinics, and health maintenance organizations.20'24 One study at four university clinics found that eliminating missed opportunities would have increased the proportions of 2 -yearolds up-to-date for DTP vaccine by 8% to 21% and for polio vaccine by 5 to 16%.25

Missed opportunities can be divided into several types. One type occuts when a provider does not simultaneously administer all vaccines that are due at a single visit. Eliminating this type of missed opportunity by adopting simultaneous administration would significantly improve coverage rates of 24-montholds.23'24 However, our literature search did not find any studies of interventions specifically directed at reducing this type of missed opportunity.

Another type of missed opportunity results from failure to use acute care visits to screen and immunize. Systems that prompt physicians to deliver the appropriate immunizations at the time of all clinic visits, both acute and preventive, might address this problem. In one study that used a quasi-experimental design, the intervention consisted of reviewing the clinic appointment schedule each morning and attaching a label to the chart that described the immunization status of each child scheduled for a clinic visit. At the end of the year, 61 .3% in the intervention group had received the 3rd DPT/OPV compared…

Physician's offices, private clinics, and health maintenance organizations (HMOs) are the usual source of care for more than 85% of all US children.1 Thus, office-based interventions are a potentially powerful means of improving US immunization coverage. National standards based on expert consensus2 include several recommendations for office-based interventions: sending reminder or recall messages, reducing missed opportunities by administering simultaneous vaccine doses and using all visits to screen and immunize, and performing audits to assess immunization coverage levels.

The challenge for clinicians and administrators in office settings is that there are many possible interventions, and resources are often limited. Choosing die most promising interventions should ideally be based on scientific evidence about effectiveness and cost-effectiveness. This review identifies office-based interventions that have been shown to improve the delivery of immunizations to children and also describes evidence on cost-effectiveness.

METHODS

A literature search of English-language publications on interventions to improve the delivery of immunizations to children was conducted. The Medline database was searched from 1966 to the present. The keyword immunization identified a list of 13,097 articles, which was then narrowed using a variety of keywords including intervention (n=186, of which 21 abstracts were reviewed), incentive (n=ll articles, of which 3 abstracts were reviewed), computer reminder system (n=73, of which 14 abstracts were reviewed), and provider reminder (n=23, of which 4 abstracts were reviewed). The reference lists of relevant articles were reviewed. Senior researchers in the field were asked to identify additional relevant studies. To classify studies based on scientific quality and to classify the strength of our recommendations, we used a modified version of scales from the U.S. Preventive Services Task Force (Table I).3

Table

TABLE 1Rating Systems for Quality of Evidence and Strength of Recommendations, Modified from US Preventive Services Task Force

TABLE 1

Rating Systems for Quality of Evidence and Strength of Recommendations, Modified from US Preventive Services Task Force

RESULTS

Immunization Messages Sent to Families

The effectiveness and cost-effectiveness of immunization messages directed to the households of invididual families is affected by three factors: the timing of messages, tlie mode of transmission, and the population served. Immunization messages sent to parents in advance of routine due dates are termed reminder messages, whereas those sent to parents of children who are overdue for immunizations are termed recall messages. The possible modes of transmission are letters, postcards, personal telephone calls, or automated telephone calls. Among low- income populations, the baseline immunization coverage rate tends to be lower than among privately insured populations. Trie baseline coverage rate, the existence of computer hardware and software, and the size and mobility of the population being served all affect the effectiveness and cost-effectiveness of message interventions.

Table 2 summarizes the studies of immunizationspecific messages to parents that we reviewed. Most were randomized trials that included no- intervention control groups. For routine infant and toddler immunization delivery, there is strong evidence that both reminder and recall strategies are effective.4"10 However, recall strategies result in more dramatic increases in the proportions of targeted children who subsequently receive immunizations compared with reminder strategies.7 There are also more available studies of recall strategies than of reminder strategies. For children with asthma, two studies found that reminder letters for influenza vaccination were effective.11,12

There is limited evidence about which mode of transmission, ie, mail or telephone, is most effective. Although concern is sometimes raised that lowincome populations may not be accessible by telephone, Linkins and colleagues found automated telephone messages increased immunization delivery among county health department patients.7 In an HMO population, Lieu and colleagues found that a recall strategy of letters followed by automated telephone messages was more effective and cost-effective than either type of message alone.13 Interventions to increase immunization delivery are likely to be more cost-effective where there are low baseline coverage rates, less complex requirements for computer hardware, or stable populations with lower rates of telephone and address changes.10,13

Appointment Reminders Sent to Families

Many studies have evaluated the effectiveness of messages sent to parents in advance of preventive clinic appointments to remind them to bring their children to the appointments. Because attendance at preventive visits is strongly correlated with receipt of immunizations, we briefly review these interventions below.

Table

TABLE 2Studies of the Effectiveness on Immunizaation-Specific Messages to Parents

TABLE 2

Studies of the Effectiveness on Immunizaation-Specific Messages to Parents

Table

TABLE 2Studies of the Effectiveness on Immunizaation-Specific Messages to Parents

TABLE 2

Studies of the Effectiveness on Immunizaation-Specific Messages to Parents

Table

TABLE 3Quality of Evidence and Strength of Recommendations for Interventions to Improve Childhood Immunization Delivery

TABLE 3

Quality of Evidence and Strength of Recommendations for Interventions to Improve Childhood Immunization Delivery

Randomized controlled trials give strong evidence that postcards, letters, personal and automated telephone calls all increase the rates of appointments attended by children and parents compared with no intervention.14'18 One study suggested that by reducing missed appointments, mailed postcards saved $7-50 for every dollar invested.15 There are few studies that compare the effectiveness of the various possible strategies relative to one another. Studies suggest that there is little difference in effectiveness between letters and postcards16 or between postcards and telephone messages.19

Missed Opportunities and Provider Prompting

A missed opportunity is defined as any office visit at which an immunization is due for, but is not administered to, an age-eligible child. Missed opportunities are common in a variety of settings, including suburban office practices, hospital-based clinics, and health maintenance organizations.20'24 One study at four university clinics found that eliminating missed opportunities would have increased the proportions of 2 -yearolds up-to-date for DTP vaccine by 8% to 21% and for polio vaccine by 5 to 16%.25

Missed opportunities can be divided into several types. One type occuts when a provider does not simultaneously administer all vaccines that are due at a single visit. Eliminating this type of missed opportunity by adopting simultaneous administration would significantly improve coverage rates of 24-montholds.23'24 However, our literature search did not find any studies of interventions specifically directed at reducing this type of missed opportunity.

Another type of missed opportunity results from failure to use acute care visits to screen and immunize. Systems that prompt physicians to deliver the appropriate immunizations at the time of all clinic visits, both acute and preventive, might address this problem. In one study that used a quasi-experimental design, the intervention consisted of reviewing the clinic appointment schedule each morning and attaching a label to the chart that described the immunization status of each child scheduled for a clinic visit. At the end of the year, 61 .3% in the intervention group had received the 3rd DPT/OPV compared with 16.8% in the pre- intervention group.26

Likewise, a recent randomized controlled trial tested a system to prompt physicians to immunize at all possible visits.27 Under this system, a nurse attempted to screen immunization status for all children making visits and attach an immunization prompting card to each chart. Although this intervention appeared to reduce missed opportunities, it did not result in increased immunization coverage rates. However, the effectiveness of the intervention was limited by the fact that the immunization prompting cards were placed only on 33% of the vaccine-eligible patients' charts.

In other settings, including Northern California Kaiser Permanente, computerized immunization tracking systems are now being used to rapidly check and print out a child's immunization needs at the time of all acute care and preventive visits. A study of influenza vaccinations in the elderly suggests that such computerized prompting systems may be more effective at reducing missed opportunities than systems that rely on manual chart review by nurses.28

Incentives to Providers

We found no published studies of incentives to providers to improve immunization delivery in pediatric settings. However, two studies in other settings suggest this may be an effective intervention. In one family practice physician group, performance on various preventive outcome measures, including childhood MMR rates, were factored into the reimbursement multiplier along with utilization information and results of member surveys.29 Over 3 years, the percentage of offices with fewer than 90% of charts meeting the MMR standard fell from 57% in the first year to 42% the second and 12% in the third year (P<0.05).

Another study randomized providers admitting to one hospital to the intervention group and compared their rates of elderly vaccinated for influenza to those admitting to another hospital. The intervention group providers were allowed to use any interventions they chose, including postcards or telephone messages, and were told they would be reimbursed at a higher rate for achieving at least a 70% immunization coverage rate. The average immunization rate for the intervention group physicians was 73.1% compared to 55.7% in the control group.30

Incentives to Parents

Apart from Women, Infants & Children (WIC) clinic settings, which are reviewed in a separate article in next months issue (by Abby Shefer, MD), we found one study that examined the effect of providing incentives to parents for immunizations. Children in a public health clinic behind in their immunizations were assigned to one of six experimental groups, including general or specific prompt, increased access with weekend/weeknight immunization services, monetary incentive, contact control, or control.31 The monetary incentive group, a lottery to win a cash award, had the largest effect, a 31.6% increase in number of shots received over the control group. This figures compares with a 24-5% increase for the specific prompt group.

Assessment and Feedback

Physicians tend to estimate the immunization coverage rates for their own practices as being higher than they actually are.52 Thus, assessing coverage rates and providing feedback where problems exist might help improve immunization delivery. A program in public clinics in Georgia from 1988 through 1994 has annually assessed vaccination records to establish each clinic's proportion of 2 -year-old patients who are up-to-date for immunizations. This coverage rate is provided to the clinic along with information about missed opportunities and the number of children receiving their initial vaccination more than I month late. During the study period, the median clinic vaccination coverage level increased from 53% to 89%. 33 Assessment and feedback in private offices also has been reported to improve immunization coverage rates (Houts and Bushneil, Proceedings of the 29th National Immunization Conference). Office-based assessments are described in more detail in a related article in next month's issue (by Paul Darden).

Case Management

A recent randomized trial found that case management that included home visits 2 weeks prior to the dates immunizations were scheduled improved immunization coverage rates for inner-city AfricanAmerican 1 -year-olds in Los Angeles. However, the intervention was not cost-effective, costing more than $12,000 per additional child immunized.34

DISCUSSION

Evidence from published studies provides strong support for sending recall messages to families whose children who are overdue for immunizations and sending reminders in advance of preventive clinic appointments. The literature gives somewhat weaker evidence to support most other interventions to improve immunization delivery to children. The quality of evidence on the interventions we reviewed, and the strength of recommendations we make based on this evidence, is summarized in Table 3.

In real-life office settings, choices among these interventions will depend greatly on existing resources. For example, a physician group that already owns an automated telephone message system would most likely find it more cost-effective to send immunization recall messages by telephone than by mail. A physician group that does not own such a system needs to weigh the potential cost of purchasing one ($1,000 or more) against probable long-term savings in postage and any predicted differences in effectiveness between telephone and mailed messages. Systems to prompt physicians to administer all immunizations due at every visit are likely to be awkward when they rely on chart review but more effective when they rely on computerized immunization tracking systems.

The characteristics of the provider group and the patient population will also be an important factor in choices among interventions. For example, integrated HMO-type systems such as the British National Health Service and the Permanente Medical Group of Northern California may be able to institute incentives to providers more readily than network-model individual practice associations or preferred provider organizations in which one provider may serve many different health plans. In populations in which parents are insured through employment, automated telephone messages sent during the early evening may be more effective than attempts to place personal telephone calls during the day.

Cost-effectiveness is another important consideration in choosing interventions. Reminders in advance of preventive clinic appointments appear to result in cost savings.15'17 For immunization-specific recall messages, automated telephone messages (approximately $10.00 per additional child immunized in one study) appear to be effective and cost-effective relative to mailed messages.13 A lottery incentive to parents resulted in a higher cost per child vaccinated ($9.46 vs. $5.86) compared with a specific prompting message without the incentive.31 There has not been sufficient economic analysis of provider prompting, provider incentives, or assessment and feedback to enable discussion of cost-effectiveness.

Two interventions that may have limited promise in office-based settings are case management involved home visits and parent incentives. Case management that includes home visits appears to be relatively costineffective (approximately $12,000 per additional child immunized in one study).34 Incentives to parents, although intellectually appealing, may be politically unpalatable because this may be perceived as paying parents to receive services they should be seeking for the benefit of their children rather than for financial motives.

For some problems that hamper immunization delivery, it seems unlikely that more evidence will ever become available from specific interventional studies. For example, studies of missed opportunities suggest that giving all possible immunizations simultaneously at the first visit at which they are due would increase coverage rates. However, it would be difficult to design an intervention that could be directed at this problem alone, because messages or incentives to providers or parents to encourage simultaneous administration would likely affect other immunization-related behaviors as well.

It is also important to note that most of the interventions reviewed here can be most effectively implemented in populations that already have a computerized immunization tracking system. The challenges of developing immunization registries are discussed elsewhere in this issue (by Robert Linkins and Suzanne Feikema pp 349-354). but we believe the investment in such systems will yield excellent returns in terms of improving immunization coverage rates.

We conclude that among interventions to improve immunization delivery to children, recall messages to parents are most strongly supported by published evidence on effectiveness and cost-effectiveness. Other interventions, especially provider prompting and incentives to providers, appear promising based on more limited evidence from either the pediatric or non-pediatric literature. Further research on the effectiveness and cost-effectiveness of these interventions is needed.

REFERENCES

1 . Newacheck PW, Stoddard JJ, Hughes DC, et al. Health insurance and access to primary care for children. N Engt J Med. 1998;338:513-519.

2. Ad Hoc Working Croup for the Development of Standards for Pediatric Immunization Practices: Standards for pediatric immunization practices. JAMA. 1993;269:1817-1822.

3. U.S. Preventive Services Task Force: Guide to Clinical Preventive Services, 2nd ed. Baltimore: Williams & Wilkins. 1996.

4. Young SA, Hatpin TJ , Johnson DA, et al. Effectiveness of a mailed reminder on the immunization levels of infants at high risk of failure to complete immunizations. AMJ PuMc Heal*. 1980;70:422-424.

5. Tollestrup K, Hubbard BB. Evaluation of a follow-up system in a county health department's immunization clinic. AmJ PtevMed. 1991;7:24-28.

6. Stehr-Green PA, Dini EF, Lindegren ML, et al. Evaluation of telephoned computer-generated reminders to improve immunization coverage at inner-city clinics. Public Health Reports. 1993;108:426-430.

7. Linkins RW, Dini EF, Watson G, et al. A randomized trial of the effectiveness of computer-generated telephone messages in increasing immunization visits among preschool children. Arch Pediatr Adoiesc Mei 1994;148:908-914.

8. Alto WA, Fury D, Condo A, et al. Imrproving the immunization coverage of children less than 7 years old in a family practice residency. J Am Board Fam Proci. 1994:7:472-477.

9. Abrameon JS, O'Shea TM, Ratledge DL1 et al- Development of a vaccine tracking system to improve the rate of age-appropriate primary immunization in children of lower socioeconomic status. J Pediatr. 1995;126:583-586.

10. Lieu TA, Black SB, Ray P. Computer-generated recall letters for underinununized children: how cost-effective? Pediatr Infect Dis). 1997;16:28-33.

11. Szilagyi PG, Rodewald LE, Savageau J, et al. Improving influenza vaccination rates' in children with asthma: A test of a computerized reminder system and an analysis of factors predicting vaccination compliance. Pediatria. 1992;90:871-875.

1 2. Kemper KJ, Goldberg H. Do computer-generated reminder letters improve the rate of influenza immunization in an urban pediatric clinic? American Journal of Diseases in Children. 1993;147:717-718.

13. Lieu TA, Capra AM, Makol J, et al. Effectiveness and cost-effectiveness of letters, automated telephone messages, ot both for underimmunized children in a health maintenance organization. Pediatrics. 1998: In press.

14- Nazarian LF, Mechabet J , Charney E, et al: Effect of a mailed appointment reminder on appointment keeping. Pediatrics. 1974;53:349-352.

15. Quattlebaum TG, Darden PM, Sperry JB. Effectiveness of computer-generated appointment reminders. Pediatrics. 1991;88:801-805.

16. Campbell JR. Szilagyi PG, Rodewald LE1 et al. Patient-specific reminder letters and pediatric well-child-care show rates. CIm Pediatr. 1994;33:268-272.

17. Dini EF, Linkins RW, Chaney M. Effectiveness of computer-generated telephone messages in increasing clinic visits. Arch PeOKttr Adoiesc Med. 1995; 149:902-905.

18. Levy R. Claravall V. Differential effects of a phone reminder on appointment keeping for patients with long and short between-visit intervals. Medical Care. 1977;15:435-438.

19. Shepard DS, Moseley TA. Mailed versus telephoned appointment teminders to reduce broken appointments in a hospital outpatient department. Medical Care. 1976;14:268-273.

20. Szilagyi PG, Rodewald LE, Humiston SG, et al. Missed opportunities for childhood vaccinations in office practices and the effect on vaccination status. Pediatrics. 1993;91:1-7.

21. Holt E, Guyer B, Hughart N, et al. The contribution of missed opportunities to childhood undetimmunization in Baltimore. Pediatrics. 1996;97:474-480.

22. McConnochie KM, Roghmann KJ. Immunization opportunities missed among urban poor children. Pediatrics. 1 992;89: 1019- 1026.

23. Lieu TA, Black SB, Sorel M, et al. Would better adherence to guidelines improve childhood immunization rates? Pediatrics. 1996;98<6 Pt 1 ):1062-1068.

24. Dietz VJ, Stevenson J, Zell ER, et al. Potential impact on vaccination coverage levels by administering vaccines simultaneously and reducing dropout rates. Arch Pediatr Adoiesc Med. 1994;148:943-949.

25. CDC: Impact of missed opportunities to vaccinate preschool-aged children on vaccination coverage levels- selected U.S. sites, 1991-1992. MMWR. 1994;43: 709-718.

26. Brink SG. Provider reminders: Changing information format to increase infant immunizations. Medical Care. 1989;27:648-653.

27. Szilagyi PG, Rodewald LE, Humiston SG, et al. Reducing missed opportunities for immunizations: Easier said than done. Arch Pediorr Adoiesc Med. 1996; 1 50: 11931200.

28. Harris RP, O'Malley MS, Fletchet SW, et al. Prompting physicians for preventive procedures: A five year study of manual and computer reminders. Am J Pre* Med. 1990;6:145-152.

29. Morrow RW, Gooding AD, Clark C. Improving physician's preventive health care behavior through peer review and financial incentives. Arch Fam Med. 1995:4:165169.

30. Kouides RW, Lewis B, Bennett NM, et al. A performance-based incentive program for influenza immunization in the elderly. Am J Prev Med. 1993;9:250-254.

31. Yokley JM, Glenwick DS. Increasing the immunization of preschool children: An evaluation of applied community interventions. J Appi Behav Anal. 1984; 17: 313-325.

32. Bordley WC, Margolis PA, Lannon CM. The delivery of immunizations and other preventive services in private practice. Pediatrics. 1996;97:467-473.

33. Le Baron CW, Chaney M, Baughman AL, et al. Impact of measurement and feedback on vaccination coverage in public clinics, 1988-1994. JAMA, 1997:277:631635.

34- Wood D, Halfon N, Donald-Sherboume C, et al. Increasing immunization rates among inner-city, African American children. JAMA. 1998;279:29-34.

TABLE 1

Rating Systems for Quality of Evidence and Strength of Recommendations, Modified from US Preventive Services Task Force

TABLE 2

Studies of the Effectiveness on Immunizaation-Specific Messages to Parents

TABLE 2

Studies of the Effectiveness on Immunizaation-Specific Messages to Parents

TABLE 3

Quality of Evidence and Strength of Recommendations for Interventions to Improve Childhood Immunization Delivery

10.3928/0090-4481-19980601-10

Sign up to receive

Journal E-contents