Skip directly to search Skip directly to A to Z list Skip directly to site content
CDC Home

The content, links, and pdfs are no longer maintained and might be outdated.

  • The content on this page is being archived for historic and reference purposes only.
  • For current, updated information see the MMWR website.

Evaluation of Vaccination Recall Letter System for Medicaid-Enrolled Children Aged 19–23 Months — Montana, 2011

Reminder and recall systems alert the parents of children due (reminder) or overdue (recall) for vaccination and have been associated with increased vaccination coverage (1–3). To evaluate the potential of a state-generated recall letter to increase vaccination coverage among Montana children, the Montana Department of Public Health and Human Services (DPHHS) pilot tested a recall letter system targeted at parents of children aged 19–23 months enrolled in Montana Medicaid and not known to have completed a subset of the routinely recommended vaccination series. Data extracted from Medicaid billing records and the web-based immunization registry database (WIZRD) then in use by Montana were used to ascertain whether children were up-to-date for the study vaccination series. Of the 1,865 children enrolled in Montana Medicaid and aged 19–23 months, 878 (47%) were eligible for study participation. One recall letter was sent to parents of 438 (50%) eligible children selected randomly. A reassessment of each child's vaccination status was completed 3 months after the initial mailing. At 3 months, 32% of children whose parents were sent letters were known to have completed the study vaccination series, which was not significantly different from the 28% of children who were vaccinated but whose parents had not been sent letters. Further research is needed to determine why the recall letter had limited effectiveness in this pilot study and to develop more effective methods for increasing vaccination coverage in Montana.

The Advisory Committee on Immunization Practices recommends that children aged 0–18 months receive routine vaccinations for protection against diseases caused by 14 pathogens (4). Despite these recommendations, the National Immunization Survey reported that in 2009, for children aged 19–35 months, the estimated vaccination coverage nationally for the recommended modified series (the recommended series with Haemophilus influenzae type b conjugate vaccine [Hib] excluded because of a Hib shortage*) was just over 70% and coverage varied substantially among states (5). In Montana, the estimated coverage for the recommended modified series among children aged 19–35 months was 61.7%, ranking among the lowest 10 states. To improve vaccination coverage, the Task Force on Community Preventive Services recommends the use of reminder and recall systems (2). Vaccine reminder and recall systems alert the parents of children due or overdue for vaccinations and are effective at increasing child and adult vaccination coverage whether conducted by a health-care provider, academic center, or health department (3). The Montana DPHHS does not use a vaccine reminder and recall system of its own, relying instead on vaccine providers to contact parents of children overdue for vaccination. However, among surveyed health-care providers who provide vaccines to Montana adolescents, only 21% reported using reminder and recall systems. In response, DPHHS pilot tested a state-generated recall letter that was sent to parents of Medicaid-enrolled children aged 19–23 months and not known to be fully immunized with the study vaccination series.

Children enrolled in Montana Medicaid with birthdates from December 2, 2008, through May 1, 2009, were assessed for coverage with the study vaccination series. For these children, data were extracted from Medicaid billing records and WIZRD and imported into the Comprehensive Clinic Assessment Software Application.§ Medicaid billing data were extracted on December 28, 2010, and included data entered through December 1, 2010. Children known to have received each of the vaccines in the study vaccination series or those with home addresses outside of Montana were excluded from study participation. The study was powered to have a 99.9% likelihood and a 72% likelihood of detecting a statistically significant difference given a 15 percentage-point difference and 6 percentage-point difference, respectively, between the intervention and control cohorts, assuming 250 children per cohort, α = 0.05, and a two-sided test.

Using the Comprehensive Clinic Assessment Software Application random number generator tool, 50% of children not known to have completed the study vaccination series on December 1, 2010, were randomly assigned to the intervention cohort. On January 21, 2011, using addresses from Montana Medicaid, a letter was mailed to the parent(s) of each child reminding them to take their child to their health-care provider to receive the missed vaccines. The letters did not include an individualized listing of the missed vaccines. The remaining 50% of children were assigned to the control cohort (i.e., no letter). Letters returned as undeliverable were resent to addresses listed in WIZRD if different from the address listed in the Medicaid database. Letters were not resent if the Medicaid and WIZRD addresses were identical. In June 2011, a reassessment of vaccination status for each child was completed using the methodology for vaccines received through April 30, 2011. Pearson's chi-square test was used to evaluate the difference in participant characteristics, vaccines received by the intervention and control cohorts, and coverage for each cohort between baseline and follow-up.

Of the 1,865 children enrolled in Montana Medicaid and aged 19–23 months by December 1, 2010, a total of 878 (47%) were eligible for study participation (Table 1). Of those, 464 (53%) were male, and the median age was 21 months. Among the participants, 184 (21%) children were classified as American Indian/Alaska Native (AI/AN). Race information was not available for the other participants. The county of residence was categorized as rural or frontier for 87% of participants. Among participants, 357 (41%) had not received at least one or two of the recommended vaccines. The vaccines most commonly missing were the fourth dose of diphtheria and tetanus toxoids and acellular pertussis vaccine (DTaP), which 612 (70%) participants had not received, and the fourth dose of pneumococcal vaccine (PCV), which 539 (61%) participants had not received. No significant differences existed between the intervention and control cohorts for age, sex, AI/AN classification, population density for county of residence, and number of missing vaccines. Recall letters were sent to parents of 438 (50%) children; 83 (20%) of those letters initially were returned as undeliverable, of which 45 were resent.

Three months after the single recall letter was sent, 139 (32%) of the children whose parents had been sent a recall letter had completed the study vaccination series and 125 (28%) of control children had completed the series (p=0.28) (Table 2). For 14 (70%) of 20 vaccinations, the percentages of children who received the missing vaccine by 3 months was higher in the intervention cohort compared with the control cohort, but the difference was only statistically significant for the third and fourth doses of PCV. No significant differences were found between the cohorts for the percentage of 184 AI/AN children who completed the study vaccination series (intervention = 40.4%, control = 29.4%; p=0.12). Likewise, no significant differences were found when cohorts were stratified by county of residence for the 110 urban children (intervention = 34.4%, control = 43.5%; p=0.34), 537 rural children (intervention = 30.5%, control = 23.1%; p=0.06), and 231 frontier children (intervention = 33.3%, control = 34.1%; p=0.9). In this study, 30 recall letters would need to be sent to result in one extra child being up-to-date for the study vaccination series (95% confidence interval = 10.6–∞).

Reported by

Cody L. Custis, MS, Steven D Helgerson, MD, James S. Murphy, Montana Dept of Public Health and Human Svcs. Carolyn A. Parry, MPH, Immunization Svcs Div, National Center for Immunization and Respiratory Diseases; Randall J. Nett, MD, Office of Public Health Preparedness and Response, CDC. Corresponding contributor: Randall J. Nett, gge5@cdc.gov, 406-444-5917.

Editorial Note

The findings in this study demonstrate that a single, state-generated recall letter to parents resulted in no significant increase in vaccination coverage among predominantly rural, Medicaid-enrolled children aged 19–23 months. Of children whose parents were not sent recall letters, 28% had completed the study vaccination series at 3 months. In comparison, 32% of children whose parents had been sent letters had completed the series.

Reminder and recall systems have been shown to be effective in increasing vaccination coverage in pediatric and adult populations; for universally recommended vaccines and targeted vaccines; when conducted by a health-care provider, an academic center, or a health department; and, when carried out using postcards, mailed letters, or telephone calls (3,6). However, as found in this study, specific reminder and recall systems and methods are not effective in every setting. For example, among urban adolescent populations, text message reminders have been shown to significantly increase vaccination coverage while automated telephone messages have not (7,8).

Previous studies have shown the effectiveness of certain reminder and recall systems in rural settings. Reminder postcards were effective in improving vaccination coverage among a predominantly low-income, rural, and Latino pediatric population (9). That study differed from the investigation presented here in that the population was predominantly Hispanic, the system was community health center–based, and multiple mailings were used. In another study, automated telephone reminders and recalls conducted by rural county health departments in Georgia were effective at increasing immunization visits (6). Unlike the Montana investigation, the Georgia study used multiple attempts until contact was made with the parent. These findings highlight the importance of the exact methods chosen to implement a reminder and recall system.

Selecting the method most likely to be effective in a particular community might require pilot testing and an evaluation of the results. The findings of this investigation suggest that studies conducted in suburban and urban areas might not predict the success of interventions implemented in rural areas and certain types of reminder and recall systems might not be effective in rural settings. Compared with urban populations, rural populations are likely to be less educated, less affluent, and have less access to transportation (10); these factors and others might influence childhood vaccination coverage and the effectiveness of certain vaccine reminder and recall methods.

The findings in this report are subject to at least seven limitations. First, recall letters were not sent by certified mail; therefore, no confirmation that the intended recipients received the letters was obtained. A low percentage of successfully delivered letters might diminish the difference in vaccination coverage between the intervention and control cohorts. Second, an average delay of 4 weeks occurs between administration of a vaccine and Montana Medicaid's receipt of the health-care provider's billing statement. However, health-care providers have up to 1 year to bill Medicaid for vaccines administered, so delays in billing for some vaccines might hide some differences in vaccination coverage between intervention and control cohorts. Third, only 93% of public health-care providers and 74% of private health-care providers are known to be active users of WIZRD (DPHHS, unpublished data, 2011). Therefore, the immunization rates presented in this study might be underestimated. Fourth, children with delayed initiation of the PCV or Hib series might have been eligible to receive fewer doses of those vaccines and thus be considered up-to-date per Advisory Committee on Immunization Practices recommendations but underimmunized for PCV or Hib for this study. Fifth, only a single recall letter was sent; the use of multiple letters might have resulted in higher vaccination coverage. Sixth, the children sent letters might have differed from the children not sent letters regarding certain factors that were not assessed; these factors might have diminished the impact of the letters in increasing vaccination coverage. Finally, the medical records of study participants were not available for review; thus, the completeness of the vaccination status for each child cannot be confirmed.

This intervention aimed at increasing vaccination coverage among children enrolled in Montana Medicaid by mailing a single, state-generated vaccine recall letter to their parents resulted in no significant increase in vaccination coverage of their children. Based on these findings and a review of the literature, 1) health-care providers should use reminder and recall systems to improve vaccination coverage among their patients (1,2); 2) state and local health departments should use the reminder and recall system(s) most likely to improve vaccination coverage in their population; 3) users of reminder and recall systems should evaluate their system to determine its effectiveness and adjust their strategy as needed to improve system performance; and 4) public health authorities should conduct further research to identify effective reminder and recall system(s) for improving vaccination coverage, particularly in rural underserved areas.

Acknowledgments

Carol Ballew, PhD, Eric Higginbotham, Bekki Wehner, Kathleen Grady, Vicci Stroop, Heather Zimmerman, MPH, Montana Dept of Public Health and Human Svcs.

References

  1. CDC. Recommendations of the Advisory Committee on Immunization Practices, the American Academy of Pediatrics, and the American Academy of Family Physicians: use of reminder and recall by vaccination providers to increase vaccination rates. MMWR 1998;47:715–7.
  2. CDC. Vaccine-preventable diseases: improving vaccination coverage in children, adolescents, and adults. A report on recommendations from the Task Force on Community Preventive Services. MMWR 1999;48(No. RR-8).
  3. Jacobson VJ, Szilagyi P. Patient reminder and patient recall systems to improve immunization rates. Cochrane Database Syst Rev 2005(3):CD003941.
  4. CDC. General recommendations on immunization—recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR 2011;60(No. RR-2).
  5. CDC. National, state, and local area vaccination coverage among children aged 19–35 months—United States, 2009. MMWR 2010;59:1171–7.
  6. Linkins RW, Dini EF, Watson G, Patriarca PA. A randomized trial of the effectiveness of computer-generated telephone messages in increasing immunization visits among preschool children. Arch Pediatr Adolesc Med 1994;148:908–14.
  7. Stockwell MS, Karbanda EO, Martinez RA, et al. Text4Health: impact of text message reminder-recalls for pediatric and adolescent immunizations. Am J Public Health 2012;102:e15–21.
  8. Szilagyi PG, Schaffer S, Barth R, et al. Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial. Arch Pediatr Adolesc Med 2006;160:157–63.
  9. Hicks P, Tarr GA, Hicks XP. Reminder cards and immunization rates among Latinos and the rural poor in northeast Colorado. J Am Board Fam Med 2007;20:581–6.
  10. National Organization of State Offices of Rural Health. National rural health issues. Sterling Heights, MI: National Organization of State Offices of Rural Health; 2006. Available at http://www.nosorh.org/pdf/Rural_Impact_Study_States_IT.pdf. Accessed October 10, 2012.

* The modified series, excluding Haemophilus influenzae type b conjugate vaccine (Hib), includes ≥4 doses of diphtheria, tetanus toxoid, and acellular pertussis vaccine (DTaP)/diphtheria and tetanus toxoids vaccine (DT)/diphtheria and tetanus toxoids and pertussis vaccine (DTP); ≥3 doses of inactivated poliovirus vaccine (IPV); ≥1 dose of measles antigen-containing vaccine (MMR); ≥3 doses of hepatitis B vaccine (HepB); ≥1 doses of varicella vaccine (VAR); and ≥4 doses of pneumococcal vaccine (PCV). Hib vaccine was excluded from national reporting of the vaccine series because of the Hib vaccine shortage that occurred during 2007–2009.

The study vaccination series includes ≥4 doses of DTaP; ≥3 doses of IPV; ≥1 dose of MMR; ≥4 doses of Hib; ≥3 doses of HepB; ≥1 dose of VAR; and ≥4 doses of PCV.

§ Coverage for Hib vaccine for the primary series was based on receipt of ≥2 or ≥3 doses, depending on product received. The Merck Hib vaccines require a 2-dose primary series with doses at ages 2 months and 4 months, and the Sanofi Pasteur Hib vaccines require a 3-dose primary series with doses at ages 2, 4, and 6 months. Coverage for the full series, which includes the primary series and a booster dose, was based on receipt of ≥3 or ≥4 doses, depending on product received. Both Merck and Sanofi Pasteur Hib vaccines require a booster dose at ages 12–15 months (5). The number of Hib doses a child is eligible to receive is dependent upon the vaccine type, the age at series initiation, and the age at which the doses are administered. The number of PCV doses a child is eligible to receive is dependent upon the age at series initiation and the age at which the doses are administered. Therefore, children might not have been eligible to receive the number of Hib and/or PCV doses needed to be considered up-to-date for the purposes of this study.

Frontier is defined as ≤6 persons per square mile and either ≥50 miles or 60-minute drive to essential services. Additional information is available at http://www.raconline.org/topics/frontier/frontierfaq.php#definition, and at http://www.nal.usda.gov/ric/ricpubs/what_is_rural.shtml.


What is already known on this topic?

The use of reminder and recall systems by health-care providers, academic centers, and health departments has been shown to be associated with increased vaccination coverage.

What is added by this report?

A single, state-generated recall letter did not significantly improve vaccination coverage in a rural, underserved, and underimmunized pediatric population in Montana.

What are the implications for public health practice?

Users of reminder and recall systems should adapt the system for the targeted patient population. Reminder and recall systems should be evaluated regularly to determine their effectiveness and modified, if necessary.


TABLE 1. Participant characteristics at initiation of recall letter study among Montana Medicaid-enrolled children aged 19–23 months — Montana, 2011

Characteristic

Total

Intervention cohort

Control cohort

p-value*

No.

(%)

No.

(%)

No.

(%)

No. of participants

878

(100)

438

(50)

440

(50)

Sex

0.46

Male

464

(53)

237

(54)

227

(52)

Female

414

(47)

201

(46)

213

(48)

Median age

21 mos

21 mos

21 mos

American Indian/Alaska Native§

184

(21)

89

(20)

95

(22)

0.64

County of residence

0.09

Urban

110

(13)

64

(15)

46

(10)

Rural

537

(61)

269

(61)

268

(61)

Frontier

231

(26)

105

(24)

126

(29)

No. of missing vaccines

0.96

1–2

357

(41)

175

(40)

182

(41)

3­–5

204

(23)

101

(23)

103

(24)

6–10

121

(14)

62

(14)

59

(13)

11–20

196

(22)

100

(23)

96

(22)

% vaccinated with selected vaccines**

DTaP fourth dose

266

(30)

138

(32)

128

(29)

0.44

HepB third dose

523

(60)

253

(58)

270

(61)

0.28

Hib fourth dose††

424

(48)

211

(48)

213

(48)

0.94

IPV third dose

539

(61)

263

(60)

276

(63)

0.41

MMR first dose

507

(58)

246

(56)

261

(59)

0.34

PCV fourth dose§§

339

(39)

170

(39)

169

(38)

0.90

VAR first dose

459

(52)

221

(50)

238

(54)

0.28

Abbreviations: DTaP =diphtheria, tetanus toxoid, and acellular pertussis vaccine; HepB = hepatitis B vaccine; Hib = Haemophilus influenzae type b conjugate vaccine; IPV = inactivated poliovirus vaccine; MMR = measles, mumps, and rubella vaccine; PCV = pneumococcal vaccine; VAR = varicella vaccine.

* p-value calculated using chi-square test.

Children living in Montana, enrolled in Montana Medicaid, with birthdates December 2, 2008–May 1, 2009, and not known to have received each of the following: ≥4 doses of DTaP; ≥3 doses of IPV; ≥1 dose of MMR; ≥4 doses of Hib; ≥3 doses of HepB; ≥1 dose of VAR; and ≥4 doses of PCV.

§ Data for other race classifications were not available.

Categories based on U.S. Department of Agriculture descriptions, available at http://www.nal.usda.gov/ric/ricpubs/what_is_rural.shtml.

** The individual vaccines in this analysis represent the last possible dose in the selected vaccine's series that can be administered to a child aged 18 months.

†† Coverage for Hib vaccine for the primary series was based on receipt of ≥2 or ≥3 doses, depending on product received. The Merck Hib vaccines require a 2-dose primary series with doses at ages 2 months and 4 months, and the Sanofi Pasteur Hib vaccines require a 3-dose primary series with doses at ages 2, 4, and 6 months. Coverage for the full series, which includes the primary series and a booster dose, was based on receipt of ≥3 or ≥4 doses, depending on product received. The Merck and Sanofi Pasteur Hib vaccines require a booster dose at ages 12–15 months. The number of Hib doses a child is eligible to receive depends on the vaccine type, the age at series initiation, and the age at which the doses are administered. Therefore, children might not have been eligible to receive the number of Hib doses needed to be considered up-to-date for this study.

§§ The number of PCV doses a child is eligible to receive depends on the age at series initiation and the age at which the doses are administered. Therefore, children might not have been eligible to receive the number of doses needed to be considered up-to-date for this study.


TABLE 2. Children eligible to receive the study vaccination series* and individual vaccines at baseline and number of eligible children vaccinated at 3 months following the mailing of a vaccine recall letter to parents of Montana Medicaid-enrolled children aged 19–23 months — Montana, 2011

Study vaccination series

Intervention cohort

Control cohort

Odds ratio

(95% CI)

Eligible to receive vaccine at baseline

Eligible children vaccinated at 3 mos

Eligible to receive vaccine at baseline

Eligible children vaccinated at 3 mos

No.

No.

(%)

No.

No.

(%)

Participants

438

139

(32)

440

125

(28)

1.2

(0.9–1.6)

DTaP

First dose

63

14

(22)

63

14

(22)

1.0

(0.4–2.3)

Second dose

101

22

(22)

93

16

(17)

1.3

(0.7–2.7)

Third dose

151

44

(29)

140

28

(20)

1.7

(1.0–2.8)

Fourth dose

300

101

(34)

312

102

(33)

1.0

(0.7–1.5)

HepB

First dose

70

11

(16)

57

14

(25)

0.6

(0.2–1.4)

Second dose

104

37

(36)

87

39

(45)

0.7

(0.4–1.2)

Third dose

185

95

(51)

170

90

(53)

0.9

(0.6–1.4)

Hib§

First dose

72

15

(21)

68

12

(18)

1.2

(0.5–2.9)

Second dose

91

11

(12)

95

12

(13)

1.0

(0.4–2.3)

Third dose

148

43

(29)

146

35

(24)

1.3

(0.8–2.2)

Fourth dose

227

47

(21)

227

39

(17)

1.3

(0.8–2.0)

IPV

First dose

76

14

(18)

69

13

(19)

1.0

(0.4–2.2)

Second dose

106

25

(24)

102

19

(19)

1.3

(0.7–2.6)

Third dose

175

51

(29)

164

35

(21)

1.5

(0.9–2.5)

MMR

First dose

192

60

(31)

179

50

(28)

1.2

(0.8–1.8)

PCV

First dose

86

18

(21)

79

13

(16)

1.3

(0.6–3.0)

Second dose

110

24

(22)

107

16

(15)

1.6

(0.8–3.2)

Third dose

170

51

(30)

157

29

(18)

1.9

(1.1–3.2)

Fourth dose

268

49

(18)

271

26

(10)

2.1

(1.3–3.5)

VAR

First dose

217

62

(29)

202

47

(23)

1.3

(0.9–2.0)

Abbreviations: DTaP = diphtheria, tetanus toxoid, and acellular pertussis vaccine; HepB = hepatitis B vaccine; Hib = Haemophilus influenzae type b conjugate vaccine; IPV = inactivated poliovirus vaccine; MMR = measles, mumps, and rubella vaccine; PCV = pneumococcal vaccine; VAR = varicella vaccine; CI = confidence interval.

* The study vaccination series includes ≥4 doses of diphtheria, tetanus toxoid, and acellular pertussis vaccine (DTaP); ≥3 doses of inactivated poliovirus vaccine (IPV); ≥1 dose of measles, mumps, and rubella vaccine (MMR); ≥4 doses of Haemophilus influenzae type b conjugate vaccine (Hib); ≥3 doses of hepatitis B vaccine (HepB); ≥1 dose of varicella vaccine (VAR); and ≥4 doses of pneumococcal vaccine (PCV).

Children living in Montana, enrolled in Montana Medicaid, with birthdates December 2, 2008–May 1, 2009, and not known to have received each of the following: ≥4 doses of DTaP; ≥3 doses of IPV; ≥1 dose of MMR; ≥4 doses of Hib; ≥3 doses of HepB; ≥1 dose of VAR; and ≥4 doses of PCV.

§ Coverage for Hib vaccine for the primary series was based on receipt of ≥2 or ≥3 doses, depending on product type received. The Merck Hib vaccines require a 2-dose primary series with doses at ages 2 months and 4 months, and the Sanofi Pasteur Hib vaccines require a 3-dose primary series with doses at ages 2, 4, and 6 months. Coverage for the full series, which includes the primary series and a booster dose, was based on receipt of ≥3 or ≥4 doses, depending on product type received. The Merck and Sanofi Pasteur Hib vaccines require a booster dose at ages 12–15 months. The number of Hib doses a child is eligible to receive depends on the vaccine type, the age at series initiation, and the age at which the doses are administered. Therefore, children might not have been eligible to receive the number of Hib doses needed to be considered up-to-date for this study.

The number of PCV doses a child is eligible to receive depends on the age at series initiation and the age at which the doses are administered. Therefore, children might not have been eligible to receive the number of doses needed to be considered up-to-date for this study.


Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of Health and Human Services.

References to non-CDC sites on the Internet are provided as a service to MMWR readers and do not constitute or imply endorsement of these organizations or their programs by CDC or the U.S. Department of Health and Human Services. CDC is not responsible for the content of pages found at these sites. URL addresses listed in MMWR were current as of the date of publication.


All MMWR HTML versions of articles are electronic conversions from typeset documents. This conversion might result in character translation or format errors in the HTML version. Users are referred to the electronic PDF version (http://www.cdc.gov/mmwr) and/or the original MMWR paper copy for printable versions of official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices.

**Questions or messages regarding errors in formatting should be addressed to mmwrq@cdc.gov.

 
USA.gov: The U.S. Government's Official Web PortalDepartment of Health and Human Services
Centers for Disease Control and Prevention   1600 Clifton Road Atlanta, GA 30329-4027, USA
800-CDC-INFO (800-232-4636) TTY: (888) 232-6348 - Contact CDC–INFO
A-Z Index
  1. A
  2. B
  3. C
  4. D
  5. E
  6. F
  7. G
  8. H
  9. I
  10. J
  11. K
  12. L
  13. M
  14. N
  15. O
  16. P
  17. Q
  18. R
  19. S
  20. T
  21. U
  22. V
  23. W
  24. X
  25. Y
  26. Z
  27. #