The 2007 National Survey of Children’s Health reported that nationwide, 15% of children 4 months to 5 years of age are at moderate risk for developmental, behavioral, or social delays; 10% are at high risk.
In children who have been identified as having developmental delays, those who receive early intervention services are more likely to experience higher levels of education, lower dropout rates, and higher rates of full-time employment in adulthood.
The American Academy of Pediatrics (AAP) 2006 Developmental Screening and Surveillance Policy Statement recommends developmental screening at age 9 months and 18 months and at either 24 or 30 months, and developmental surveillance at every well-child preventive care visit for children from birth through age 5 years.3 Pediatric clinicians continue to incorporate standardized screening into primary care practice. Provider-reported use of at least one screening tool in practice increased from 23% of visits in 2002 to 47.7% in 2009.4 Multiple commonly used developmental screening tools are available, one of which is the Child Development Review (CDR).3
Previous studies have investigated how developmental screening impacts the identification and referral of children at risk for developmental delays in general pediatric populations.5–7 However, these studies have not evaluated whether clinicians use developmental screening tools in accordance with the recommended instructions or how thoroughly they document and bill for use of the screening instrument.
This study examined the implementation of the CDR screen in a large, single-site pediatric practice where distribution of the screening tool was standardized and the clinicians received prestudy education on the correct utilization of CDR. The study objectives were to evaluate whether clinicians completed and interpreted the CDR according to the recommended instructions; describe clinicians’ documentation and billing rate for developmental screening; evaluate the effect of demographic factors on CDR screen failure rates; and compare clinician opinions of their own competency using the CDR for developmental screening before and after the study period.
The institutional review boards of the University of Minnesota and Mayo Clinic deemed the present quality improvement project exempt from any approval requirement.
The project setting was an academic center–based pediatric primary care clinic and clinical training site for pediatric residents in Rochester, MN. Clinicians consisted of 13 attending pediatricians, six pediatric nurse practitioners, and 35 pediatric residents.
Well-child visits at this clinic typically last 15 minutes. Patients are asked to arrive 20 minutes before their appointment time with their clinician to complete the nurse assessment and parent-completed questionnaires. Well-child visits can be extended to 30 minutes for children with complex medical needs; those with considerable additional acute medical concerns; and those who are seeing a medical resident in the continuity clinic.
Children with government insurance receive well-child visits at intervals recommended by the AAP.8 For children with private insurance, visits at age 6, 12, and 18 months, as well as age 4 years, are not part of the routine well-child care schedule.
Historically in this practice, clinicians have been encouraged to use the CDR at the 4-month, 9-month, 15-month, 3-year, and 5-year well-child visits. However, before the study, it was not standard to bill patients for CDR administration and interpretation. This clinic also mails the Ages and Stages Questionnaire to all parents of children 18 months of age.
Derived from the Child Development Inventories (CDI) (previously known as the Minnesota Child Development Inventories), the CDR is a screening instrument used to identify children with developmental delays, behavior problems, and health problems.9 The CDR is a research-based developmental screening tool designed for the provider to gather information from the caregiver regarding the child’s health and development.9 The CDR questionnaire consists of two distinct forms, with the Infant Development Inventory for infants 0 to 18 months and the Child Development Review Parent Questionnaire for children 18 months to 5 years. Both questionnaires contain a front side with open- and close-ended questions and a back side with a list of developmental milestones appropriate for chronological age.
In a study comparing the 300-item CDI to two validated developmental assessment instruments (the Bayley Scales of Infant Development and the Clinical Adaptive Test/Clinical Linguistic and Auditory Milestone Scale), the sensitivity of the CDI was 80% and 100%; the specificity was 96% and 94%.10 When compared with the gold standard of the Battelle Developmental Inventory in a population of 18-month-old general pediatric patients, the positive predictive value with the CDI was 50% vs 34% with the Ages and Stages Questionnaire.7
The CDR is available in English and Spanish, and in an electronic version.11 For this study, only the CDR in English was used; interpreters assisted non–English-speaking families with the forms.
To complete the CDR, the caregiver is asked about milestones at half the child’s age and proceeds until there are three skills listed in a row that the child has not attained. A passing score is indicated by a completed CDR where the child has attained skills in all developmental domains at or above 70% of the expected skills according to the child’s age, adjusted for prematurity. A failed screen, indicated by a child not attaining a skill in one or more domains over the 70% line, should result in further developmental assessment.
To better differentiate between an incomplete and a failed CDR, study investigators created a third completion category of “probable pass.” Probable pass was defined by at least two check marks above the 70% line in each domain.
Clinician Survey Procedure
Identical pre- and post-study questionnaires consisted of 10 questions to assess the attitudes and practice of clinicians regarding developmental screening and use of the CDR. All 54 participating clinicians were asked to answer survey items using a five-point Likert scale.
After completion of the presurvey, clinicians participated in a 30-minute educational training session on developmental screening and use of the CDR. Clinicians not in attendance were provided the pre-survey and met individually with one of the investigators to review the training session content. All clinicians were sent an electronic communication that included a summary document with flow sheet of the study protocol.
Before project implementation, investigators met with the office management staff to plan the logistical flow for clinical administration of the CDR. A 1-hour training session was provided for the clinic nursing staff, explaining the importance of developmental screening, the use of the CDR, and their role in the study. Office staff were responsible for ensuring that each caregiver with a child age 5 years or younger being seen for a well-child visit received the CDR for completion. Nursing staff encouraged caregivers to complete the CDR before the provider visit and collected the CDR after the clinician review.
Clinicians were responsible for ensuring that the CDR was completed, results were reviewed with caregivers, and children with failed screening were referred for further developmental assessment. In addition, clinicians were to document and submit billing for screening services and return the screening instrument for study collection. Midway through the study, clinicians were sent a report on study metrics and the areas needing improvement.
For all children screened with the CDR during the 3-month study period, electronic medical records and completed CDR questionnaires were reviewed by one investigator to assess for: correct CDR completion; correct interpretation of appropriately completed screens; appropriate referral of patients who did not pass the CDR screen to specialized developmental services; documentation of the use and the result of the CDR screen in the electronic medical record; and appropriate billing. When questions arose regarding data collection, individual charts and CDR questionnaires were additionally reviewed by the co-investigators.
All statistical analyses were completed using JMP software version 9.0 (SAS Institute Inc. Cary, NC). The Likert scale responses from the pre- and poststudy survey were grouped into either agree (strongly agree or agree) or disagree (neutral, disagree, or strongly disagree), and group comparisons were performed with the McNemar test. Descriptive statistics were used to analyze demographic variables. In addition, the demographic risk factors of sex, race, and insurance type were analyzed for an association with CDR screen failure through univariate logistic regression models. Subsequent contrast comparisons within the demographic variables of sex, race, and insurance type were completed.
Of the 1,648 administered CDR screens, 1,335 (81%) were returned. Most patients were white (57%) and had institution-based insurance or private insurance (68% combined). Males (P<.009) and those with government-issued insurance (P<.005) were significantly more likely to not pass the CDR screen.
The CDR was completed correctly for 43% of visits. Documentation was completed for 89.1% of visits and accurate billing was submitted for 71.1% of visits. Of all children screened, 41 children (3.1%) did not pass the screening, and 22 (1.9%) of these were referred to specialized developmental services for further assessment. Three children had documentation of a pass on the CDR but were referred to specialized developmental services, and 24 children (1.8%) were already involved with specialized developmental services before the study.
A total of 45 clinicians (83.3%) returned both pre- and post-surveys. After training, clinicians reported more comfort with interpreting (P<.001), documenting (P< .001), referring (P<.008), and billing (P< .001) using the CDR. Nearly 98% of clinicians reported that developmental screening is an important component to well-child visits and the use of a developmental screening tool is an important element in assessing for developmental delays.
Developmental screening instruments have been increasingly used in pediatric primary care during the past decade.4 The 2006 AAP Policy Statement on Developmental Screening outlined a recommended screening schedule with surveillance at other well-child visits and lists appropriate instruments, including the CDR. This study’s correct CDR completion rate of 43% suggests that clinicians may not consistently be using the CDR as it was designed.
Developmental delays in children are more likely to be recognized using a standardized tool than with surveillance or systematic surveillance alone.3,12 When completing developmental screening instruments at the point of care, which is what is currently recommended and reimbursed,3 the clinician faces time constraints and lack of office personnel that can limit the effectiveness of screening.4 Many clinicians likely modified the instrument, as indicated by our probable pass rate of 72%. Correct completion of screening tools is an important outcome metric that should be measured in future studies of developmental screening implementation by general pediatric clinics.
The prevalence of developmental disability has risen from 12.84% to 15.04% in the past 12 years.13 This rate is significantly higher than the present study’s findings that 1.8% of children were connected with developmental services before screening and 3.1% did not pass the CDR screening and were identified as at risk for developmental delay.
One plausible reason for this discrepancy is that this study’s population was primarily privately insured and many were dependent children of employees at this institution. However, it is also possible that the prevalence was lower because of incorrect use of the CDR, resulting in an underestimated true failure rate. Clinicians who incorrectly use the questions on the CDR as a form of surveillance may put patients at risk for underdetection of developmental delays.14 The third reason for the low screen failure rate could be that clinicians who disagreed with the results of the instrument chose to monitor the child rather than document failure. King and colleagues6 reported that children who do not pass the developmental screening tools are appropriately referred only 61% of the time. Future studies should investigate the factors that influence clinicians’ decisions to refer after failed developmental screening and the outcomes of those children who are referred.
Previous studies have demonstrated that, compared with private insurance, government insurance is associated with an increased risk for developmental delay and screening failure.5,13,15 Boyle and colleagues13 identified an almost doubled prevalence of any reported developmental disability among children insured by Medicaid compared with children insured by private insurance. From a public health perspective, disadvantaged children who qualify for Medicaid coverage may benefit from a more robust developmental screening schedule than children with private insurance.
Consistent with previous studies,5,13 this study found that male children were significantly more likely to not pass the CDR screen. Hix-Small and colleagues5 reported that 73% of their referred patients were male. Boyle and colleagues13 also noted a higher prevalence of developmental delays in boys than girls (18.4% vs 9.5%). There are limited data suggesting why more males than females are identified with developmental delays; further research is needed to identify the potential causes for this finding.
Many developmental screening questionnaires are available online, including the CDR.11 The use of electronic screening instruments could help improve the correct completion rate of questionnaires but would require time and effort to develop and test data entry formats and scoring algorithms.16 More investigation is needed to identify the current rate and effectiveness of electronic developmental screening using different screening instruments.
The main strength of the present study is that it is a large prospective study with investigator review of all CDR screens. A majority of the children in the study sample were white and privately insured, which may limit the findings’ scope of application. The large number of incomplete CDR questionnaires limited the ability to analyze the risk factors for screen failure and the outcomes of screening in the study.
Developmental screening with the CDR is well-accepted by clinicians. In the present study, it identified 3.1% of children at risk for developmental delay, with a disproportionate number of these children having government insurance. High-risk populations may benefit from more frequent developmental screening than low-risk populations. Future prospective studies are recommended to explore optimal screening frequencies and implementation strategies for different pediatric populations.
Standardized developmental screening is recommended and can be completed in general pediatric practice with multiple instruments, including the CDR. Screening instruments need to be completed and interpreted correctly to optimize the detection of children who have developmental delays. The completion of electronic developmental screening questionnaires before a well-child appointment, combined with improved education for parents, nursing staff, and clinicians on how to complete screening tools correctly, may improve the use of developmental screening and should be further evaluated.
- The National Survey of Children’s Health. Portland (OR): The Child and Adolescent Health Measurement Initiative. Data Resource Center for Child and Adolescent Health. 2007 NSCH data search [cited 2011 Nov 5]. Available at: www.childhealthdata.org. Accessed Feb. 15, 2012.
- Reynolds AJ, Temple JA, Ou SR, et al. Effects of a school-based, early childhood intervention on adult health and well-being: a 19-year follow-up of low-income families. Arch Pediatr Adolesc Med. 2007;161(8):730–739. doi:10.1001/archpedi.161.8.730 [CrossRef]
- Council on Children With Disabilities; Section on Developmental Behavioral Pediatrics; Bright Futures Steering Committee; Medical Home Initiatives for Children With Special Needs Project Advisory Committee. Identifying infants and young children with developmental disorders in the medical home: an algorithm for developmental surveillance and screening. Pediatrics. 2006;118(1):405–420.
- Radecki L, Sand-Loud N, O’Connor KG, Sharp S, Olson LM. Trends in the use of standardized tools for developmental screening in early childhood: 2002–2009. Pediatrics. 2011;128(1):14–19. doi:10.1542/peds.2010-2180 [CrossRef]
- Hix-Small H, Marks K, Squires J, Nickel R. Impact of implementing developmental screening at 12 and 24 months in a pediatric practice. Pediatrics. 2007;120(2):381–389. doi:10.1542/peds.2006-3583 [CrossRef]
- King TM, Tandon SD, Macias MM, et al. Implementing developmental screening and referrals: lessons learned from a national project. Pediatrics. 2010;125(2):350–360. doi:10.1542/peds.2009-0388 [CrossRef]
- Rydz D, Srour M, Oskoui M, et al. Screening for developmental delay in the setting of a community pediatric clinic: a prospective assessment of parent-report questionnaires. Pediatrics. 2006;118(4):e1178–e1186. doi:10.1542/peds.2006-0466 [CrossRef]
- Hagan JF Jr, Shaw JS, Duncan PM, eds. Bright Futures: Guidelines for Health Supervision of Infants, Children, and Adolescents. 3rd ed. Elk Grove Village, IL: American Academy of Pediatrics; 2008.
- Ireton H, Vader H. Child Development Review: Instructions Manual. Minneapolis, MN: Behavior Science Systems, Inc; 2004.
- Doig KB, Macias MM, Saylor CF, Craver JR, Ingram PE. The Child Development Inventory: A developmental outcome measure for follow-up of the high-risk infant. J Pediatr. 1999;135(3):358–362. doi:10.1016/S0022-3476(99)70134-4 [CrossRef]
- Behavior Science Systems, Inc. Child Development Review: Screening and Assessing the Development of Young Children Using Parent Report and Professional Observations. Behavior Science Systems, Inc. 2011. Available at: www.childdevrev.com. Accessed Feb. 15, 2012.
- Thomas SA, Cotton W, Pan X, Ratliff-Schaub K. Comparison of systematic developmental surveillance with standardized developmental screening in primary care. Clin Pediatr (Phila). 2012;51(2):154–149. doi:10.1177/0009922811420711 [CrossRef]
- Boyle CA, Boulet S, Schieve LA, et al. Trends in the prevalence of developmental disabilities in US children, 1997–2008. Pediatrics. 2011;127(6):1034–1042. doi:10.1542/peds.2010-2989 [CrossRef]
- Bethell C, Reuland C, Schor E, Abrahms M, Halfon N. Rates of parent-centered developmental screening: disparities and links to services access. Pediatrics. 2011;128(1):146–155. doi:10.1542/peds.2010-0424 [CrossRef]
- Hillemeier MM, Morgan PL, Farkas G, Maczuga SA. Perinatal and socioeconomic risk factors for variable and persistent cognitive delay at 24 and 48 months of age in a national sample. Matern Child Health J. 2011;15(7):1001–1010. doi:10.1007/s10995-010-0656-x [CrossRef]
- Jensen RE, Chan KS, Weiner JP, Fowles JB, Neale SM. Implementing electronic health record-based quality measures for developmental screening. Pediatrics. 2009;124(4):e648–e654. doi:10.1542/peds.2008-3091 [CrossRef]