Journal of Nursing Education

Major Article 

Strategies for the Planning and Handling of Missing Data in Nursing Research

Dawn M. Aycock, PhD, RN; Matthew J. Hayat, PhD

Abstract

Background:

Missing data are an inevitable reality in research. Nurse educators can promote proactive thinking about this topic to help avoid excessive missingness. The purpose of this article is to encourage nurses to view missing data as an accepted reality and to consider strategies for anticipating and minimizing missing data throughout the research process.

Method:

The common causes of missing data and ways to minimize their occurrence are discussed, along with suggestions for adopting a statistical mindset about missing data. Rubin's framework for missingness as a random process, modern statistical methods for analyzing missing data, and recommendations for reporting also are discussed.

Conclusion:

Nurse educators and researchers should understand all aspects of missing data, including the types, occurrence, causes, potential problems, and strategies for preventing, handling, and reporting missing data. By doing so, the occurrence of missing data can be lessened, thereby minimizing various problems that can result. [J Nurs Educ. 2020;59(5):249–255.]

Abstract

Background:

Missing data are an inevitable reality in research. Nurse educators can promote proactive thinking about this topic to help avoid excessive missingness. The purpose of this article is to encourage nurses to view missing data as an accepted reality and to consider strategies for anticipating and minimizing missing data throughout the research process.

Method:

The common causes of missing data and ways to minimize their occurrence are discussed, along with suggestions for adopting a statistical mindset about missing data. Rubin's framework for missingness as a random process, modern statistical methods for analyzing missing data, and recommendations for reporting also are discussed.

Conclusion:

Nurse educators and researchers should understand all aspects of missing data, including the types, occurrence, causes, potential problems, and strategies for preventing, handling, and reporting missing data. By doing so, the occurrence of missing data can be lessened, thereby minimizing various problems that can result. [J Nurs Educ. 2020;59(5):249–255.]

Missing data are a common, potential problem that nurse educators and researchers must consider when conducting and evaluating research. Regardless of the amount or type, missing data potentially can lead to invalid or misleading study conclusions. For example, one major issue with missing data is loss of statistical power due to a reduced sample size. This loss may occur at an item level with nonresponse to some questions in a survey or at a participant level because of withdrawal from a study. Either type of loss may lead to bias in sample representativeness and estimation of parameters (Kang, 2013).

In addition, missing data can present challenges to analyzing data. For example, a substantial issue may arise with generalizing results because the validity of statistical inference and modeling methods may be questioned in the presence of missing data. Such methods rely on an assumption of a random sample drawn from the population of inference, and missing data potentially biases the representativeness of that study sample. Although there are many statistical techniques developed to handle and analyze missing data, all of the techniques necessitate making unverifiable assumptions about why the data are missing. The only real certainty about missing data is that the data were unobserved.

Nurse researchers conducting studies involving human participants can encounter missed values in the data collection process. Rules of informed consent for ethical approval of human subjects research requires that participants have freedom to withdraw or refuse to answer a question at any time during a research study. This may result in missing information for one or more questions, or at one or more study time points. Attrition and loss to follow up also are common in studies with measurements collected over time. Further, human participants do not always adhere or comply. As a result, missing data to some extent is a natural and expected occurrence in research. However, anticipating the potential for missing data and strategizing during research design, conduct, and analysis can help to mitigate the problem.

This article, intended as educational and informative, is the result of an interdisciplinary collaboration between a nurse researcher and biostatistician. Throughout the study planning and data collection process of a feasibility study (Aycock et al., 2017) and pilot study (Aycock, 2015) of a randomized control trial, we discovered an opportunity to educate and learn from each other. The nurse researcher had considerable experience and content knowledge, including forethought and insight into the potential causes and solutions for missing data in relation to instrumentation, data collection, and recruitment and retention strategies. The biostatistician had methodological expertise and thoughts into study design and analytic thinking to anticipate and prepare for handling missing data that were likely to occur. We recognized an opportunity to construct an innovative article that integrates the two disciplines and communicates about our interaction in a way that would benefit nurse educators and researchers. We conducted a literature review of articles in the nursing education and research literature about missing data, and provided an overview of those works, along with insights resulting from our experiences and collaboration.

A carefully planned study can aim to reduce the occurrence of missing observations through design and measurement considerations (Bannon, 2015; Harel & Boyko, 2013). However, given the inevitable likely presence of at least some missing data in a scientific investigation, there is potential benefit in accepting and embracing the reality that to some extent, missing data will occur. Creative scientists have considered this idea and even planned for missing data with the development of planned missing data study designs (Little et al., 2014; Pokropek, 2011). For example, subsets of questions in a survey may be partitioned so that not all questions are asked of all participants. Fewer questions per participant equates to fewer survey questions, shorter needed attention span, potential for reduced cost, and targeted precision of data collection. However, missing data study designs are not commonly used in nursing research; as a result, other strategies must be considered.

The purpose of this article is to educate and encourage nurse researchers to consider missing data as an accepted reality and handle it accordingly in health and education-related study planning, design, and analysis. This work is particularly designed to assist novice researchers (i.e., doctoral students and early career investigators) by providing an overview of missing data terminology and practical experiential, as well as evidence-based, strategies for addressing missing data. The information also is beneficial for nurse educators, as they are key to promoting the significance of considering and planning for missing data in all endeavors involving data collection.

The next section describes the types of missing data, including a description of Rubin's (1976) commonly accepted framework for missingness as a random process. This is followed by sections describing the ways that missing data can occur and the causes of missing data, as well as strategies for anticipating and minimizing missing data. Statistical methods for handling and analyzing missing data are covered, including a review of outdated and naïve methods and a description of modern methods and recent methodological developments. Finally, a summary of statistical software and procedures for implementing appropriate methods for analyzing missing data is provided, concluding with reporting missing data.

Types of Missing Data

Rubin (1976) defined the terminology for describing types of missing data, as well as the framework for making statistical inferences with incomplete data. Missing data were classified as ignorable or nonignorable. The term “ignorable” suggests there are ways to yield meaningful and unbiased parameter estimates without modeling or explaining the missingness. Nonignorable missingness is problematic and a challenge to deal with, as the only way to produce unbiased parameter estimates is to explain or model the missing mechanism.

Ignorable missing data are defined as missing completely at random (MCAR) if the reason for missingness is truly random. In other words, MCAR indicates that the probability of a missing value is independent of the outcome or other unobserved or observed study variables. An example of MCAR could be a participant missing a study visit because of mechanical problems with his or her car, a factor that is attributed to chance occurrence and has nothing to do with the study. A second type of ignorable missing data is missing at random (MAR). In this case, the reason for missingness is dependent on or can be explained by observed data. For example, in a study of a cardiovascular risk intervention, older participants may be less inclined to self-report physical activity level. Controlling for age in the statistical analysis may account for this confounder and result in unbiased estimates. Once this is done, the missing information then is assumed to be MCAR.

Nonignorable missing data occurs if the missing mechanism depends on unobserved data. Referred to as missing not at random (MNAR), this type of missing data is nonignorable since the reason for missingness needs to be explained in order to eliminate or minimize bias. An example is in the primary outcome of systolic blood pressure missing for hypertensive participants in a study of a cardiovascular risk intervention.

Ways Missing Data Can Occur

Because missing data is common in research, it is sensible to plan ahead and anticipate its occurrence. Determining the reason for missing data is important in an observational or experimental study. However, there are different kinds of missing data, and planning for and handling it may differ by study design. Missing data may result from the loss of single items and entire instruments in all designs, whereas missed study visits and participant dropouts or withdrawals are more likely to occur in experimental designs or designs with multiple data collection time points.

The loss of single items occurs when participants mistakenly skip over questions, refuse to answer questions, or provide two or more responses when only one is needed. This is more likely to occur with self-administered questionnaires than with interviews. The loss of entire questionnaires can occur in the same manner but are more likely to result from faulty administration of the questionnaires and when individuals are required to complete questionnaires and return them, for example, by mail to the study site. When multiple data-collection time points or follow-up visits are required, participants may miss a visit due to scheduling conflicts, transportation issues, forgetting about the visit, or health problems. Finally, the more challenging form of missing data occurs when participants drop out or are withdrawn from a study. This can occur due to time constraints, the burden of participating in a study, a loss of interest, a move, or an illness or death.

Causes for Missing Data

A variety of factors may contribute to missing data; however, most of these factors can be prevented if anticipated and planned for ahead of time. How questionnaires are designed or formatted may increase the likelihood of missing data. The visual layout should be clean, simple, and consistent, with enough spacing to differentiate between items. Participants may skip over questions depending on what is being asked or how questions are asked. For example, they may feel uncomfortable answering questions related to sensitive information, such as age, income, alcohol and recreational drug use, and sexual activity. When participants have difficulty understanding a question or do not know how to answer a question, there is the potential for item omission. Finally, missing items are common when participants lack interest in the survey topic and when items are inappropriate or do not relate to them. For example, the Piper Fatigue Scale has an item with a high potential for omission: “To what degree is the fatigue you are feeling now interfering with your ability to engage in sexual activity?” (Responses range from “1” [none] to “10” [a great deal]) (Piper et al., 1998). Respondents may feel uneasy answering the question because they may not want to share personal information, they may not know how to best answer the question, or if they are not sexually active, they may feel it is irrelevant.

Faulty communication by the research team also can result in missing data. This can occur when participants are not provided with appropriate instructions for participating in a study, including how to complete questionnaires, how to return questionnaires or diaries to a study site, and when to return for follow-up visits. When projects are not well organized or managed, the potential for missing data is increased. Missing data also can occur when participants are distracted and lose focus while filling in questionnaires; they may be too busy to participate in the study, lose interest in the study, or complete the questionnaires in an environment that is not conducive to research (i.e., the temperature of the room is uncomfortable or there is a high noise level). Having to complete lengthy surveys or multiple surveys during a study visit can be burdensome and lead to items being omitted or participants dropping out.

Strategies for Anticipating and Minimizing Missing Data

Several strategies can be implemented to minimize missing data during the design of a study, prior to data collection, and during data collection.

Study Design

When designing a study, effort should be taken to minimize the amount of data that needs to be collected and the number of follow-up visits. It is sometimes helpful to focus on collecting only essential information and minimizing extraneous subject data. This may be an effective strategy in lessening the amount of missing data (Biswas, 2012; Kang, 2013; Lin et al., 2012).

Researchers also should try their best to consider how much missing data may warrant problems and set a priori targets for unacceptable levels of missing data (Kang, 2013). Some authors have taken a strong stance regarding acceptable missing data levels. For example, Schulz and Grimes (2002) suggest that having more than 20% of the data missing is a marker indicating that trial validity could be threatened. The amount of missing data can be quantified by the percentage of missing data per study participant, the percentage of study participants who have missing data values, and the percentage of participants who complete the study (e.g., attrition rate).

Based on the target of missing data that is set, a missing data plan should be devised. Working with a biostatistician can be helpful in thinking about the analytical framework and how missing data may affect the overall study aims. Roberts et al. (2017) illustrated how a nurse scientist and statistician worked together to determine the best approaches to missing data in a longitudinal study of premature infants. In general, it is important to account for anticipated attrition and nonresponse by elevating the target study sample size to ensure the number of participants needed is achieved. Lin et al. (2012) provide additional recommendations to include accounting for missing data in the sample size calculation to minimize the possibility of having an underpowered study, identifying the analytic strategies for dealing with missing data a priori, and using a data safety and monitoring board to monitor missing data and to make decisions on how to address the missing data.

Knowing the target population and who may be more prone to omitting items can help with anticipating and monitoring missing data. Missing data is prevalent when measuring quality of life, and researchers have found that sociodemographic characteristics and health status (i.e., older age, female sex, low education, low economic status, poor health, and quality of life) are major determinants of items missing on the commonly used SF-36 Health Survey (Peyre et al., 2010). Brouwer et al. (2013) found that cultural variables contributed to survey nonresponse in a sample of elderly, immigrant women with hypertension and recommended that researchers consider the cultural appropriateness of survey items for culturally sensitive topics such as mental health and sexuality.

A more advanced strategy is using a planned missing data design. Planned missing data designs have been used for many years. These are designs that allow participants to answer some questions and not others, which results in missing data by design. Pokropek (2011) and Little et al. (2014) provide more information and detailed examples of planned missing data designs.

Prior to Data Collection

When selecting measures to collect data, it is well-known that reliable and valid tools should be used. In addition, testing the application of these tools in the researcher's own setting is often beneficial for identifying potential problems that may lead to missing data. Feasibility studies often are conducted with this purpose in mind. However, when these activities cannot be performed, researchers can test the tools among small groups of people (e.g., groups of 5 to 10 individuals). Researchers may observe problems with how a questionnaire is formatted, participant difficulty with understanding questions, and problems with answering questions that are sensitive in nature. Researchers also can assess the time taken to complete the questionnaires to estimate participant burden or mental fatigue. By identifying these problems early, corrections can be made or plans for anticipating potential areas for missing data can be set prior to initiating the larger study.

A helpful tool that can be used to minimize missing data is a data collection checklist. The checklist provides a sequence or steps for the data collection process (e.g., informed consent, individual questionnaires). It serves as a reminder for data collectors and provides documentation of completed questionnaires. A checklist can provide structure and consistency across team members involved with data collection. Aycock et al. (2016) provide an example of a data collection checklist and other helpful tools to track data and facilitate research project management.

Sufficient resources should be provided to facilitate data collection including research personnel time, adequate space, and parking for participants (Biswas, 2012). To lessen missing data that may result from researcher error, data collectors also should undergo training on the importance of minimizing missing data; how to anticipate, identify, and retrieve missing data; and how to use data collection checklists. Data collectors should receive a manual of operations and undergo mock data collection training that includes questionnaire administration and communication with participants to ensure accuracy and consistency.

During Data Collection

To minimize missing data during the data collection process, data collectors should anticipate that missing data might occur and monitor for satisfactory completion of questionnaires. Researchers should consider all possible factors that could lead to missing data and observe these factors so that appropriate adjustments can be made (Lin et al., 2012). After participants complete questionnaires, data collectors should scan the questionnaires to identify whether any items were skipped. If missing items are found, the data collector can point out the missing items to participants and ask if they meant to skip over the items. If participants mistakenly skipped the items, they can complete the items at that time; if they did not understand the question, the data collector can provide an explanation as appropriate; and if participants intentionally skipped the questions, they have the ethical right to do so, and the data collector can make note that the participant declined to answer the question. Maintaining a log of problems that occur during the course of a study that includes missing data may be helpful in determining any trends in missing data and describing missing data in study reports.

Strategies also can be implemented to improve participant compliance with study visits and to promote retention. The researcher-participant relationship and communication are key. Maintaining logs to store participants' contact information and for tracking study visits can provide organization of study data and help facilitate the communication process (Aycock et al., 2016). Other strategies include providing sufficient information about the study and the time commitment during the recruitment and consent process and ensuring that potential participants understand. It is also possible to include eligibility criteria directly stating that participants are willing to take part in the study. This can serve as a reminder to query participants about their willingness to participate. Researchers should provide appropriate incentives for participation and increase incentives for follow-up visits in longitudinal studies. Frequently thanking participants for volunteering their time on the study also is recommended.

Biswas (2012) provides additional recommendations for study personnel to minimize missing data including emphasizing to participants the importance of completing the study, creating a welcoming and caring atmosphere for participants, and being flexible in time windows for scheduling in-person visits. For participants who are at greatest risk of dropping out, retention strategies should include frequent contacts by phone or e-mail, assistance with transportation, and alternate methods for collecting data as needed (Yeatts & Martin, 2015). If participants are not able to return for a study visit, researchers can conduct telephone interviews, use telemedicine, or make home visits as appropriate. Within the tracking logs, missed study visits and documentation of participant dropouts and withdrawals should be included. Recording these data will be helpful when it is time for analyzing the data and for preparing research reports.

Missing Data in Electronic Data Collection

More researchers are using web-based programs (e.g., Qualtrics®, SurveyMonkey™, and REDcap®), including mobile apps to collect, enter, and store data. Data quality has been found to be higher with electronic data capture versus traditional paper-based questionnaires because of lower rates of missing data (Marcano Belisario et al., 2015; Smith et al., 2013). However, caution also should be taken when designing and using electronic data collection tools. Similar thinking should be applied when considering missing data for either a printed questionnaire or an electronic survey.

Some factors to consider when designing web-based surveys to improve data quality include number of items per page, vertical versus horizontal layout, and open-versus closed-ended questions (De Bruijne & Wijnant, 2014; Mavletova & Couper, 2016). The specifics about these survey aspects are not elaborated as they depend on the research aims and survey structure. Regmi et al. (2016) described additional strategies as well as unique ethical concerns to consider. Most electronic formats allow designers to set up questions that require responses before advancing; however, requiring participants to provide responses is unethical. An alternative for sensitive questions would be to add “don't know” or “prefer not to answer” as responses to improve response accuracy and limit skipped items or participant dropouts.

Technical issues and user error may contribute to missing data with electronic data collection. Researchers should receive adequate training to deal with technical problems or have personnel with expertise in information technology available. When participants are expected to complete electronic surveys (e.g., at home or work), they also need instruction on how to handle technical issues (e.g., correcting problems with Internet connectivity, forgetting their user name or PIN, being locked out after using the back button, or getting an error message when submitting their survey). A back-up plan should be in place when these technical issues occur.

In an Internet-based diary study for women with migraines, researchers provided participants with pre-printed diary pages to complete if they experienced computer problems (Moloney et al., 2009). Participants noted having a supportive information technology team, a simple and user-friendly website, and researchers who developed good online relationships with participants minimized missing data and attrition. As with printed questionnaires, researchers also should be knowledgeable of the population that is more likely to encounter problems with missing data for electronic surveys. Older adults and individuals who are less computer savvy may require additional assistance.

Statistical Approaches for Handling Missing Data

There are many ways to handle missing data in a statistical analysis. It is ideal to plan ahead and specify a priori data analysis plan. Extensive literature exists about statistical methods for analyzing missing data. Although not the main focus of this article, we briefly discuss five statistical approaches to handling missing data:

  • Complete case analysis.
  • Last observation carried forward.
  • Single imputation.
  • Full information maximum likelihood.
  • Multiple imputation.

It is of note that these statistical methods are applicable for observational and experimental study designs and can be applied before use of a statistical method or test.

Complete Case Analysis

Historically, due to classical statistical methods and limitations of statistical software and computing power, complete case analysis was the conventional approach for many years (Fox-Wasylyshyn & El-Masri, 2005). This method also is referred to in the health literature as listwise deletion. In a complete case analysis, if any observations on a variable included in an analysis are missing for a study subject, that individual is not included in the analysis. Parameter estimates from a complete case analysis are not biased if the data are MCAR. However, it is rare that all missing data are MCAR. In addition, although parameter estimates will not be biased, statistical power will decrease with the loss of one or more study subjects in the analysis.

Last Observation Carried Forward

Last observation carried forward is an approach for handling missing data in a longitudinal study. In the last observation carried forward method, a missing value at one or more follow-up time points is replaced with the previously observed value. Literally, the last observation is carried forward. Its common use is due in part to Food and Drug Administration guidelines that explicitly recommend that submissions use last observation carried forward to address missing data (Lachin, 2016). As a result, there are a host of peer-reviewed articles in the health literature that have made use of this method, and it is mentioned here for this reason. However, it is important to note there are no known peer-reviewed publications that demonstrate its validity, and it is not possible to assess the potential bias with carrying observations forward. Last observation carried forward should not be used in any statistical analysis (Lachin, 2016).

Single Imputation

Single imputation entails replacing a missing value with a single number that represents a best guess at the real missing value. A simplistic approach is mean substitution for continuous data, which includes substitution of the sample mean for all individuals in a treatment group. A more sophisticated approach for imputation of values of a missing continuous variable is to use linear regression to predict the mean as a function of other study variables. However, all types of single imputation suffer from incorrect and underestimated standard error estimates, since sampling variability is not accounted for with imputation of only one value (Kneipp & McIntosh, 2001). As a result, standard errors will be underestimated and biased, resulting in incorrect statistical inferences.

Full Information Maximum Likelihood

Full information maximum likelihood is a computationally intensive lesser known approach for handling missing data (Schminkey et al., 2016). Unlike imputation, missing values are not replaced with full information maximum likelihood. Instead, parameter estimates make use of all observed data for each subject to find values of the parameters that are most likely to generate the observed data. An example can be seen in a longitudinal data analysis with one or more missing follow-up visits. Parameter estimates for treatment effects or covariates use every observed data point, including partial information on study subjects with missing visits. This method is now available in standard statistical software packages (e.g., SPSS®, SAS®, etc.).

Multiple Imputation

Multiple imputation is the gold standard of statistical methods for handling missing data (Patrician, 2002). As mentioned, the previously described methods have limitations that introduce bias that limits generalization and interpretability. Missing observations are replaced with several imputed values through specification of an imputation model. Multiple imputation accounts for sampling variability and random error that naturally results from drawing samples. The technique is readily available in SPSS and other statistical software packages. There are several assumptions needed to perform multiple imputation, including specification of a correct imputation model, distributional assumptions, and the need for missing data to be MAR. Other publications in the nursing research literature have presented accessible and thorough introductions to multiple imputation (Kneipp & McIntosh, 2001; McCleary, 2002; Patrician, 2002).

Reporting Missing Data

A study's data analysis plan should detail how missing data will be analyzed (Papageorgiou et al., 2018; Schlomer et al., 2010). The study results should include the amount of missing data as a percentage of complete data. In a study with more than one instrument or construct, it is advisable to report the range of missing data across study measures. Tables of descriptive statistics should include the sample size and quantities of missing data for each variable. It is important to distinguish missing values (e.g., nonresponse) for items from participant dropout and attrition. Details of attrition are necessary results for longitudinal studies, and randomized trial summaries should include specifics about attrition within treatment group with use of the standard CONSORT (Consolidated Standards of Reporting Trials) diagram. It is meaningful to provide readers with information about statistical analysis of missing data, including details about missing patterns discovered, imputations made, and comparisons drawn between observed and missing values on other variables of interest.

Summary

Missing data is a natural phenomenon in human subject research. Nonresponse may be random or related to an aspect of a study protocol or design. Careful planning and forethought can reduce or alleviate the occurrence of missing data. This article has described several simple to more complex strategies for anticipating and planning for missing data, including strategically developed planned study designs, thoughtful construction of study time points, development of instrumentation for collecting information, and creative use of electronic and online data collection platforms. Many statistical methods have been developed to analyze missing data. As expected with attempting to create information that we do not have, inferences about missing data usually necessitate making unverifiable assumptions. The optimal strategy is to minimize missingness whenever possible and implement a full reporting strategy when summarizing study results and writing for publication and dissemination.

References

  • Aycock, D.M. (2015). Stroke counseling for risk reduction in young adult African Americans. Unpublished application (NIH K01NR015494) funded by National Institute of Nursing Research. Copy in possession of author, School of Nursing Georgia State University, Atlanta, Georgia.
  • Aycock, D.M., Clark, P.C. & Hayat, M. (2017). Reducing stroke risk among young adult African Americans: A feasibility study. Research in Nursing & Health, 40(2),153–164.
  • Aycock, D.M., Clark, P.C., Thomas-Seaton, L., Lee, S. & Moloney, M. (2016). Simple tools to facilitate project management in a nursing research project. Western Journal of Nursing Research, 39(3), 440–443. doi:10.1177/019394591665660 [CrossRef]
  • Bannon, W. (2015). Missing data within a quantitative research study: How to assess, treat it, and why you should care. Journal of the American Association of Nurse Practitioners, 27(4), 230–232. doi:10.1002/2327-6924.12208 [CrossRef]
  • Biswas, K. (2012). Prevention and management of missing data during conduct of a clinical study. Shanghai Archives of Psychiatry, 24(4), 235–237. doi:10.3969/j.issn.10020829.2012.04.008 [CrossRef]
  • Brouwer, A.M., Mosack, K.E., Wendorf, A.R. & Sokolova, L. (2013). Patterns of missing data in ethnic minority health research: A survey project with Russian-speaking immigrant women with hypertension. Research and Theory for Nursing Practice, 27(4), 276–295.
  • De Bruijne, M. & Wijnant, A. (2014). Improving response rates and questionnaire design for mobile web surveys. Public Opinion Quarterly, 78(4), 951–962. doi:10.1093/poq/nfu046 [CrossRef]
  • Fox-Wasylyshyn, S.M. & El-Masri, M.M. (2005). Handling missing data in self-report measures. Research in Nursing & Health, 28(6), 488–495. doi:10.1002/nur.20100 [CrossRef]
  • Harel, O. & Boyko, J. (2013). Missing data: Should we care?American Journal of Public Health, 103, 200–201. doi:10.2105/AJPH.2012.300904 [CrossRef]
  • Kang, H. (2013). The prevention and handling of the missing data. Korean Journal of Anesthesiology, 64(5), 402–406. doi:10.4097/kjae.2013.64.5.402 [CrossRef]
  • Kneipp, S.M. & McIntosh, M. (2001). Handling missing data in nursing research with multiple imputation. Nursing Research, 50(6), 384–389.
  • Lachin, J.M. (2016). Fallacies of last observation carried forward analyses. Clinical Trials, 13(2), 161–168. doi:10.1177/1740774515602688 [CrossRef]
  • Lin, J.Y., Lu, Y. & Tu, X. (2012). How to avoid missing data and the problems they pose: Design considerations. Shanghai Archives of Psychiatry, 24(3), 181–184. doi:10.3969/j.issn.1002-0829.2012.03.010 [CrossRef]
  • Little, T.D., Jorgensen, T.D., Lang, K.M. & Moore, E.W.G. (2014). On the joys of missing data. Journal of Pediatric Psychology, 39(2), 151–162. doi:10.1093/jpepsy/jst048 [CrossRef]
  • Marcano Belisario, J.S., Jamsek, J., Huckvale, K., O'Donoghue, J., Morrison, C.P. & Car, J. (2015). Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods. Cochrane Database of Systematic Reviews, (7), MR000042. doi:10.1002/14651858.MR000042.pub2 [CrossRef]
  • Mavletova, A. & Couper, M.P. (2016). Device use in web surveys: The effect of differential incentives. International Journal of Market Research, 58(4), 523–544. doi:10.2501/IJMR-2016-034 [CrossRef]
  • McCleary, L. (2002). Using multiple imputation for analysis of incomplete data in clinical research. Nursing Research, 51(5), 339–343.
  • Moloney, M., Aycock, D., Cotsonis, G., Myerburg, S. & Farino, C. (2009). Migraine headache diary: Issues in Internet-based research. Headache, 49(5), 673–686. doi:10.1111/j.1526-4610.2009.01399.x [CrossRef]
  • Papageorgiou, G., Grant, S.W., Takkenberg, J.J.M. & Mokhles, M.M. (2018). Statistical primer: How to deal with missing data in scientific research?Interactive Cardiovascular Thoracic Surgery, 27(2), 153–158
  • Patrician, P.A. (2002). Multiple imputation for missing data. Research in Nursing & Health, 25(1), 76–84. doi:10.1002/nur.10015 [CrossRef]
  • Peyre, H., Coste, J. & Leplege, A. (2010). Identifying type and determinants of missing items in quality of life questionnaires: Application to the SF-36 French version of the 2003 Decennial Health Survey. Health and Quality of Life Outcomes, 8(16), 1–6. doi:10.1186/1477-7525-8-16 [CrossRef]
  • Piper, B.F., Dibble, S.L., Dodd, M.J., Weiss, M.C., Slaughter, R.E. & Paul, S.M. (1998). The revised Piper Fatigue Scale: Psychometric evaluation in women with breast cancer. Oncology Nursing Forum, 25(4), 677–684.
  • Pokropek, A. (2011). Missing by design: Planned missing-data designs in social science. Research & Methods, 20(1), 81–105.
  • Regmi, P.R., Waithaka, E., Paudyal, A., Simkhada, P. & van Teijlingen, E. (2016). Guide to the design and application of online questionnaire surveys. Nepal Journal of Epidemiology, 6(4), 640–644. doi:10.3126/nje.v6i4.17258 [CrossRef]
  • Roberts, M.B., Sullivan, M.C. & Winchester, S.B. (2017). Examining solutions to missing data in longitudinal nursing research. Journal for Specialists in Pediatric Nursing, 22(2), 1–12 doi:10.1111/jspn.12179 [CrossRef]
  • Rubin, D.B. (1976). Inference and missing data. Biometrika, 63(3), 581–592. doi:10.1093/biomet/63.3.581 [CrossRef]
  • Schlomer, G.L., Bauman, S. & Card, N.A. (2010). Best practices for missing data management in counseling psychology. Journal of Counseling Psychology, 57(1), 1–10. doi:10.1037/a0018082 [CrossRef]
  • Schminkey, D.L., von Oertzen, T. & Bullock, L. (2016). Handling missing data with multilevel structural equation modeling and full information maximum likelihood techniques. Research in Nursing & Health, 39(4), 286–297. doi:10.1002/nur.21724 [CrossRef]
  • Schulz, K.F. & Grimes, D.A. (2002). Sample size slippages in randomised trials: Exclusions and the lost and wayward. The Lancet, 359(9301), 781–785. doi:10.1016/S0140-6736(02)07882-0 [CrossRef]
  • Smith, A.B., King, M., Butow, P. & Olver, I. (2013). A comparison of data quality and practicality of online versus postal questionnaires in a sample of testicular cancer survivors. Psychooncology, 22(1), 233–237. doi:10.1002/pon.2052 [CrossRef]
  • Yeatts, S.D. & Martin, R.H. (2015). What is missing from my missing data plan?Stroke, 46(6), e130–e132. doi:10.1161/STROKEAHA.115.007984 [CrossRef]
Authors

Dr. Aycock is Associate Professor, Byrdine F. Lewis College of Nursing and Health Professions, and Dr. Hayat is Professor, School of Public Health, Georgia State University, Atlanta, Georgia.

This article was supported by a K01 training grant from NIH/NINR (K01 NR015494) to Dawn M. Aycock. The authors thank Nannan Zhang, MPH, Varsha Neelam, MPH, and Natalie Tripp, MPH, for assistance with the literature review and formatting.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

Address correspondence to Dawn M. Aycock, PhD, RN, Associate Professor, Byrdine F. Lewis College of Nursing and Health Professions, Georgia State University School of Nursing, P.O. Box 4019, Atlanta, GA 30302-4019; e-mail: daycock@gsu.edu.

Received: October 11, 2019
Accepted: December 18, 2019

10.3928/01484834-20200422-03

Sign up to receive

Journal E-contents