For more than a decade, a guiding principle of higher education has been that all college graduates need to develop intellectual and practical skills that come from extensive practice with inquiry and analysis (National Leadership Council for Liberal Education, 2007). These skills are especially important for nursing students to develop as research and evidence-based practice (EBP) increasingly drive the provision of high-quality care (Coyne et al., 2018; Ryan, 2016). Nurses with a Bachelor of Science in Nursing (BSN) degree are expected to understand the research process and apply evidence to improve outcomes, including having the ability to identify clinical questions, appraise evidence, and integrate theory into clinical practice (American Association of Colleges of Nursing, 2008). The challenge for faculty is that prelicensure nursing students frequently have difficulty comprehending the relevance of research to their future practice and may not see its value (Coyne et al., 2018; Mitchell et al., 2020; Opsahl et al., 2020).
Building confidence in research and EBP abilities comes with use over time (Ryan, 2016). Engaging in undergraduate research is one effective approach to help BSN students develop research and EBP knowledge, skills, and attitudes (Hensel, 2016; Hickey et al., 2019; Mitchell et al., 2020). Undergraduate research is a high-impact educational practice that promotes deep learning and helps students to develop practical skills and personal and social competence (Kuh & O'Donnell, 2013). More than enrollment in a lecture course, the Council on Undergraduate Research (2012) defined undergraduate research as an investigation by an undergraduate student that results in an original intellectual or creative contribution to a discipline.
Research suggests that participating in undergraduate research is more effective in positively shaping attitudes toward research than enrollment in a traditional didactic course (Coyne et al., 2018). When nursing students engage in undergraduate research, they develop confidence and a skill set that distinguishes them from their peers (Antior & Pugh, 2015; Hensel, 2016). One study found that participation in undergraduate research helped students to feel empowered to practice evidence-based nursing and see themselves as future nurse scientists and educators (Mitchell et al., 2020). Additional outcomes of undergraduate research include the development of oral and written skills to disseminate findings, enhanced problem solving to effect practice change, increased satisfaction with faculty mentors, and intensified value of professional development, graduate education, and lifelong learning (Burkhart & Hall, 2015; Coyne et al., 2018; Hensel, 2016; Hickey et al., 2019).
There is a growing body of literature surrounding models for undergraduate research, including honors models, mentorships, research assistantships, and summer internships (Burkhart & Hall, 2015; Coyne et al., 2018; Hensel, 2016; Hickey et al., 2019; Mitchell et al., 2020; O'Brien & Hathaway, 2018). It is unclear how actual research design (i.e., doing quantitative versus qualitative studies) affects student learning.
Hensel (2016) suggested that Q-methodology was an ideal method to use for undergraduate research because it was a doable method to generate meaningful research about a wide variety of subjective phenomena. Q-methodology, or Q for short, is a philosophy and a set of techniques created by William Stephenson to systematically study subjective phenomena (Ramlo, 2016). Now classified as a mixed-methods research approach, Q-methodology integrates quantitative and qualitative steps to study opinions, beliefs, and attitudes (Ramlo, 2016). The method requires that participants actively express their preferences by sorting a set of subjective statements or stimuli according to the researcher's instructions (Ho, 2017). Instead of correlating people to tests, Q-methodology studies use by-person factor analysis to correlate sorts to each other to find naturalistic groups of participants with shared perspectives (Ho, 2017). Using relatively small sample sizes, Q-methodology is an efficient method to explore subjectivity (Ellingsen et al., 2010). Compared with reporting group opinions as averaged data obtained from a Likert scale, information gathered in a Q-methodology study specifically helps explain how viewpoints differ within a group (Ho, 2017; Opsahl et al., 2020).
Q-methodology is a flexible method to use for research and program evaluation (Hensel, 2016; Opsahl et al., 2020; Ramlo, 2016). Advocates suggest that Q studies hold promise for exploring the complexities of nursing practice but use of the method to date in nursing is limited (Simons, 2013; Stone et al., 2017). An emerging body of literature suggests that undergraduates can learn the basics of Q-methodology and use the method for research projects (Hensel, 2016; Opsahl et al., 2020; Rhoads & Brown, 2019). The purpose of this study was to evaluate changes in attitudes about research among a cohort of nursing students who conducted Q-methodology studies as part of an undergraduate honors program.
This program evaluation project met the university's institutional review board criteria for low-risk, exempt research. The purposive participant sample included all five senior, female BSN students who independently conducted various Q-methodology studies with a faculty mentor as part of a 1-year undergraduate research honors program. The evaluation process began by collecting a set of opinion statements known as the “concourse” (Ramlo, 2016). Each participant provided five reasons why they believed students would engage in undergraduate research and five reasons why they would not. We removed repetitious items from the original 50 statements to create an unstructured, 36-item Q-sample used for sorting.
Participants sorted the statements twice on a +4 to −4 quasi-normal, forced-distribution sorting grid (also known as a sort) under two different conditions of instruction. First, participants ranked their level of agreement and disagreement with each statement according to their current beliefs about undergraduate research after designing a study. Then participants repeated the process based on their beliefs about undergraduate research before participating in the honors program.
The process yielded 10 completed sorting grids for analysis. We completed factor analysis with principal components extraction and varimax rotation using Ken-Q Analysis Web Application for Q-Methodology ( https://shawnbanasick.github.io/ken-q-analysis-beta/#section1). We retained factors with eigenvalues ≥ 1.0. Auto flagging identified factor loadings. Factors scores for each statement were calculated as Z scores and Q scores.
A two-factor final solution explained 70% of the variance. We gave the factors names to align with their characterizing statements. Nine sorts loaded positively on one of two factors. We excluded the one confounded sort—that loaded positively on both factors—from the factor score calculations. Six sorts loaded positively on factor one, named Getting Ahead. The characterizing statements with Q factor scores of +4 were “Undergraduate research could distinguish me among other graduates,” and “Doing undergraduate research could help me get into advanced education programs, like graduate school.” The Getting Ahead factor explained 42% of the variance. Sorts loading on Getting Ahead included all five sorts representing current attitudes (loadings = .80–.89) and one sort representing beliefs before beginning the program (loading = .43).
Three sorts were associated with factor two named, Too Much Effort. This factor had loadings ranging from .77 to .84 and explained 28% of the variance. The statements for Q factor scores of +4 were “Undergraduate research is intimidating,” and “I don't have time to participate in undergraduate research.” All sorts loading on Too Much Effort represented attitudes before beginning the honors program.
Fourteen statements distinguished the two factors at the p ≤ .01 level of significance. An additional eight statements distinguished the factors at the p ≤ .05 level. The statement rated most differently was “I believe undergraduate research could increase my confidence,” averaging a Q score of +2 with the Getting Ahead perspective and −3 with the Too Much Effort viewpoint. The two groups did not rank 14 statements as significantly different. Participants generally agreed with the statement “I feel that undergraduate research could connect me with a faculty member as a mentor.” Participants generally disagreed with the statements: “Research isn't important for my career,” “Research is not important in the undergraduate level,” “I already know enough about research,” and “I don't believe undergraduate research would benefit me.”
This article describes using Q-methodology to evaluate affective domain learning related to research among BSN honor students conducting Q-methodology studies as part of an undergraduate honors program. Program evaluation typically takes the form of satisfaction surveys or, in some cases, preand posttests to measure changes in knowledge or attitudes. A unique feature of Q-methodology studies is that participants can sort statements under single or multiple conditions of instruction (Ellingsen et al., 2010). Using multiple conditions of instruction helps to reveal different angles and the multiple layers of an individual's perspective (Ellingsen et al., 2010). Based on the differences found under the two conditions of instruction, we concluded that participation in this undergraduate research program resulted in a shifting of positive attitudes toward research.
Kuh and O'Donnell (2013) suggested the goal of undergraduate research is actively engaging students in hopes of creating a sense of excitement from working to answer important questions. The same researchers found that undergraduate research was associated with gains in deep learning approaches, general education, practical competence, and personal and social development. Practical competence refers to how a student will perform after college, especially in the workplace (Kuh et al., 2006). In this study, the statements with the highest Q-scores on the Getting Ahead perspective related to practical competence. Students believed their experience with undergraduate research would benefit their future career by giving them a unique skill set. Students also believed participation in undergraduate research helped them to gain confidence. This was a promising finding because lack of confidence with research is a major barrier to EBP in the work environment (Ryan, 2016).
After engaging in one semester of an honors program and designing a Q-methodology study, students reported gaining intellectual and practical skills and no longer found research to be as intimidating. Ultimately, all students completed their studies and disseminated the findings at a professional conference. Being able to research an interest area intensifies student engagement (Ryan, 2016), and one of the advantages of using Q-methodology for undergraduate research is that it can be used to study a wide variety of interests (Hensel, 2016; Rhoads & Brown, 2019). Although the applications of Q-methodology to study subjectivity are virtually unlimited, the steps are consistent. They involve identifying statements about a topic of interest, selecting the statements for sorting, selecting participants, administering the sort, completing factor analysis, and interpreting the data (Ellingsen et al., 2010). Hensel (2016) suggested that the well-defined research steps of a Q-methodology study provide enough structure to help novice researchers to be successful. However, it remains unknown what role study design played in shifting these students' attitudes. It is also unknown whether students would have had different outcomes if they had conducted other types of studies.
Like other qualitative research methods, the findings of Q-methodology studies are not meant to be broadly generalized. Rather, the findings of Q-methodology studies provide rich data and clarity to help to understand participant's viewpoints that can provide direction for future research. A limitation of this study was that the statements used for sorting addressed general attitudes towards undergraduate research. No statements specifically addressed attitudes toward Q-methodology. This was a purposeful decision so that the study might be replicated with students completing other types of studies at some future point. Future research comparing results from groups who conduct different types of studies is needed to fully understand how research design influences student success.
- American Association of Colleges of Nursing. (October20, 2008). The essentials for baccalaureate education for the professional practice of education. http://www.aacn.nche.edu/Education/pdf/BaccEssentials08.pdf
- Antior, K. R. & Pugh, A. K. (2015). The experience of completing an undergraduate nursing honors research project on the first professional nursing position. Indiana University Journal of Undergraduate Research, 1(1), 40–44 https://scholarworks.iu.edu/journals/index.php/iujur/article/view/13724/25149 doi:10.14434/iujur.v1i1.13724 [CrossRef]
- Burkhart, P. V. & Hall, L. A. (2015). Developing the next generation of nurse scientists. Nurse Educator, 40(3), 160–162 doi:10.1097/NNE.0000000000000121 [CrossRef] PMID:25581434
- Council on Undergraduate Research. (2012). Characteristics of excellence in undergraduate research. https://www.cur.org/assets/1/23/COEUR_final.pdf
- Coyne, B. M., Kennedy, C., Self, A. & Bullock, L. (2018). A comprehensive approach to undergraduate nursing students' research experiences. Journal of Nursing Education, 57(1), 58–62 doi:10.3928/01484834-20180102-12 [CrossRef] PMID:29381163
- Ellingsen, I. T., Størksen, I. & Stephens, P. (2010). Q methodology in social work research. International Journal of Social Research Methodology, 13(5), 395–409 doi:10.1080/13645570903368286 [CrossRef]
- Hensel, D. (2016). Q methodology: An alternative design for undergraduate nursing research. Journal of Nursing Education, 55(11), 659–662 doi:10.3928/01484834-20161011-10 [CrossRef] PMID:27783822
- Hickey, J. E., Adam, M., Elwadia, I., Nasser, S. & Topping, A. E. (2019). A process-environment model for mentoring undergraduate research students. Journal of Professional Nursing, 35(4), 320–324 doi:10.1016/j.profnurs.2019.02.001 [CrossRef] PMID:31345513
- Ho, G. W. (2017). Examining perceptions and attitudes: A review of Likert-type scales versus Q-methodology. Western Journal of Nursing Research, 39(5), 674–689 doi:10.1177/0193945916661302 [CrossRef] PMID:27456460
- Kuh, G. D., Kinzie, J. L., Buckley, J. A., Bridges, B. K. & Hayek, J. C. (2006). What matters to student success: A review of the literature (Vol. 8). National Postsecondary Education Cooperative.
- Kuh, G. D. & O'Donnell, K. (2013). Ensuring quality & taking high-impact practices to sale. Association of American Colleges and Universities.
- Mitchell, K., Rekiere, J. & Grassley, J. S. (2020). The influence of undergraduate research assistant experiences on future nursing roles. Journal of Professional Nursing, 36(3), 128–133 doi:10.1016/j.profnurs.2019.09.006 [CrossRef] PMID:32527634
- National Leadership Council for Liberal Education and America's Promise. (2007). College learning for the new global century. Association of American Colleges and Universities.
- O'Brien, T. & Hathaway, D. (2018). Students and faculty perceptions of an undergraduate nursing research internship program. Nurse Educator, 43(2), E1–E4 doi:10.1097/NNE.0000000000000412 [CrossRef] PMID:28665823
- Opsahl, A., Judge, D. & Hensel, D. (2020). Teaching Q methodology to prelicensure nursing students. Nursing Education Perspectives, 41(1), 39–40 doi:10.1097/01.NEP.0000000000000447 [CrossRef] PMID:31860485
- Ramlo, S. (2016). Mixed method lessons learned from 80 years of Q methodology. Journal of Mixed Methods Research, 10(1), 28–45 doi:10.1177/1558689815610998 [CrossRef]
- Rhoads, J. C. & Brown, Z. (2019). Q methodology. In Lambert, M. (Ed.), Practical research methods in education: An early researcher's critical guide. Routledge. doi:10.4324/9781351188395-9 [CrossRef]
- Ryan, E. J. (2016). Undergraduate nursing students' attitudes and use of research and evidence-based practice—An integrative literature review. Journal of Clinical Nursing, 25(11–12), 1548–1556 doi:10.1111/jocn.13229 [CrossRef] PMID:26990808
- Simons, J. (2013). An introduction to Q methodology. Nurse Researcher, 20(3), 28–32 Advance online publication. doi:10.7748/nr2013.01.20.3.28.c9494 [CrossRef] PMID:23346776
- Stone, T. E., Maguire, J., Kang, S. J. & Cha, C. (2017). Practical issues of conducting a Q methodology study: Lessons learned from a cross-cultural study. Advances in Nursing Science, 40(3), 289–297 doi:10.1097/ANS.0000000000000164 [CrossRef] PMID:27930400