The Doctor of Nursing Practice (DNP) curriculum at our institution was revised in 2014 and is aligned with the American Association of Colleges of Nursing (AACN) recommendations (AACN, 2015) and the Essentials of Doctoral Education for Advanced Nursing Practice (AACN, 2006), and is described in our DNP student handbook. The DNP project proposal course, hereafter referred to as “the course,” is the first in a series of three project planning, implementation, and evaluation and dissemination courses occurring at the end of the DNP program. In the course, a proposal for the DNP project is constructed by the student over the semester describing the problem and background, evidence to support the problem and method to improve, and elements of project planning (i.e., time line, budget). The blended course (two face-to-face classes, on-demand electronic course site, 45 project-planning hours) prepares students who enter the DNP program from different educational backgrounds and levels of professional maturity to lead systems-focused health care improvement projects that meet the DNP Essentials (AACN, 2006). DNP students collaborate closely with course faculty and both faculty and practice mentors during the course.
Early on, some students entered the course without a specific problem on which their project would focus or an identified faculty mentor. Bachelor of Science in Nursing (BSN)-to-DNP students lacked a clear understanding of the DNP project problem in the practice setting and were naive about working with stakeholders. Student evaluation comments described anxiety and confusion, as well as inconsistent messaging between course faculty and faculty mentors to the student.
In addition to the problems stated by the students, the course faculty found that the proposals submitted during the course often had broad and unclear problems or PICO (Population, Intervention, Comparison, Outcome) statements, and missing linkages to outcomes and methods for improvement. Synthesis of literature, clarity, and brevity in writing style were subpar in some cases. Faculty mentors, mostly PhD prepared, were likewise confused about the project and their role mentoring the student. Faculty mentors contacted course faculty asking for guidance and/or individual instruction. Course faculty spent time responding to individual faculty, program directors, and administrators regarding the DNP project proposal and project. Rising enrollment each year drove the need to accelerate course improvement.
A rapid move toward evidence-based quality improvement projects and away from more traditional research-based projects as part of DNP education demands an improvement approach to engage students and faculty in the change. Nationally, faculty accustomed to research design and measurement were charged with developing DNP programs to include evidence-based DNP projects (Mayo, 2017). Unlike research projects, DNP projects are evaluated using outcome measures expected in the practice setting. Furthermore, the vast majority (87%) of DNP program directors describe faculty in their programs as being dissatisfied with the DNP projects (Dols, Hernández, & Miles, 2017). Reasons for the dissatisfaction include lack of faculty knowledge of evidence-based practice and quality improvement, lack of consensus on the DNP project, lack of faculty resources for DNP projects, and students' lack of scholarly writing skills. The International Academy of Nursing Editors (INANE) has also criticized graduate (master's and DNP) student writing skills by identifying concerns about poor writing, lack of detail, clarity, appropriate referencing, logical progression, students' inability to synthesize, and mistakenly describing projects as research design (Kennedy, Newland, & Owens, 2017).
The model for improvement (Langley et al., 2009) (Figure 1) guided the changes course faculty made over four semesters. The model's three questions frame the improvement:
- What are we trying to accomplish?
- How will we know that a change is an improvement?
- What changes can we make that will result in an improvement?
Model for Improvement diagram. From The Model for Improvement is a valuable roadmap for the project that emphasizes and encourages an iterative learning process. From The Improvement Guide: A Practical Approach to Enhancing Organizational Performance (2nd ed., p. 512), by G.J. Langley, R.D. Moen, K.M. Nolan, T.W. Nolan, C.L. Norman, and L.P. Provost, 2009, San Francisco, CA: Wiley. Copyright 2009 by Wiley. Adapted with permission.
Using improvement science principles (Langley et al., 2009), Plan-Do-StudyAct (PDSA) cycles were used each semester (Figure 1). PDSA cycles test a change by planning it, trying it, observing the results, and acting on what is learned (Institute for Healthcare Improvement, 2018). Continued cycles based on learning from the previous cycle are used to accelerate improvement.
The purpose of this activity was to improve and maintain the quality of the course in the face of rising enrollment, to guide students to deliver a scholarly and feasible proposal for the DNP project and develop faculty expertise in mentoring successful DNP projects.
DNP student enrollment expanded to 494 students in the BSN-to-DNP and post-master's-to-DNP program in 2017. Incremental changes were undertaken as part of the quality improvement approach to make it easier for students to write a rigorous proposal and develop a project plan with their mentors. The first step in quality improvement is listening to the voice of the customer to understand problems from their perspective (Lloyd, 2019). Student course evaluation scores and free-text comments were used to understand the problem from the student perspective. Input from faculty were received via feedback following faculty mentor trainings and informal communications.
Course faculty targeted student issues in the first PDSA cycle by creating and deploying tools and resources within the course. Next cycles included faculty mentors and students, using the same improvement strategy to improve confidence (satisfaction) with proposal writing and project planning. Paraphrased student evaluation comments informed improvement and identified remaining gaps, which were analyzed by course faculty in the S (Study) phase of the cycle (Table 1) and used to plan and predict in the next PDSA cycle.
Voice of the Customer
Study of the Interventions/Measures
Using the model for improvement, a change was determined to be an improvement (Study PDSA cycle) if evaluation comments described less confusion and anxiety, comments of a positive nature increased, “quality of the course” scores remained favorable, proposal quality improved and faculty complaints decreased, and course processes improved. Four PDSA cycles were conducted over four semesters.
A claim of exemption was granted for activities in cycle 4 (Research Protocol # 2017-0916 “Improving Student and Faculty Mentor Knowledge about DNP Project Proposal”).
PDSA Cycle 1
Plan. Focus on student confusion about the nature of the project proposal and difficulty navigating the course. Course faculty predicted that students would be aided by a proposal template.
Do. An instructional designer worked with course faculty to improve course alignments and the electronic site for ease and utility. Course documents were offered as both a Microsoft® Word file and a PDF to improve access. A student advisor was chosen from the previous semester because she verbalized displeasure with inconsistencies between course faculty and faculty mentor understanding of requirements. A proposal template and rubric were created to guide students with proposal requirements. Course content was added to develop student expertise in identifying a significant measurable and relevant practice problem. Additionally, weekly audio podcasts addressed course readings and common concerns from students. Two face-to-face classes provided content on project requirements and literature review.
Study. The student advisor's course experience feedback was candid, specific, and valuable for course improvement. Course evaluation comments from students were considered by faculty (Table 1). Literature review synthesis supporting the existence of a problem or method to improve was lacking and stylistic errors were common in proposal submissions. Translation methods content was absent from the current course textbooks and students struggled with the methods section of the proposal. Linkages between the problem, methods, and outcomes were unclear in many proposal submissions. Students proposed using research methods and statistics for measurement that were not appropriate for quality improvement projects.
Act. The proposal template needed further refinement for utility, and student scholarly writing skills needed improvement and sometimes distracted from the overall message of the proposal.
PDSA Cycle 2
Plan. Students overwhelmingly rated weekly audio podcasts as very helpful but asked for more face-to-face faculty support. The next cycle was to support student success by providing content and tools to develop the proposal, and the proposal template was revised to include more detail of what is needed in each section. Student use of writing resources was optimized by using health sciences librarians, writing centers, and academic centers for excellence to improve literature review and scholarly writing. The literature review lecture was revised to emphasize evidence, synthesis, and references and linkage to evidence of problems or methods to improve. An Information About Methods document was created to frame this challenging part of the proposal, and a textbook by White, Dudley-Brown, and Terhaar (2016) that includes clear and specific content on translational methods for practice was added (Table 1). An additional face-to-face class was added to introduce the methods section. Using principles of the flipped classroom (Hawkins, 2014), students were asked to come to class prepared through readings, video, and an elevator speech of their problem and methods for small group work. Proposal exemplars from previous students were added to the course site.
Do. The health science librarians attend the first live class session. The librarian tracks student interactions and manages a DNP resource guide, including electronic textbooks and resources.
Study. Student evaluation comments supported the additional methods face-to-face class and were helpful for proposal writing. The DNP proposal template needed further revision, evidenced by continued frustrations that revolved around the difference between quality improvement and research, as well as inconsistent messaging between faculty mentors and course faculty via e-mails, telephone calls, and face-to-face questions from faculty mentors about the course.
Act. Messaging about proposal requirements, problem identification, project planning, and student/mentor team communication need improvement. Faculty and student understanding of the requirements for the proposal and roles of each team member (DNP student, faculty mentor, and practice mentor) also need improvement. Clarity on research versus quality improvement is a persistent gap.
PDSA Cycle 3
Plan. The next PDSA cycle added faculty mentors to the improvement cycle. Course faculty anticipated that faculty mentors desired more information and involvement. Faculty mentor understanding of course requirements and their role and responsibilities for the project needed improvement. Section detail requirements and page limits were considered to improve the DNP proposal template. Updated exemplars aligned with course improvements need to be included.
Do. In a small test of change, faculty mentor training about the course requirements and project process was delivered to a group from one department within our college. Suggested project communication meetings by students with mentors were incorporated in the course assignment document. Page limits on different sections were added to the revised DNP template, forcing brevity in writing. In addition, faculty feedback and student response boxes within the DNP proposal template were added under each section to avoid scrolling. This design is intentional to mirror actual manuscript submission revision.
Study. Course faculty gained a great deal of understanding from the comments and questions during the faculty training. Proposal page limits mandated brevity in writing and promoted synthesis of concepts and the reviewer boxes were well received by the students (Table 1).
Act. Consistent messaging by all faculty needs to be delivered and sustained for student success, and faculty mentors need to understand the requirements of the proposal and project. Some course responsibilities with increasing enrollment can be assumed by a teaching assistant.
PDSA Cycle 4
Plan. To adequately prepare for 85 students in the course in the next semester, training, communication, tools, and resources are necessary to maintain quality. Creation of a course introductory video for both DNP students and faculty mentors would aid in achieving consistent messaging. Additionally, a pre- and post-video knowledge survey about the course for both students and faculty can help course faculty improve content. A process and content for larger scale faculty mentor training to help sustain faculty mentor training and access to on-demand training documents needed to be developed. Addition of a teaching assistant for the course to help with American Psychological Association format in proposal submissions and questions about required project forms would help free faculty for more time for teaching responsibilities.
Do. A two-student team met with course faculty and an instructional designer to create a video describing the proposal course and DNP project from their perspective. The video is embedded in a presentation about the proposal course requirements in the electronic course site and viewed by faculty mentors and DNP students before the start of the proposal course. Based on the small test of change and other lessons learned, content for faculty mentor training was developed and delivered to 50 faculty in four live sessions with remote electronic access for faculty on all campuses. A cloud-based link was shared with all faculty and includes important course document forms and a link to the training video for on-demand learning. An electronic course site shell for faculty mentors was assigned to faculty who attended the training. A DNP project proposal hot-line with e-mails of faculty with experience mentoring projects was shared during training. A checklist for mentors was created for student and faculty team awareness of submission deadlines, project milestones, and to facilitate team communication. As part of the effort to facilitate communication, exemplars now included both faculty and student comments in the reviewer boxes. In order to further aid students, the librarian offered specific one-to-one meeting times for DNP students in the course to assist with the literature review, and a teaching assistant was hired. A course handoff checklist was created and used during the course preparation meeting and to sustain improvements. Course faculty and two DNP program directors presented a special seminar for PhD students and faculty, titled: A Tale of Two Projects, Research versus Quality Improvement.
Study. No comments were made regarding confusion among students in course evaluations (Table 1). Faculty had many questions during the training, especially about the difference between research and quality improvement. Faculty appreciated the trainings and access to materials as needed via the electronic link. Course faculty describe improved scholarly writing and formatting in the proposals.
Act. There is a need to provide consistent and sustainable support for faculty to mentor these projects. Scholarly writing skill and student confidence using translation methods such as quality improvement and outcome measurement needs continued improvement.
We completed four PDSA cycles to improve the course, and our outcomes indicate continual improvement with each successive cycle. The course structure was first aligned with proposal submission assignments; then, tools and resources were added to the electronic course site. Process improvements included the proposal template revisions, library consultation processes, faculty mentor training (live and on demand), and a course update checklist used by course faculty to sustain the changes.
Student evaluation scores for “quality of the course” were consistently rated “excellent” (Figure 2) for five semesters of the course, despite initial confusion about course requirements. Librarian meetings to assist with the literature review for the project were tracked in detail in the last PDSA cycle (38 of 83 students helped in 57 consultations). University writing resource usage tracking depends on the student agreeing to notify course instructors and was not tracked. Fifty-nine faculty mentors attended training and have access to tools and resources to mentor students.
Student evaluations (“Rate the quality of the course.”) are reported on a 1 (poor) to 5 (excellent) scale. The course has a lecture and practicum, separately evaluated and shown as an average in the control chart. Response rates ranged from 47% to 65% (mean = 54%; SD = 8%) of all enrolled students. CL (central line) is the mean; UCL (upper control limit) and LCL (lower control limit) indicate that score variation is due to common cause and the process is in control.
We know the change was an improvement because the voice of the customer comments changed over time—fewer comments about anxiety, confusion, and inconsistent messaging and more about how course structure helped students deliver a scholarly and polished proposal. Faculty mentors are increasingly engaged in the process and offer suggestions for improvement and best practices from their mentoring experience.
The improvement work was conducted in one college of nursing and although lessons can be learned and used by others, the findings are not generalizable.
Using the model for improvement (Langley et al., 2009), PDSA cycles were used to deliver an interactive and engaging course to help students deliver a feasible proposal for the DNP project. We learned together as course faculty that the changes are an improvement because student evaluation scores related to “quality of the course” were favorable, and comments reflecting confusion and anxiety decreased after each cycle. In addition, the quality of students' proposals improved to meet course requirements, and the number of concerns raised by faculty mentors, program directors, and administrators to course faculty diminished substantially.
The next PDSA cycle includes ongoing faculty development, introduction of the DNP project at “DNP Student Launch Day,” an on-demand module or scholarly writing course, content on improvement science and outcome measurement, and a small test of change on a standardized structure for the DNP project. The population health core course is now a prerequisite to the project proposal course to promote a broader view of systems change (Root, Nuñez, Velasquez, Malloch, & Porter-O'Grady, 2018) and for logical sequencing.
With increasing numbers of DNP students, alternate project models that will address the DNP essentials are under consideration. However, as our faculty become comfortable with practice problem identification, proposal requirements and DNP project planning, the process will continue to improve and mature, likely impacting health care outcomes through the scholarly work of advanced practice DNP students.
- American Association of Colleges of Nursing. (2006). The essentials of doctoral education for advanced nursing practice. Retrieved from http://www.aacnnursing.org/Portals/42/Publications/DNPEssentials.pdf
- American Association of Colleges of Nursing. (2015). The Doctor of Nursing Practice: Current issues and clarifying recommendations. Retrieved from http://www.aacnnursing.org/Portals/42/DNP/DNP-Implementation.pdf
- Dols, J.D., Hernández, C. & Miles, H. (2017). The DNP project: Quandaries for nursing scholars. Nursing Outlook, 65, 84–93. doi:10.1016/j.outlook.2016.07.009 [CrossRef]
- Hawkins, D. (2014). Rationale and method for developing team-based learning education. In Hawkins, D. (Ed.), A team-based learning guide for faculty in the health professions (pp. 1–10). Bloomington, IN: AuthorHouse.
- Institute for Healthcare Improvement. (2018). Science of improvement: Testing changes. Retrieved from http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementTestingChanges.aspx
- Kennedy, M.S., Newland, J.A. & Owens, J.K. (2017). Findings from the INANE survey on student papers submitted to nursing journals. Journal of Professional Nursing, 33, 175–183. doi:10.1016/j.profnurs.2016.09.001 [CrossRef]
- Langley, G.J., Moen, R.D., Nolan, K.M., Nolan, T.W., Norman, C.L. & Provost, L.P. (2009). The improvement guide: A practical approach to enhancing organizational performance (2nd ed.). San Francisco, CA: Wiley.
- Lloyd, R.C. (2019). Measuring the voice of the customer. In Lloyd, R. (Ed.), Quality health care: A guide to developing and using indicators (2nd ed., pp. 47–94). Burlington, MA: Jones & Bartlett Learning.
- Mayo, A.M. (2017). Time to define the DNP capstone project. Clinical Nurse Specialist, 31, 63–65.
- Root, L., Nuñez, D.E., Velasquez, D., Malloch, K. & Porter-O'Grady, T. (2018). Advancing the rigor of DNP projects for practice excellence. Nurse Leader, 16, 261–265. doi:10.1016/j.mnl.2018.05.013 [CrossRef]
- White, K.M., Dudley-Brown, S. & Terhaar, M.F. (2016). Translation of evidence into nursing and health care (2nd ed.). New York, NY: Springer. doi:10.1891/9780826117830 [CrossRef]
Voice of the Customer
|“…more direction in writing the methodology section, and choosing framework. Little time was devoted to helping, maybe having more frequent check ins with ability to rewrite or improve writing ability.”
“…more in-person sessions for this course, and possibly should include our faculty mentors. I felt like the class was so dissociated from my mentors and I was constantly the middle man—it would be helpful if there was more communication there to improve the process of writing this proposal. This being the very first time writing any kind of proposal, and also without a clear idea of what a DNP [Doctor of Nursing Practice] project should entail, I felt very lost in this course a lot of the time.”
“Some of the directions of writing was vague with minimal direction, especially writing the methods section. Not everyone is a scholarly writer... yet. We are still students learning.”|
|Course faculty to DNP program director about new course text: “White et al. (White, Dudley-Brown, & Terhaar, 2016) suggests 15 methods that include concepts such as academic detailing, audit and feedback, bundles, toolkits, and clinical pathways. The articulation of these methods by the authors seems groundbreaking to me as students (and perhaps faculty, too) seem to be struggling a great deal with coming up with methods that are not research (i.e., pretest/posttest design).”
“The course module was so organized and packed with practical information! I loved the examples of the DNP proposals. I also liked that we met at least three times (not too much helped me juggle work, school, and life in general). During class, it was really great to break out in small groups to get feedback from one another….”
“PhD-prepared faculty are used to a completely different product (a research study) and seem to still be learning the difference between the DNP project and a dissertation.”
“Course faculty need better understanding of the next steps and time line for the project, not just what we need to get done for the proposal.”
“The assignment submission was very cumbersome; responding to the comments in the document and then at the end of the document, following the document in edit mode etc.”|
|“The in-class meetings were beneficial. Additionally, the communication on the assignment was excellent. The feedback between the professor and student really helped at least for me see where I needed to continue to change and improve my proposal.”
“Meeting my fellow classmates was good to learn more about projects, and assisted with idea sharing.
To improve the course—providing the project mentors with guidelines to guide us.”|
|“I don't know that the class needs to be meet in person. Could all participants join electronically?”
“The in-person class really helped me prepare for each submission!”
“Feedback from the instructors to strengthen my proposal was great. I like the format (using the feedback boxes).”
Comments from the pre and post survey about the course introduction video for both faculty and students:
Student: “It gave me a general overview of the course and put myself at ease in regards to the course load.”
Faculty: “I thought the video offers helpful, proactive suggestions to keep students on track for completing their projects.”|
|“This course walked me through every step necessary to produce well-constructed DNP proposal. I'm in amazement that I created this!”
“I can't believe I put this whole thing together on my own and it looks pretty good.”|