The assessment of clinical competencies is an essential requirement of health professions education (Bartfay, Rombough, Howse, & Leblanc, 2004; Robinson-Smith, Bradley, & Meakim, 2009). Among assessment methods, the Objective Structured Clinical Examination (OSCE) has emerged as the gold standard for the assessment of clinical competence of health professionals. The OSCE provides objectivity, structure, simulation of real clinical situations, and an assessment of various skills and competencies, using a wide range of examiners and consistency, as students experience the same questions, difficulty, and examiners (Hodges, Hanson, McNaughton, & Regehr, 2002; Selim, Ramadan, El-Gueneidy, & Gaafer, 2012; Walsh, Bailey, & Koren, 2009).
Although the OSCE has become a well-established method of competency assessment in nursing and medical education (Alinier, 2003; Hermanns, Lilly, & Crawley, 2011; Mitchell, Henderson, Groves, Dalton, & Nulty, 2009; Szpak & Kameg, 2013), it has been only recently introduced to psychiatric nursing education (Selim et al., 2012).
A limitation of the OSCE is the required complex cost of organizing and matching each individual student with a simulated patient and an assessor (Hodges & Lofchy, 1997; Hulsman, Mollema, Oort, Hoos, & de Haes, 2006; Iqbal, Khiza, & Zaidi, 2009). Budgeting and costs of the OSCE fall into the following major categories: standardized patient training and performance costs, examiner costs, support staff costs, supplies, space and equipment, catering, and psychometric analysis costs (Hodges et al., 2002; Wallace, Rao, & Haslam, 2002). The major barriers faced by the authors in conducting the OSCE were a lack of adequate real standardized patients, a lack of OSCE laboratories, an increasing number of enrolled nursing students, increasing restrictions on allowing nursing students to care for psychiatric patients, and limited psychiatric clinical placements and resources.
To overcome these obstacles in the current study, standardized videotaped case studies in a computerized Objective Structured Video Examination (OSVE) were adapted to measure the cognitive scripts underlying video demonstrations of psychopathology and psychotropic medications’ side effects, using questions that assess knowledge, understanding, and performance.
The OSVE was first developed by Humphris and Kaney (2000) as a video-based written examination that proved to be efficient, quick, and reliable. The OSVE was designed to assess medical students’ recognition and understanding of the consequences of various communication skills. In the OSVE, students watch video cases, followed by written questions that test students’ ability to recognize communication skills and offer alternative responses to enhance the interactions. The OSVE is a quick, economical, and reliable assessment tool to assess a large number of students simultaneously in the classroom (Hulsman, Mollema, Hoos, de Haes, & Donnison-Speijer, 2004; Humphris & Kaney, 2000; Zick, Granieri, & Makoul, 2007). The value of the OSVE relies in assessing covert cognitive scripts (implicit knowledge) underlying overt communication behavior, which is demonstrated in the videos. The OSVE has emerged as a new assessment of knowledge and understanding of communication skills (Hulsman et al., 2006; Humphris, 2002; Srinivasan, Hwang, West, & Yellowlees, 2006).
The use of the OSVE is not limited to assessing medical students’ communication skills; it has been adapted by several researchers in developing and assessing other clinical skills, such as surgical skills (El Shallaly & Ali, 2004; El Shallaly & Mekki, 2012); spinal microsurgical skills (Heiland et al., 2014); neurology (Lieberman, Frye, Litwins, Rasmusson, & Boulet, 2003); professionalism, system-based practice, and practice-based learning in geriatrics (Simpson et al., 2006); and clinical reasoning and decision making in medical exit examinations (Sulaiman & Hamdy, 2013).
To the authors’ knowledge, the literature lacks studies concerning the use of OSVE in assessing clinical psychiatric nursing competencies. Accordingly, an OSVE was conducted to assess undergraduate nursing students’ knowledge, observation, and clinical reasoning in clinical psychiatric nursing.
Aim of the Study
The current study aimed to assess:
- The validity and reliability of the OSVE as an assessment method in a clinical psychiatric and mental health nursing course.
- Nursing students’ evaluation of the OSVE in a clinical psychiatric and mental health nursing course.
Design and Setting
A quantitative descriptive research design was used in the current study.
Ethical approval was received from the research unit at the college of nursing to use data concerning students’ marks (Note: a mark is a numerical score of the examination, whereas grades are the transference of marks into letter levels of performance, such as A+, A, B+) and their evaluations of the psychiatric and mental health nursing OSVE for research purposes. After the nature and aim of the current study was explained, informed written consent was obtained from each participant who agreed to complete the OSVE Student Evaluation Questionnaire.
All of the students enrolled in the psychiatric and mental health nursing course in two consecutive academic semesters—spring 2012 (n = 61) and fall 2013 (n = 26)—were included in the study. The total number of participants was 87 nursing students who took the psychiatric and mental health nursing OSVE. The OSVE has been used in the psychiatric and mental health nursing course since the spring semester of the 2012 academic year as an assessment method that substitutes for the OSCE, which was difficult to administer due to the obstacles discussed earlier in this article.
The current study was conducted using the OSVE and the OSVE Student Evaluation Questionnaire.
Psychiatric and mental health nursing clinical assessment at the college of nursing is conducted through a variety of formative and summative methods that include the OSVE (midterm and summative final), role-playing, clinical seminar student presentations, case study presentations, supervised clinical student–patient interactions, mental status examinations, nursing care plans, and the psychoeducation of patients and their families. Written multiple choice question examinations (mid-term and summative final) are performed to assess both the clinical and theory components of the course (Table 1).
Methods of Student Assessment in a Psychiatric and Mental Health Nursing Course
Description of the OSVE
OSVE Preparation. Although the OSVE was originally used to measure medical students’ knowledge and understanding of communication skills (Hulsman et al., 2006; Humphris & Kaney, 2000; Karabilgin, Vatansever, Caliskan, & Durak, 2012), it was used in the current study as a video-based written test to assess students’ observation skills, clinical reasoning, and critical thinking skills in the identification and understanding of psychopathology and medication side effects and complications, as well as the writing of the patients’ nursing care plan.
Using videos in the OSVE was not a new experience for the psychiatric nursing students. Each semester, the students were introduced to videos supporting each clinical seminar and theory lecture. The videos comprise a wide range of psychiatric nursing topics, such as assessment using a mental status examination, psychopathology, psychiatric nursing care plan, clinical picture, management of a variety of psychiatric disorders, psychotropic medication side effects, psychiatric emergencies, and case study presentation. The daily debriefing of students’ clinical training involved a video display and discussion of the scenarios.
In the eighth week of the semester, the students were exposed to a small-scale midterm OSVE that comprised three stations—two video stations (communication techniques, hallucinations, and delusions) and one written case scenario station (agranulocytosis). The midterm OSVE was developed by the course instructors (A.A.S., E.D.), following the same process as the development of the final OSVE. The midterm OSVE was worth 5% (five marks) of the total course marks.
The current final OSVE was developed by the course instructors and was composed of six working stations to assess the achievement of student learning outcomes in the clinical psychiatric and mental health nursing course (Table 2). The examination included two video stations, one laboratory investigation report station, three written case scenario stations, and two rest stations (Table 1). The current OSVE is a final assessment, and its mark constituted 10% (10 marks) of the whole course mark.
Objective Structured Video Examination (OSVE) Reliability: Internal Consistency Among the OSVE Stations
The number, contents, and tasks of the stations were developed to measure the achievement of student learning outcomes in the clinical psychiatric and mental health nursing course. The questions and tasks reflected those taught in the course.
Content validity in terms of clarity domain and adequacy of the OSVE content domain were secured by a thorough review of the examination by the course instructors and a panel of six psychiatric nursing faculty experts from various nursing colleges in the Middle East. The experts judged questions for their relevance, clarity, and representativeness of the clinical content and the learning objectives of the course. In addition, the panel evaluated the model answer and the rubric and distribution of scores for each station. The final version of the examination and the model answer were obtained upon achievement of consensus among the experts and course instructors.
An OSVE booklet was distributed to the students at the beginning of the examination. The booklet included general instructions for the examination; an answer sheet for each station, composed of required short essay questions; a detailed rubric of each question; and the time limit.
The rubric and model answer sheet were developed and structured for each OSVE station by the course instructors to rate the examination. Each model answer sheet included all of the possible correct answers to each question and a detailed rubric for each. The rubric of each station included the distribution of marks and grades on the required tasks (Table 2). Three course instructors were trained by the authors on marking the OSVE stations answer sheets against the model answer during an intensive 2-hour workshop. Each rater was assigned two different stations to ensure consistency and reliability of the marking process.
Implementation of the Final OSVE. Following students’ feedback and recommendations after the midterm OSVE, a 1-hour orientation session was conducted 2 weeks before administering the final OSVE for all students enrolled in the course. In addition, a mock OSVE (rehearsal examination with no marks) was conducted 1 week before the final OSVE, using different videos and scenarios to familiarize the students with the comprehensive nature of the examination to decrease their level of anxiety and to ensure the ultimate understanding of the experience.
The students took the final OSVE in the 16th week of the semester in a lecture hall with a Blackboard™ system that allowed for video demonstrations. All of the students were gathered in one lecture hall and were offered an OSVE booklet for the paper-and-pencil answer format, which was collected by the course instructors and teaching assistants at the end of the examination. The OSVE was conducted in 59 minutes, and all of the students experienced the same videos, written scenarios, and questions simultaneously.
The start and end time for each station and the movement from one station to another was guided by the course instructors and teaching assistants. Prior to the video display, the students read the instructions, the corresponding questions, and the time allowed at each video station. The videos were displayed only once for the students.
OSVE Student Evaluation Questionnaire
After reviewing the related literature (Alinier, 2003; McWilliam & Botwinski 2010; Nulty, Mitchell, Jeffrey, Henderson, & Groves, 2011; Pierre, Wierenga, Barton, Branday, & Christie, 2004; Sauer, Hodges, Santhouse, & Blackwood, 2005; Selim et al., 2012), the course instructors developed a self-administered, semistructured questionnaire to elicit the students’ evaluation of the psychiatric and mental health nursing OSVE experience. The questionnaire was composed of two parts. The first part included 27 statements; each statement was rated on a Likert scale of 1 to 3 (1 = disagree, 2 = neutral, 3 = agree). The rating of the negative statements (statements 3, 5, and 11) were reversed (i.e., 1 = agree, 2 = neutral, 3 = disagree). The total questionnaire scores ranged from 27 to 81, with higher scores indicating positive views of the OSVE.
In the second part of the questionnaire, students were asked to rate each station as a whole on a scale of 0 to 5, with 0 indicating the lowest score and 5 indicating the highest score. An additional open-ended question was added at the end of the questionnaire, inviting the students to share their suggestions and recommendations to improve on future OSVEs in psychiatric nursing. The students were assured that their evaluation of the OSVE was voluntary and anonymous and would not affect their grades. They were informed that their evaluation would be used for research purposes only and for the future improvement of the examination structure and implementation. The students’ evaluation questionnaire was administered immediately after finishing the last station in the OSVE. The reliability of the questionnaire was calculated using Cronbach’s alpha, with a value of α = 0.9, indicating strong/excellent reliability of the questionnaire. Students’ recommendations and suggestions to improve the OSVE were analyzed according to seven themes.
Data were entered and analyzed using SPSS® version 16.0. Descriptive analyses, including frequencies and percentages, were performed. Criterion validity of the OSVE was calculated using Pearson’s correlation coefficient of the final written examination. Cronbach’s alpha was used to test the internal consistency of the OSVE stations and the reliability of the OSVE students’ evaluation questionnaire.
Validity and Reliability of the OSVE
The validity of the OSVE was tested against the summative written examination as a gold standard reference, as a comprehensive examination covering a wide range of knowledge related to the clinical portion of the course. The OSVE validity analysis detected a statistically significant correlation between the OSVE and the summative written examination in psychiatric nursing (r = 0.6, p = 0.000). The OSVE showed acceptable reliability (α = 0.714) when calculating the internal consistency among the six OSVE stations (Table 2).
Students’ Marks in the OSVE
The students’ total mean mark on the OSVE was above average—6.88 of 10 (68.8%). The students’ highest scores were recorded in the written scenario stations—nursing care of patients before receiving electroconvulsive therapy (station 3, 89.6%) and suicide attempt management (station 2, 80%). Above-average scores were attained at the video stations of visual and auditory hallucinations and delusion of persecution (72.3%) and tardive dyskinesia (70.8%). The lowest, below-average scores were in station 5, laboratory investigation report (55.3%), and station 6, written scenario of hypertensive crisis (55.3%).
Student Evaluation of the OSVE
The total score of the student evaluation questionnaire of the OSVE ranged from 41 to 79, with a mean score of 64.17 ± 9.01 (Table 3). The students’ positive view of the OSVE was reflected in the following: the majority of students agreed that the OSVE covered a wide area of knowledge (74.7%); the OSVE instructions were clear (82.1%); the quality of the videos sounds and pictures were clear (79.3%); the videos simulated psychopathology exhibited by real patients (73.6%); and that the written scenarios were clear, concrete, and easily understood (78.2%). Moreover, more than half of the students agreed that the OSVE was fair (58.8%), was well-administered (59%), was well-structured and sequenced (65.5%), covered a wide range of clinical skills (69%), the OSVE tasks reflected those learned in the course (58.3%), the tasks required in each station were fair (53.5%), the sequence of stations was logical and appropriate (66.3%), the OSVE provided opportunities to learn (57.5%), the grades were clearly identified (69.4%), and the OSVE was a practical and useful experience (62.8%). In addition, 57% of the students disagreed that the OSVE was tricky and indirect.
Students’ Evaluation of the Psychiatric and Mental Health Nursing Objective Structured Video Examination (N = 87)
Nearly half of the students reported that the grades provided a true measure of the essential clinical skills in psychiatric nursing (48.3%), they were aware of the level of information needed for the OSVE (49.4%), and they were fully aware of the nature of the OSVE (47.7%). However, more than one third of the students agreed that the OSVE allowed students to compensate in some areas (37.9%), it highlighted areas of students’ weakness in the course (36.5%), and approximately one fourth of students agreed that the OSVE minimized the chance of failing (25.6%).
Regarding the students’ negative perspectives, more than half of the students agreed that they needed more time at some stations (55.3%) and that it was very stressful (52.3%), whereas 41.4% of the students disagreed with the statement that the OSVE was less stressful than other examinations.
The students’ overall evaluation of the six OSVE stations ranged from 0 to 30, with a mean total score of 22.75 ± 4.92 (Table 4). More than half of the students rated station 3, which had a written scenario, as a 5 of 5. Video stations 1 and 4 were rated as a 5 of 5 by 49.4% and 41%, respectively, of the students. More than 10% of students rated station 5 (laboratory investigation report) as a 0 of 5.
Students’ Ratings of the Objective Structured Video Examination Stations (N = 87)
Only 13 (14.94%) students gave their feedback to the open-ended questions inviting them to reflect on their suggestions and recommendations to improve the OSVE. The students’ comments were that the examination was a great experience (3 students); the examination was OK/good (3 students); too many stations and clinical contents and tasks (3 students); the examination required more time (3 students); students’ anxieties need to be worked on before the examination (1 student); the number of instructors during the examination should be decreased, as it is stressful (1 student); and it is better to conduct oral instead of written OSVE (1 student).
Developing and implementing a summative OSVE in a psychiatric and mental health nursing course proved to be a successful experience in the face of increasing numbers of students enrolled in the course, the limited number of real psychiatric patients and faculty in the field, and allocated resources. The success of any summative measure of students’ clinical performance must be based on the psychometric properties of the assessment in terms of validity and reliability (McWilliam & Botwinski, 2010; Turner & Dankoski, 2008). Hodges et al. (2002) argued that the basic psychometric data for an examination included the overall examination mean score and standard deviation, mean and standard deviation for each station, highest grade, lowest grade, and Cronbach’s alpha.
Assessing the validity and reliability of the OSVE was one of the current study’s aims. Conducting the OSVE demonstrated validity against the summative written examination. Written examinations are considered objective, given that all of the students are presented with the same questions at the same time, and that they cover a wide range of knowledge (McLay, Rodenhauser, Anderson, Stanton, & Markert, 2002). Further, the stations’ contents, clarity, sequence, allotted time, and grades related to the relevance, clarity, and representativeness to the clinical content and learning objectives of the course were rigorously reviewed by experts in the field. In addition, the OSVE showed acceptable reliability in terms of internal consistency. Several studies indicated similar validity and reliability of the OSVE (Baribeau, Mukovozov, Sabljic, Eva, & deLottinville, 2012; Humphris & Kaney, 2000; Lieberman et al., 2003). Hulsman et al. (2004, 2006) reported that the OSVE reliability benefits from highly standardized assessment and ratings. In the OSVE, each station was presented exactly in the same way to every student, and a structured manual (model answers) was used for rating (grading), thus reducing any variation. In addition, Hulsman et al. (2006) stated that OSVE reliability can be improved by increasing the number of cases to seven.
The OSVE highlighted students’ strengths and weaknesses relating to the material covered in the clinical psychiatric and mental health nursing course. The highest and above-average grades were obtained in the written scenario stations of nursing care of a patient before receiving electroconvulsive therapy and suicide attempt management and the video stations of visual and auditory hallucinations and delusion of persecution and tardive dyskinesia. These findings reflect the effort and time spent throughout the course on patient assessments by using the mental status examination at clinical placement sites in which the student must provide evidence of objective and subjective patient data to validate psychopathology, the patient’s needs and complaints, and the formulation of a relevant nursing care plan.
The lowest below-average grades were at the laboratory investigation report of severe lithium toxicity (station 5) and the written scenario about a hypertensive crisis (station 6). In light of these results, it appears that the OSVE highlighted students’ weakness in psychiatric and mental health nursing course material and psychopharmacology, which calls for further review of the teaching and learning methods used by the students and the course instructors. Although the topics in stations 5 and 6 are of clinical significance because they represent psychiatric emergencies that are emphasized in the curriculum, they were not encountered by the students in their clinical training, which might explain the low marks obtained in those stations. The low marks obtained in stations 5 and 6 affected the total mean score of the students (6.89 of 10), as their scores ranged from 2 to 10. Hulsman et al. (2006) reported a similar average OSVE score of 60.8 of 100 by medical students who completed three video cases for the assessment of their communication skills.
One of the current study’s strengths is the students’ evaluation of the OSVE. In the current study, the students’ main positive evaluation statements were that the OSVE covered a wide area of knowledge, was clear, covered a wide range of clinical skills, simulated real clinical settings, was well-structured, and was not tricky. Similar positive appraisal of the OSVE by the examinees was reported in the studies by El Shallaly and Ali (2004); El Shallaly and Mekki (2012), and Sulaiman and Hamdy (2013). The video stations were positively evaluated by the students in terms of the quality of sound and display and of simulation to real patients. The same results were reported by Hulsman et al. (2004) and Sulaiman and Hamdy (2013). On the other hand, the students negatively evaluated the OSVE as being stressful and requiring more time at the stations, which was also reported in other studies (El Shallaly & Ali, 2004; El Shallaly & Mekki, 2012; Sulaiman & Hamdy, 2013).
Some of the themes of the open-ended question reemphasized the students’ evaluation of the OSVE. The students reported that the OSVE required more time and had too many stations, too much clinical content, and too many tasks. In addition, the students commented that the examination was a good to a great experience, that students’ anxieties need to be alleviated before the examination, and that the number of instructors should be decreased during the examination, as it was a source of stress.
The OSVE used a small number of video stations (two) due to limited resources and difficulty in allocating professional videos that simulate real psychiatric patients from the region.
The OSVE cannot assess the performance of communication and interview skills that are essential competencies in psychiatric and mental health nursing. In other words, the OSVE cannot measure actual behavioral and performance competencies in psychiatric and mental health nursing; it measures only student knowledge and understanding of clinical competencies (Hulsman et al., 2006). Hence, the OSVE cannot be used as the sole assessment method to assess clinical psychiatric and mental health nursing competencies.
The current OSVE has proven to be a valid, reliable, time saving, efficient, and low-cost summative assessment method of a clinical psychiatric and mental health nursing course that overcomes the assessment problems resulting from increased numbers of nursing students and limited faculty numbers and learning resources. The OSVE could be used among other assessment methods, as it rates the achievement of student knowledge and cognitive learning outcomes related to psychiatric nursing clinical competencies.
- Alinier, G. (2003). Nursing students’ and lecturers’ perspectives of objective structured clinical examination incorporating simulation. Nurse Education Today, 23, 419–426. doi:10.1016/S0260-6917(03)00044-3 [CrossRef]
- Baribeau, D.A., Mukovozov, I., Sabljic, T., Eva, K.W. & deLottinville, C.B. (2012). Using an objective structured video exam to identify differential understanding of aspects of communication skills. Medical Teacher, 34, 242–250. doi:10.3109/0142159X.2012.660213 [CrossRef]
- Bartfay, W.J., Rombough, R., Howse, E. & Leblanc, R. (2004). Evaluation. The OSCE approach in nursing education. Canadian Nurse, 100(3), 18–23.
- El Shallaly, G.E. & Ali, E. (2004). Use of video-projected structured clinical examination (ViPSCE) instead of the traditional oral (Viva) examination in the assessment of final year medical students. Education for Health, 17, 17–26. doi:10.1080/13576280310001656123 [CrossRef]
- El Shallaly, G.E. & Mekki, A.M. (2012). Use of computer-based clinical examination for assessment of medical students in surgery. Education for Health Change in Learning & Practice, 25, 148–152. doi:10.4103/1357-6283.109789 [CrossRef]
- Heiland, D.H., Petridis, A.K., Maslehaty, H., Thissen, J., Kinzel, A., Scholz, M. & Schreiber, L. (2014). Efficacy of a new video-based training model in spinal surgery. Surgical Neurology International, 5, 1–5. doi:10.4103/2152-7806.124973 [CrossRef]
- Hermanns, M., Lilly, M.L. & Crawley, B. (2011). Using clinical simulation to enhance psychiatric nursing training of baccalaureate students. Clinical Simulation in Nursing, 7, 41–46. doi:10.1016/j.ecns.2010.05.001 [CrossRef]
- Hodges, B., Hanson, M., McNaughton, N. & Regehr, G. (2002). Creating, monitoring, and improving a psychiatry OSCE: A guide for faculty. Academic Psychiatry, 26, 135–161. doi:10.1176/appi.ap.26.3.134 [CrossRef]
- Hodges, B. & Lofchy, J. (1997). Evaluating psychiatric clinical clerks with a mini objective structured clinical examination. Academic Psychiatry, 21, 219–225. doi:10.1007/BF03341435 [CrossRef]
- Hulsman, R.L., Mollema, E.D., Hoos, A.M., de Haes, J.C. & Donnison-Speijer, J.D. (2004). Assessment of medical communication skills by computer: Assessment method and student experiences. Medical Eduction, 38, 813–824. doi:10.1111/j.1365-2929.2004.01900.x [CrossRef]
- Hulsman, R.L., Mollema, E.D., Oort, F.J., Hoos, A.M. & de Haes, J.C. (2006). Using standardized video cases for assessment of medical communication skills: Reliability of an objective structured video examination by computer. Patient Education and Counseling, 60, 24–31. doi:10.1016/j.pec.2004.11.010 [CrossRef]
- Humphris, G.M. (2002). Communication skills knowledge, understanding and OSCE performance in medical trainees: A multivariate prospective study using structural equation modelling. Medical Education, 36, 842–852. doi:10.1046/j.1365-2923.2002.01295.x [CrossRef]
- Humphris, G.M. & Kaney, S. (2000). The objective structured video exam for assessment of communication skills. Medical Education, 34, 939–945. doi:10.1046/j.1365-2923.2000.00792.x [CrossRef]
- Iqbal, M., Khiza, B. & Zaidi, Z. (2009). Revising an objective structured clinical examination in a resource-limited Pakistani medical school. Education for Health, 22, 209.
- Karabilgin, O.S., Vatansever, K., Caliskan, S.A. & Durak, H.I. (2012). Assessing medical student competency in communication in the pre-clinical phase: Objective structured video exam and SP exam. Patient Education and Counseling, 8, 293–299. doi:10.1016/j.pec.2011.10.008 [CrossRef]
- Lieberman, S.A., Frye, A.W., Litwins, S.D., Rasmusson, K.A. & Boulet, J.R. (2003). Introduction of patient video clips into computer-based testing: effects on item statistics and reliability estimates. Academic Medicine, 78(10 Suppl.), 48–51. doi:10.1097/00001888-200310001-00016 [CrossRef]
- McLay, R.N., Rodenhauser, P., Anderson, D.S., Stanton, M.L. & Markert, R.J. (2002). Simulating a full-length psychiatric interview with a complex patient: An OSCE for medical students. Academic Psychiatry, 26, 162–167. doi:10.1176/appi.ap.26.3.162 [CrossRef]
- McWilliam, P. & Botwinski, C. (2010). Developing a successful nursing objective structured clinical examination. Journal of Nursing Education, 49, 36–41. doi:10.3928/01484834-20090915-01 [CrossRef]
- Mitchell, M.L., Henderson, A., Groves, M., Dalton, M. & Nulty, D. (2009). The objective structured clinical examination (OSCE): Optimising its value in the undergraduate nursing curriculum. Nurse Education Today, 29, 398–404. doi:10.1016/j.nedt.2008.10.007 [CrossRef]
- Nulty, D.D., Mitchell, M.L., Jeffrey, C.A., Henderson, A. & Groves, M. (2011). Best practice guidelines for use of OSCEs: Maximising value for student learning. Nurse Education Today, 31, 145–151. doi:10.1016/j.nedt.2010.05.006 [CrossRef]
- Pierre, R.B., Wierenga, A., Barton, M., Branday, J.M. & Christie, C.D. (2004). Student evaluation of an OSCE in paediatrics at the University of the West Indies, Jamaica. BMC Medical Education, 4, 22. doi:10.1186/1472-6920-4-22 [CrossRef]
- Robinson-Smith, G., Bradley, P.K. & Meakim, C. (2009). Evaluating the use of standardized patients in undergraduate psychiatric nursing experiences. Clinical Simulation in Nursing, 5, 203–211. doi:10.1016/j.ecns.2009.07.001 [CrossRef]
- Sauer, J., Hodges, B., Santhouse, A. & Blackwood, N. (2005). The OSCE has landed: One small step for British psychiatry?Academic Psychiatry, 29, 310–315. doi:10.1176/appi.ap.29.3.310 [CrossRef]
- Selim, A.A., Ramadan, F.H., El-Gueneidy, M.M. & Gaafer, M.M. (2012). Using objective structured clinical examination (OSCE) in undergraduate psychiatric nursing education: Is it reliable and valid?Nurse Education Today, 32, 283–288. doi:10.1016/j.nedt.2011.04.006 [CrossRef]
- Simpson, D., Helm, R., Drewniak, T., Ziebert, M.M., Brown, D., Mitchell, J. & Duthie, E. (2006). Objective structured video examinations (OSVEs) for geriatrics education. Gerontology Geriatrics Education, 26 (4), 7–24. doi:10.1300/J021v26n04_02 [CrossRef]
- Srinivasan, M., Hwang, J.C., West, D. & Yellowlees, P.M. (2006). Assessment of clinical skills using simulator technologies. Academic Psychiatry, 30, 505–515. doi:10.1176/appi.ap.30.6.505 [CrossRef]
- Sulaiman, N.D. & Hamdy, H. (2013). Assessment of clinical competencies using clinical images and videos “CIVA”.BMC Medical Education, 13(78), 1–6. doi:10.1186/1472-6920-13-78 [CrossRef]
- Szpak, J.L. & Kameg, K.M. (2013). Simulation decreases nursing student anxiety prior to communication with mentally ill patients. Clinical Simulation in Nursing, 9, 13–19. doi:10.1016/j.ecns.2011.07.003 [CrossRef]
- Turner, J.L. & Dankoski, M.E. (2008). Objective structured clinical exams: A critical review. Family Medicine, 40, 574–578.
- Wallace, J., Rao, R. & Haslam, R. (2002). Simulated patients and objective structured clinical examinations: Review of their use in medical education. Advances in Psychiatric Treatment, 8, 342–350. doi:10.1192/apt.8.5.342 [CrossRef]
- Walsh, M., Bailey, P.H. & Koren, I. (2009). Objective structured clinical evaluation of clinical competence: An integrative review. Journal of Advanced Nursing, 65, 1584–1595. doi:10.1111/j.1365-2648.2009.05054.x [CrossRef]
- Zick, A., Granieri, M. & Makoul, G. (2007). First-year medical students’ assessment of their own communication skills: A video-based, open-ended approach. Patient Education and Counseling, 68, 161–166. doi:10.1016/j.pec.2007.05.018 [CrossRef]
Methods of Student Assessment in a Psychiatric and Mental Health Nursing Course
|Assessment Method||Proportion of the Total Assessment Mark (%)|
|OSVE, summative final||10|
|Professionalism and student conduct||5|
|Patient and family psychoeducation||5|
|Mental status examination||4|
|Nursing care plan||3|
|Clinical seminar presentation||5|
|Written multiple choice questions examination, midterm||15|
|Written multiple choice questions examination, summative final||40|
|OSVE Stations Blueprint|
|Station No./Content||Required Task/Short Essay Question||Station Rubric/Mark||Duration (Minutes)|
|1: Video about one of the extrapyramidal side effects of antipsychotic medications (tardive dyskinesia)||Identify the demonstrated side effect/List other extrapyramidal side effects of antipsychotic medications.||1.0||5|
|2: Written scenario about a psychiatric patient who is newly admitted to the hospital and diagnosed with schizophrenia and suicide attempt||Write a priority nursing care plan for this patient.||1.75||7|
|3: Written scenario about a patient with major depression who will receive electroconvulsive therapy||List the nursing preparation for this patient before receiving electroconvulsive therapy.||1.25||5|
|4: Video representing a patient diagnosed with schizophrenia who is experiencing active psychotic symptoms (auditory and visual hallucinations and delusion of persecution)||Identify psychopathology and symptoms demonstrated by the patient in the video/Formulate one relevant nursing care plan.||2.75||12|
|5: Laboratory investigation report indicating severe lithium toxicity of a patient diagnosed with bipolar disorder||Identify the problem/List four signs and symptoms of the identified problem.||1.5||5|
|6: Written scenario about a patient experiencing a hypertensive crisis. After managing the crisis, a medical consultation recommended changing the patient’s current class of antidepressant to one of the SSRI class.||Identify the cause of the hypertensive crisis/Write about the nursing interventions and patient education regarding shifting the patient from MAOIs to SSRIs.||1.75||10|
|Total duration of the OSVE||59|
Objective Structured Video Examination (OSVE) Reliabilitya: Internal Consistency Among the OSVE Stations
|OSVE Station||Station Rubric/Mark||Mean Student Mark||Percentage of Mean Mark|
|Station 1: Video (tardive dyskinesia)||1.0||0.7 ± 0.36||70.8|
|Station 2: Written case scenario (suicide attempt management)||1.75||1.41 ± 0.43||80|
|Station 3: Written case scenario (nursing care before receiving electroconvulsive therapy)||1.25||1.12 ± 0.28||89.6|
|Station 4: Video (auditory and visual hallucinations and delusion of persecution)||2.75||1.99 ± 0.47||72.3|
|Station 5: Laboratory investigation report (severe lithium toxicity)||1.5||0.68 ± 0.54||45.4|
|Station 6: Written case scenario (hypertensive crisis)||1.75||0.97 ± 0.49||55.3|
|Total OSVE mark||10||6.89 ± 1.66 (range = 2 to 10)||68.8|
Students’ Evaluation of the Psychiatric and Mental Health Nursing Objective Structured Video Examination (N = 87)
|Statement||Agree (n [%])||Neutral (n [%])||Disagree (n [%])||Mean (SD)||Missing Data|
|1. The OSVE was fair.||50 (58.8)||32 (37.6)||3 (3.5)||2.55 (0.57)||2|
|2. The OSVE covered a wide area of knowledge.||65 (74.7)||20 (23)||2 (2.3)||2.72 (0.499)||—|
|3. The OSVE needed more time at the stations.||47 (55.3)||27 (31.8)||11 (12.9)||1.58 (0.71)||2|
|4. The OSVE was well administered.||49 (59)||31 (37.3)||3 (3.6)||2.55 (0.57)||4|
|5. The OSVE was very stressful.||45 (52.3)||25 (29.1)||16 (18.6)||1.66 (0.78)||1|
|6. The OSVE was well structured and sequenced.||57 (65.5)||30 (34.5)||—||2.66 (0.48)||—|
|7. The OSVE minimized the chance of failing.||21 (25.6)||44 (53.7)||17 (20.7)||2.05 (0.68)||5|
|8. The OSVE was less stressful than other types of examinations.||20 (23)||31 (35.6)||36 (41.4)||1.82 (0.79)||0|
|9. The variety of difficulty levels of the OSVE gave students the opportunity to compensate in other questions or tasks.||33 (37.9)||45 (51.7)||9 (10.3)||2.28 (0.64)||0|
|10. The OSVE highlighted areas of students’ weakness in the course.||31 (36.5)||39 (45.9)||15 (17.6)||2.19 (0.72)||2|
|11. The OSVE questions were unclear and misleading.||9 (10.5)||28 (32.6)||49 (57)||2.47 (0.68)||1|
|12. Students were aware of level of information needed for the OSVE.||42 (49.4)||31 (36.5)||12 (14.1)||2.35 (0.72)||2|
|13. The OSVE covered a wide range of clinical skills.||60 (69)||25 (28.7)||2 (2.3)||2.67 (0.52)||0|
|14. You were fully aware of the nature of the OSVE.||41 (47.7)||36 (41.9)||9 (10.5)||2.37 (0.67)||1|
|15. The OSVE tasks reflected those learned in the course.||49 (58.3)||29 (34.5)||6 (7.1)||2.51 (0.63)||3|
|16. Time at each station was adequate.||30 (35.3)||29 (34.1)||26 (30.6)||2.05 (0.82)||2|
|17. The OSVE instructions were clear.||69 (82.1)||13 (15.5)||2 (2.4)||2.8 (0.46)||3|
|18. The tasks required in each station were fair.||46 (53.5)||34 (39.5)||6 (7)||2.47 (0.63)||1|
|19. The sequence of stations was logical and appropriate.||57 (66.3)||28 (32.6)||1 (1.2)||2.65 (0.5)||1|
|20. The OSVE provided opportunities to learn.||50 (57.5)||31 (35.6)||6 (6.9)||2.51 (0.63)||—|
|21. The OSVE grades provided a true measure of essential clinical skills in psychiatric nursing.||42 (48.3)||38 (43.7)||7 (8)||2.40 (0.64)||—|
|22. The OSVE grades were clearly identified.||59 (69.4)||20 (23.5)||6 (7.1)||2.62 (0.62)||2|
|23. The OSVE was a practical and useful experience.||54 (62.8)||26 (30.2)||6 (7)||2.56 (0.63)||1|
|24. The OSVE eliminated any chance of personal bias of the instructors toward any students.||34 (39.1)||47 (54)||6 (6.9)||2.32 (0.6)||—|
|25. The quality of sounds and pictures of the videos were clear.||69 (79.3)||17 (19.5)||1 (1.1)||2.78 (0.44)||—|
|26. The videos simulated psychopathology exhibited by real patients.||64 (73.6)||22 (25.3)||1 (1.1)||2.72 (0.48)||—|
|27. The written scenarios were clear, concrete, and easily understood.||68 (78.2)||19 (21.8)||—||2.78 (0.42)||—|
|OSVE total study score range = 41 to 79a||—|
|OSVE mean total score = 64.17 ± 9.01||—|
Students’ Ratings of the Objective Structured Video Examination Stations (N = 87)
|Station||Student Ratings (%)a||Mean Rating||Missing Data|
|Station 1: Video (tardive dyskinesia)||3.6||1.2||9.6||16.9||19.3||49.4||3.95||4|
|Station 2: Written case scenario (suicide attempt management)||1.2||2.4||7.2||19.3||28.9||41.0||3.95||4|
|Station 3: Written case scenario (nursing care before receiving electroconvulsive therapy)||2.4||—||4.8||14.5||27.7||50.6||4.17||4|
|Station 4: Video (auditory and visual hallucinations and delusion of persecution)||1.2||3.6||2.4||25.3||26.5||41||3.95||4|
|Station 5: Laboratory investigation report (severe lithium toxicity)||10.8||2.4||10.8||18.1||20.5||37.3||3.47||4|
|Station 6: Written case scenario (hypertensive crisis)||7.2||4.8||13.3||30.1||19.3||25.3||3.25||4|
|Total score range = 0 to 30||4|
|Mean total score = 22.7470 ± 4.91847||4|