ORIGINAL RESEARCH ARTICLE

Agreement between vestibular elective physical therapy students and experienced physical therapists in identifying nystagmus during positional testing and diagnosing benign paroxysmal positional vertigo (BPPV)

Laura P. Kennedy-Rynne^* and Anne K. Galgon

Department of Physical Therapy, Saint Joseph’s University, Philadelphia, PA, USA

Abstract

Purpose: The aim of this study is to measure the effectiveness of a vestibular elective in preparing students to perform nystagmus identification and diagnosing benign paroxysmal positional vertigo (BPPV) through an investigation of agreement between student performances and experienced physical therapists.

Methods: Seven students watched eye movement videos during positioning testing of 14 patients. Students recorded the nystagmus characteristics for each position and made diagnoses. The students completed additional surveys. Pairwise percent agreement and kappa statistics (κ) were used to compare agreement between each students’ conclusions to those of an experienced physical therapist obtained from a previous study. Krippendorff’s alpha (α) was used to analyze the overall reliability of student ability for these skills.

Results: Interrater agreement between students and the expert was fair to almost perfect (κ = 0.22 to 1) in identifying nystagmus, and slight to almost perfect (κ = −0.02 to 1) in diagnosing BPPV. There was moderate agreement between students to identify nystagmus (α = 0.58–0.65, average agreement = 80–83%) during positional testing, and fair to moderate agreement to diagnose BPPV (α = 0.38 to 0.53, average agreement = 64–74%). Agreement was higher in cases of typical BPPV presentations. Students reported the task as difficult, had the necessary knowledge, and wanted more practice. Correlations (r = 0.76 to 0.82) were seen between clinical experience with BPPV and lower perceived difficulty.

Conclusion: Students are capable of identifying nystagmus and diagnosing typical BPPV; however, students may require more practice, clinical experience, and mentorship to improve reliability. The results may help determine educational needs to prepare clinicians to manage individuals with BPPV.

Keywords: physical therapy education; benign paroxysmal positional vertigo; reliability

 

 

Many Doctor of Physical Therapy (DPT) programs have added elective courses to raise standards and quality of care given by newly graduated clinicians. In a survey of DPT programs, about 22% indicated having vestibular rehabilitation electives. Content surrounding benign paroxysmal positional vertigo (BPPV), including diagnosis and treatment, is typically included in these electives.¹ However, it is unknown if students who participate in these electives are prepared to independently manage individuals with BPPV, or if their observation and diagnostic skills are equivalent to practicing physical therapists.

BPPV is a mechanical disorder where otoconia are dislodged from the utricle macula and become displaced into the endolymph of a semicircular canal (SCC).² Stereotypic paroxysmal nystagmus and symptoms present in distinctive patterns. Correct observation and identification of nystagmus patterns during positioning tests are key to determining diagnosis of the involved SCC and the BPPV type, where the otoconia sit in relation to the cupula of the SCC.² The diagnosis drives selection of canal-specific repositioning maneuvers (CRMs) used to return the otoconia to the utricle.^3–5 Previous studies have shown substantial to almost perfect agreement in identifying nystagmus^6,7 and diagnosing BPPV⁷ among experienced physical therapists.

There has been variability in the depth and breadth of vestibular content taught in physical therapy programs.¹ However, surveyed DPT programs rated management of BPPV as essential for entry-level education. Posterior canal canalithiasis, posterior canal cupulothiasis, and horizontal canal canalithiasis were rated as essential by 98.7, 92.3, and 90.2% of programs, respectively. More programs reported assessing students on both cognitive knowledge and psychomotor skills on posterior and horizontal canal BPPV than other vestibular disorders.¹ The assumption that graduating physical therapists are competent in these skills may lead to them treating individuals with BPPV without additional training, supervision, or mentorship.

A few studies have reported the effectiveness of electives in healthcare professional education. Elective courses have been described in a variety of programs, including pharmacology,^8–10 medicine,^11,12 interprofessional healthcare,¹³ and physical therapy.^14–16 The purposes of these electives have been to expose students to either specialty knowledge and practice, such as pain management,^12,14,15,16 musculoskeletal anatomy,¹¹ diabetes management,¹⁰ and complementary and alternative medicine,¹³ or expand cultural awareness and communication skills in underserved populations.^9,15 As far as we are aware, there has been no long-term studies that examine the effect of participation in elective courses on entry-level practitioner knowledge and skill.

The educators suggested that the electives will better prepare graduates to practice their healthcare profession.¹⁶ However, most reported effectiveness by demonstrating improved student knowledge from beginning to end of course,^8,9,12–15 increased positive student satisfaction,^8,12,13 perspectives on the learning experience,¹³ or improved student self-efficacy or confidence.^9,10 Only one study assessed student knowledge using an established practice standardized examination.¹¹ At the end of a musculoskeletal anatomy elective, medical students’ average examination scores reached a minimal core competency required for orthopedic medical residents. An alternative method to measure practice capability is by assessing students’ skills directly against experienced clinicians’ skills.

The purpose of this study was to examine the effectiveness of a vestibular rehabilitation elective in preparing student clinicians to evaluate and diagnose typical BPPV by comparing student performances directly to that of experienced vestibular physical therapists.⁷ The secondary purpose was to gain insight in the students’ perceptions of performing these skills. Student performance and perceptions were measured to reflect on the educational experiences in the elective.

Methods

Design

This was a reliability study to determine agreement between students and an expert in identifying nystagmus and diagnosing BPPV. Secondarily, survey information was gathered to interpret the student experience for better understanding of educational needs of students and clinicians. This study was approved by the Temple University’s Internal Review Board, protocol 22395.

Participants

The participants were students within the Temple University DPT program. Participants were eligible if enrolled in the vestibular elective course. Sixteen students met this inclusion criteria and were invited to participate. One student researcher of the study was excluded. No monetary nor academic incentives were provided.

Prior to participating, students attained foundational vestibular system knowledge in neuroscience and movement science courses. Vestibular rehabilitation content was taught in a neuromuscular physical therapy course and a vestibular elective course. Prior to the elective, the students completed a total 18 weeks of full-time clinical education. See Table 1 for the elective content details.

Table 1. Vestibular elective description and educational strategies for benign paroxysmal positional vertigo (BPPV) content

Pre-elective vestibular rehabilitation content
Nine hours of vestibular physical therapy lecture and laboratory activities, including 4 h of BPPV content and skills in a neuromuscular physical therapy course in the spring of the program’s second year. Student assessments: final exam questions and some vestibular case studies on the practical exam.
Vestibular rehabilitation elective content
Twenty hours (2 credits) offered in a 10-week didactic semester prior to a terminal clinical education experience. The course contained five 2-week modules covering the management of vestibular hypofunction, Benign Paroxysmal Positional Vertigo (BPPV), central vestibular disorders, persistent postural perceptual disorder, and concussion. Five hours of the learning experiences were related to BPPV. This included 50 min of direct video observations of eye movements, instruction on identifying nystagmus, practice using a nystagmus documentation, and making diagnoses using a BPPV diagnosis algorithm.7 Two hours of practicing positional testing and interventions. Student assessments: modular quizzes, competency testing of exam, intervention procedures, and case study presentations.
Instructor Driven Instructional strategies used in the BPPV module
Presentation 1. How to orient to eyes and eye movements 2. Nystagmus terminology 3. Introduction to documentation form and diagnostic algorithm	Class video observations practice and discussion 1. Large screen video observation practice 2. Students asked to state observations and diagnoses 3. Instructor feedback and discussion 4. Video examples include mostly common and a few uncommon presentations		Lab practice of positioning testing 1. Instructor demonstration of tests with verbal description of positive test referencing diagnostic algorithm. 2. Student practice of positional tests while referencing diagnostic algorithm 3. Instructor monitoring and feedback
Strategies to improve student performance for future
Goal: Increase student-driven practice, problem solving, self-reflection, and peer mentoring 1. Small screen video observation practice 2. Individual practice of observation and diagnoses 3. Individual assessment and self-reflection of performance. 4. Student pairs or small group review of observations and rationale for diagnosis 5. Instructor monitoring of individual and small group performance to identify areas for reinforcement.		Rationale: Simulate clinical experience and mentoring 1. Simulate clinical observation of eye movements 2. Simulate individual clinician practice 3. Promote self-reflection of skill and diagnosing20 4. Simulate peer problem solving for clinical reasoning development23 5. Simulate clinical instructor or expert mentorship

Procedures

All participants attended a 90-min viewing session. The students viewed one practice video as a group and were given a reference diagnostic algorithm.⁷ Each student then independently viewed 14 prerecorded videos on a laptop with headphones. This simulated the clinical experience of watching eye movements on a monitor. Students viewed of each video one time and documented the presence of nystagmus with direction, duration and intensity within each positional test on a documentation sheet.⁷ Once a video was concluded, the students recorded a diagnosis including the involved canal, side and type (canalithiasis or cupulolithiasis). The students then completed two surveys.

Videos of positional testing

A full description of the procedures for obtaining, processing, and storing videos was reported in a previous study.⁷ Briefly, videos were obtained from patients referred to physical therapy with complaints of positional dizziness. Eye movements and audio information were recorded continuously with a video goggles system during standard positioning tests, including sit to supine, right and left supine roll tests, and right and left Dix-Hallpike tests. Audio information included the physical therapist’s instructions and verbal statements when the patient reached each position in the sequence, and any patient verbal responses. The 14 selected videos were previously labeled for diagnosis with 100% agreement between three experienced clinicians and confirmed by the primary investigator. The final diagnoses included posterior canal canalithiasis (n = 6), posterior canal cupulothiasis (n = 2), lateral canal canalithiasis/geotropic presentation (n = 3), lateral canal cupulolithiasis/apogeotropic presentation (n = 1), and no BPPV (n = 2). No anterior canal BPPV diagnoses were included, as no patient videos from the previous study were confirmed with anterior canal involvement. Each video was 5–10 min in duration.

Data from three experienced therapists who previously observed and diagnosed patients in these videos were used as the standard for evaluating the students’ performance.⁷ Since the three therapists fully agreed on video diagnoses, data from one were selected to compare against student performance.

Student surveys

A demographic survey included questions on age, gender, and previous number of BPPV patients seen during clinical education.

A perception survey contained 15 questions. The first eight items were statements asking for agreement or disagreement on a 4-level Likert scale to rate confidence, preparation and perceived difficulty of the task. Two questions prompted students to enumerate the videos they correctly identified nystagmus and diagnosis. Two questions asked if observational skills were easy or difficult. Finally, three open-ended questions allowed for comments on other factors that may have impacted their performance.

Analysis

Median and ranges of ages, frequency of gender, and number of patients seen with BPPV were calculated. Pairwise kappa statistics and percent (%) agreements were calculated to determine agreement between students and between students and the experienced clinician in identifying the presence of nystagmus in each positional test (sit to supine, right and left roll, and right and left Dix-Hallpike). The kappa statistics were interpreted as poor (κ = 0), slight (κ = 0.01–.020), fair (κ = 0.2–0.40), moderate (κ = 0.41–0.60), substantial (κ = 0.61–0.80), and almost perfect (κ = 0.81–1.00) agreement, respectively.¹⁷ Krippendorff’s alpha (α) was used to determine global agreement between students.¹⁸ Pairwise % agreement was used to evaluate the agreement in identifying the direction of nystagmus in positions when all clinicians agreed that nystagmus was present. Average (range) pairwise kappa and % agreement were also used to examine each component of the diagnosis: involved side, canal, and type of BPPV (canalithiasis and cupulolithiasis), separately. A post hoc analysis of nystagmus observations of students who made incorrect diagnoses was performed to describe the number and type of errors students made.

Frequency of agreement and disagreement ratings from the perception questions were analyzed. Median and range of students’ perception of the number of correct nystagmus identified and diagnoses were also collected. Spearman correlation coefficients (r) and 95% confidence intervals (CI) were calculated to determine if there were relationships between item statement ratings and the number of patients previously seen. Frequency of assessment skills that were rated as easy or difficult was calculated. An informal review of responses to open-ended questions was performed independently by two researchers and then reviewed together for consensus categorization of similar responses.

Results

Participants

Seven students were participated, including four females and three males with a median age of 25 and a range of 24–33 years. Five participants reported seeing patients with BPPV during their prior clinical experiences, two 1–3 patients, one 4–6 patients, and two greater than eight patients.

Agreement on nystagmus

Table 2 presents the results of agreement for the presence and direction of nystagmus between students and between each student and the experienced clinician for each positioning test. The between students average % agreement on the presence of nystagmus was close to 80% for each positional test, and a global agreement was moderate to substantial (α range = 0.58 – 0.65). Average % agreement between each student and the clinician for the presence of nystagmus ranged from 78.6 to 88.8% on any given positional test, and pairwise Kappa ranged from fair to almost perfect agreement. However, there was lower and more variable agreement for the direction of nystagmus between students and the clinician. Pairwise agreement for the direction of nystagmus ranged from 40 to 100% in the supine roll tests and was between 38–75% and 48–86% in the right and left Dix-Hallpike tests, respectively.

Table 2. Average (Range) of agreement to identify nystagmus during each positional test

Positioning Test	Between novices agreement		Between novice and expert agreement
	Presence of nystagmus		Presence of nystagmus		Direction of nystagmus
	Pairwise % agreement	Krippendorff’s alpha	Pairwise % agreement	Pairwise Kappa	Pairwise % agreement
Right Roll test	82.99% (57–100%)	0.64	88.77% (71–100%)	0.76 (0.36–1)	74% (40–100%) n = 5
Left Roll test	81.63% (57–100%)	0.64	88.77% (78.6–100%)	0.77 (0.57–1)	71% (43–100%) n = 7
Right Dix-Hallpike	79.59% (50–100%)	0.58	78.60% (64–86%)	0.54 (0.36–0.7)	61% (38–75%) n = 8
Left Dix-Hallpike	82.31% (71–93%)	0.65	87.8% (78.6–100%)	0.76 (0.59–1)	73% (43–86%) n = 7
Note: n, number of videos that agreement was calculated for the direction of nystagmus. Videos were not included if experts agreed that no nystagmus was present for the positional test.

Agreement on diagnosis

Table 3 presents the results of agreement on each component of the diagnosis between students and between each student and the clinician. Average pairwise % agreement between students was 73% for side of involvement, 64% for canal of involvement, and 74% for type with global agreement being fair to moderate (α range = 0.33–0.53). On average, the students agreed with the clinician 74.5% of the time for side and canal involvement, and 71.4% for the type of BPPV. The agreement ranged from fair to substantial for side (κ = 0.36–0.68), canal (κ = 0.21–0.75), and slight to moderate for type (κ = 0.15–0.56).

Table 3. Average (range) of agreement for each component of the BPPV diagnosis

Diagnosis component	Between novices agreement		Between novice and expert agreement
	Pairwise % agreement	Krippendorff’s alpha	Pairwise % agreement	Pairwise Kappa
Side of involvement	72.79% (50–93%)	0.53	74.50% (64–78.5%)	0.57 (0.36–0.68)
Canal of involvement	63.61% (36–78.6%)	0.33	74.50% (50–86%)	0.55 (0.21–0.74)
Type CA or CU	73.81% (50–100%)	0.38	71.40% (64–78.6%)	0.35 (0.15–0.56)

On average, the students agreed with the clinician 51.47% of the time on all three components of the diagnosis. The two students who reported seeing greater than eight BPPV patients agreed with the expert 64.29% of time, the one student who saw 4–6 patients agreed 42.86% of the time, the two who only saw 1 to 3 patients agreed 64.25 and 57.14% of the time, and the two students who had no experience with BPPV agreed 42.86 and 28.47% of the time with the expert.

Diagnostic errors

Five of the seven students fully agreed on all three diagnostic components in the six patient videos displaying right and left posterior canal canalithiasis. Three or less students agreed with the clinician’s diagnoses during videos, which included right lateral canal cupulolithiasis and canalithiasis, left posterior canal cupulolithiasis, and no BPPV present. When the students did not agree on all three components of the diagnosis, an analysis of the record sheets revealed that the diagnostic algorithm was followed appropriately 79.29% of the time based on what nystagmus they perceived to have observed.

High frequency errors, occurring more than 10 times, were misdirection of identified nystagmus, misobservation of the presence of nystagmus, and incorrect identification of canalithiasis versus cupulolithiasis nystagmus. Less frequent mistakes included the following: the misidentification geotropic nystagmus with canalithiasis and apogeotropic nystagmus with cupulolithiasis, and allowing an observed nystagmus in a Dix-Hallpike position to hold more weight in clinical decision-making compared to nystagmus in the roll tests.

Perception survey results

Table 4 presents the student responses to survey statements and perceptions of accuracy of identifying nystagmus and diagnosis BPPV. More students (5/7) agreed they were confident in their assessments, they were well prepared, time to assess each video was appropriate, and the task of identifying nystagmus and diagnosing was difficult. Only three students agreed they would feel comfortable evaluating and treating individuals with BPPV as an entry-level clinician. Students who rated a high number of correct nystagmus tended to be more confident (r = 0.72, P = 0.06) and disagreed that identifying nystagmus and diagnosing was difficult (r = −0.77, P < 0.05). There was a negative relationship between feeling prepared and agreeing that the identifying nystagmus and diagnosing was difficult (r = −0.84, P < 0.05).

Table 4. Survey questions and responses and association with experience with BPPV patients

Survey statement	Agreed or strongly agreed	Disagreed or strongly disagreed	Correlation to # of previous seen patients with BPPV r (95% CI)
I am confident that my assessments of the eye movement videos agree with an expert clinician’s assessments.	5	2	0.49 (−0.44, 0.91), P = 0.27
I was well prepared to accurately assess the eye movement videos	5	2	0.49 (−0.44, 0.91), P = 0.27
The time to assess each individual video was appropriate	5	2	0.39 (−0.54, 0.89), P = 0.39
It was difficult to identify nystagmus and give an appropriate diagnosis.	5	2	−0.63 (−0.94, 0.23), P = 0.13
Additional practice would have been helpful in improving the accuracy of my assessments	4	3	−0.71 (−0.96, 0.13), P = 0.08
The instructions for how to identify nystagmus and make a diagnosis were clear.	4	3	−0.37 (−0.89, 0.55), P = 0.41
My level of knowledge was limited when examining eye movements for BPPV	3	4	−0.50, (−0.91, 0.43), P = 0.26
I would feel comfortable examining and treating patients with BPPV as an Entry-Level clinician?	3	4	0.75 (−0.04, 0.96), P = 0.05
Student Perception of Accuracy	Median	Range
Number of videos I got nystagmus correct	9	5–12	0.40 (−0.53, 0.89), P = 0.37
Number of videos I got diagnosis correct	8	5–12	0.65 (−0.23, 0.95), P = 0.10

Table 4 also presents the correlations between the number of patients with BPPV seen in clinical experiences and questionnaire responses. Students who had previously seen more patients with BPPV tended to disagree that they needed more practice, and identifying nystagmus and diagnosis was difficult. Seeing more patients may be related to feeling comfortable managing individuals with BPPV (r = 0.75, P = 0.05).

Table 5 presents the ratings on ease or difficulty of assessment skills. All students rated determining up beating versus down beating nystagmus as easy. Six students marked orienting themselves to the eye(s) in the orbit, determining which positional test was positive, and determining the side of involvement as easy. Five students marked identifying the direction of torsional nystagmus, and four marked determining duration of the nystagmus as difficult.

Table 5. Frequency of student rating of observational assessment skills as easy or difficulty

Observational assessment skill	Easy	Difficulty
Orienting yourself to the eye(s) in the orbit and/or in reference to the video image	6	1
Determining which positional test was positive	6	1
Identifying up versus down beating nystagmus	7	0
Identifying right versus left horizontal beating nystagmus	5	2
Identifying when horizontal nystagmus was geotropic versus apogeotropic*	6	1
Identifying the direction of torsional nystagmus	2	5
Determining the duration of the nystagmus	3	4
Determining the intensity of the nystagmus	4	3
Determining the side of involvement	6	1
Determining the involved canal	4	3
*The student needed to identify nystagmus in the context of right or left head roll position.

There were three consistent responses to open-ended questions identified. Five students stated that more practice would have prepared them better. Four students felt that the patients’ ability to keep their eyes open limited identification of nystagmus. Four students also made comments that video quality limited their ability to identify nystagmus.

Discussion

Correct nystagmus identification in positional tests is required for accurate BPPV diagnosis and intervention. Incorrect diagnosis can lead to inefficient selection of CRMs and delayed symptom resolution.² On average, students agreed with the clinician about 50% of the time for full diagnosis. Many new graduates may reach competency in diagnosing posterior canal BPPV, as students were better at identifying upbeat torsional nystagmus toward the left or right in the Dix-Hallpike positioning tests and diagnosing posterior canal presentations. These are important skills because posterior canal involvement occurs in 85–95% of BPPV cases.² Fewer students agreed with the clinician’s diagnosis when presentations were less common. Some students also did not rule out BPPV when clinicians had determined that the eye movements did not suggest BPPV.

We specifically evaluated if misdiagnosis resulted from misidentification of the nystagmus pattern and a correct use of the algorithm, or from correct identification of nystagmus and incorrect use of algorithm. Students made both of these errors. The students correctly used the algorithm nearly 80% of the time. Our analysis suggests students may have had difficulties prioritizing determinant nystagmus over extraneous eye movements, and/or recognizing nystagmus generated by one canal in a positioning test for a different canal. Diagnostic divergence in agreement occurred most often for side and canal involvement even when students correctly identified nystagmus patterns. Other errors in algorithm use occurred when multiple positions displayed nystagmus, when intensity or duration of nystagmus was key to identifying type, and when nystagmus direction was misidentified.

Survey responses were consistent with both perceived and actual performances of competency. Most students felt they were prepared and had adequate knowledge but needed more practice, and nystagmus identification was difficult, leading to challenges differentiating diagnoses. Students with more clinical BPPV experience felt they would be comfortable managing BPPV patients, and their performance agreed with the clinicians’ diagnoses more often. Student self-efficacy has been shown to strongly relate to prior clinical experience.¹⁹ Academics have commented that skills proficiency of new graduates is dependent on vestibular exposure within clinical experiences.¹

Clinical decision-making skill derives from a culmination of educational themes, including prior professional experience, sources of information, and reflection.¹⁹ The BPPV education students received prior to the elective would be categorized as the standard academic experience. The vestibular elective may be analogous to continuing education with the aim to enhance knowledge, observation skills, and proficiency with use of the diagnostic algorithm, and to gain mentorship from faculty. Physical therapists experienced in managing individuals with BPPV demonstrated these skills.^6,7 Clinical experience and repetition is an essential component to improve pattern recognition, reflection in/on action, forward reasoning, and collaborative and adaptive strategies to develop expert level clinical decisions.^20–22 Our study supports that students may not fully develop these skills and deductive reasoning when observing eye movements and making BPPV diagnosis without clinical practice and mentorship.

Table 1 presents the educational strategies used in the vestibular elective and suggestions for improvement based on reflections on the study results. The instructor-driven activities may not have allowed students to practice skills essential to clinical decision-making development.^20,21 Suggestions included increasing the student practice of observation and diagnosis skills with small screen viewing, using collaborative problem solving/peer mentoring,²³ and self-reflection and reinforcement which replicate clinical learning experiences.

The small sample size limits generalization of the results. The sample size did not allow us to make a strong association between skills and previous clinical experience. Video sampling bias limited the BPPV presentations observed; therefore, we cannot be sure if students could identify other types of positional nystagmus.⁷ We did not assess students’ performances who did not participate in a vestibular elective. Therefore, the results may be hard to generalize to other students or clinicians who participate in vestibular continuing education. Although students in the elective demonstrated competency in performing testing procedures, we cannot be sure of competency in performing testing and observing eye movements concurrently in a clinical environment. Further research is warranted for long-term impacts of elective course participation on clinical practice.¹⁶ Accurate diagnosis of BPPV should also incorporate patient history and complaints,²⁴ which was not provided. Thus, we cannot be sure that students would integrate patient history into clinical decision-making or screen individuals to determine need for positioning tests. We did not evaluate the quality of any clinical instructor mentorship the students may have had prior to the study, which may have impacted their confidence in performing these skills. Multicomponent decision-making assessments may be a method of understanding student learning and practice readiness for managing individuals with BPPV.

Conclusion

Comparing student to clinician performance of identifying nystagmus and diagnosing BPPV was a valuable method to evaluate the effectiveness of a physical therapy vestibular elective. Understanding student skill capability is important because patients of varying BPPV complexity can be encountered across practice settings. The vestibular elective students showed moderate to substantial agreement for identifying nystagmus and fair to moderate agreement in diagnosing BPPV. This study identified common diagnostic errors made by students, which could inform educators on improving targeted academic practices. Clinical exposure and mentorship are likely important to the development of BPPV observations and clinical decision-making skills.

Ethics statement

This study was approved by Temple University IRB, protocol 22395. All participants signed informed consent prior to participation in this study.

References

1.	Galgon AK, Roberts HJ, Littmann AE, et al. A survey of entry-level physical therapy education content for vestibular rehabilitation. J Phys Ther Ed (2022) 36(1): 65–75. doi: 10.1097/jte.0000000000000210
2.	Bhattacharyya N, Gubbels SP, Schwartz SR, et al. Clinical practice guideline: benign paroxysmal positional vertigo (update). Otolaryngol Head Neck Surg (2017) 156(3_suppl): S1–47. doi: 10.1177/0194599816689667
3.	Epley JM. The canalith repositioning procedure: for treatment of benign paroxysmal positional vertigo. Otolaryngol Head Neck Surg (1992) 107(3): 399–404. doi: 10.1177/019459989210700310
4.	Lempert T. Horizontal benign positional vertigo. Neurology (1994) 44(11): 2213–4. doi: 10.1212/wnl.44.11.2213-a
5.	Semont A, Freyss G, Vitte E. Curing the BPPV with a liberatory maneuver. Adv Otorhinolaryngol (1988) 42: 290–3. doi: 10.1159/000416126
6.	Blau P, Shoup A. Reliability of a rating scale used to distinguish direction of eye movement using infrared/video ENG recordings during repositioning maneuvers. Int J Audiol (2007) 46(8): 427–32. doi: 10.1080/14992020701355082
7.	Galgon AK, Tate A, Fitzpatrick M, et al. Agreement between physical therapists in diagnosing benign paroxysmal positional vertigo. J Neurol Phys Ther (2021) 45(2): 79–86. doi: 10.1097/npt.0000000000000349
8.	Steinhardt SJ, Kelly WN, Clark JE, et al. An artistic active-learning Approach to teaching a substance use disorder elective course. Am J Pharm Educ (2020) 84(4): 7634. doi: 10.5688/ajpe7634
9.	Jann MW, Penzak S, White A, et al. An elective course in lesbian, gay, bisexual, and transgender health and practice issues. Am J Pharm Educ (2019) 83(8): 6967. doi: 10.5688/ajpe6967
10.	Kerr JL, Stahnke AM, Behnen EM. Assessing empathy and self-efficacy levels of pharmacy students in an elective diabetes management course. Am J Pharm Educ (2015) 79(3): 42. doi: 10.5688/ajpe79342
11.	Lazarus MD, Kauffman Jr, GL, Kothari MJ, et al. Anatomy integration blueprint: a fourth-year musculoskeletal anatomy elective model. Anat Sci Educ (2014) 7(5): 379–88. doi: 10.1002/ase.1439
12.	Puljak L, Sapunar D. Web-based elective courses for medical students: an example in pain. Pain Med (2011) 12(6): 854–63. doi: 10.1111/j.1526-4637.2011.01104.x
13.	Klafke N, Homberg A, Glassen K, et al. Addressing holistic healthcare needs of oncology patients: implementation and evaluation of a complementary and alternative medicine (CAM) course within an elective module designed for healthcare professionals. Complement Ther Med (2016) 29: 190–5. doi: 10.1016/j.ctim.2016.10.011
14.	Bareiss SK, Nare L, McBee K. Evaluation of pain knowledge and attitudes and beliefs from a pre-licensure physical therapy curriculum and a stand-alone pain elective. BMC Med Educ (2019) 19(1): 375. doi: 10.1186/s12909-019-1820-7
15.	Masin HPT, Tischenko AK. Professionalism, attitudes, beliefs and transformation of the learning experience: cross-cultural implications for developing a Spanish elective for non–Spanish-speaking physical therapist students. J Phys Ther Educ (2007) 21(3): 40–6. doi: 10.1097/00001416-200710000-00006
16.	Wassinger CA. Pain knowledge, attitudes and beliefs of doctor of physical therapy students: changes across the curriculum and the role of an elective pain science course. J Man Manip Ther (2021) 29(5): 288–96. doi: 10.1080/10669817.2021.1879509
17.	Sim J, Wright CC. The Kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther (2005) 85(3): 257–68. doi: 10.1093/ptj/85.3.257
18.	Zapf A, Castell S, Morawietz L, et al. Measuring inter-rater reliability for nominal data – which coefficients and confidence intervals are appropriate? BMC Med Res Methodol (2016) 16(1): 93. doi: 10.1186/s12874-016-0200-9
19.	Scott K, Wissinger J, Hand B, et al. Considering domain-specific experiential learning: self-efficacy in pediatric physical therapy education. Pediatr Phys Ther (2021) 33(3): 163–9. doi: 10.1097/PEP.0000000000000797
20.	Wainwright SF, Shapard KF, Harman LB, et al. Factors that influence the clinical decision making of novice and experienced physical therapists. Phys Ther (2011) 91(1): 87–101. doi: 10.2522/ptj.20100161
21.	Wainwright SF, Shepard KF, Harman LB, et al Novice and experienced physical therapist clinicians: a comparison of how reflection is used to inform the clinical decision-making process. Phys Ther (2010) 90(1): 75–88. doi: 10.2522/ptj.20090077
22.	May S, Greasley A, Reeve S, et al. Expert therapists use specific clinical reasoning processes in the assessment and management of patients with shoulder pain: a qualitative study. Aust J Physiother (2008) 54(4): 261–6. doi: 10.1016/s0004-9514(08)70005-9
23.	Resnik L, Jensen GM. Using clinical outcomes to explore the theory of expert practice in physical therapy. Phys Ther (2003) 83(12): 1090–106. doi: 10.1093/ptj/83.12.1090
24.	Lindell E, Finizia C, Johansson M, et al. Asking about dizziness when turning in bed predicts examination findings for benign paroxysmal positional vertigo. J Vestib Res (2018) 28(3–4): 339–47. doi: 10.3233/ves-180637