The General Medical Council (GMC) adopted a new standard-setting approach for PLAB 2 known as the “borderline regression” method. This approach replaces the traditional borderline group method and is designed to provide a fairer, more reliable, and more transparent means of determining the pass mark.
What Was the Old Method?
Previously, the GMC used a borderline group method. In this approach, each station within the PLAB 2 Objective Structured Clinical Examination (OSCE) would have groups of candidates whose performances were deemed “borderline” by the examiners. The average score of these borderline performers helped set the pass mark for that station. Essentially, the standard depended on the subjective categorization of candidates into “borderline” performance levels and then using their mean scores to set the standard.
While this method was widely accepted for years, it had some drawbacks:
- Subjectivity: The identification of “borderline” candidates could vary depending on the examiner.
- Variability: Small differences in how examiners labeled candidates could shift the final pass mark.
- Less Statistical Robustness: The final cutoff depended more on grouping judgments rather than a continuous statistical model.
What is the Borderline Regression Method?
The borderline regression method is a more data-driven, statistically robust approach. Rather than relying on a set of borderline candidates’ average scores, this method uses a statistical analysis (specifically, linear regression) to link examiner judgments of candidate performance with the actual scores awarded.
How Does It Work?
-
Examiner Judgments: In each OSCE station, examiners assess candidates using both a checklist/domain-based scoring system and an overall “global rating” scale. The global rating scale might categorize candidates as “Unsatisfactory", "Borderline", "Satisfactory" or Good” based on clinical judgment and observed performance.
-
Regression Analysis: After the exam, these data points (overall global ratings and actual numeric scores) are fed into a statistical model. The model looks at all candidates’ scores in relation to their global ratings and plots this on a curve.
-
Finding the ‘Borderline’ Score: The model identifies a score that corresponds statistically to the “borderline” global rating. In other words, it uses the regression line to determine the score at which a candidate just edges from a borderline performance into a passing performance. This gives a predicted “borderline” score for each station that doesn’t rely on picking out a separate subgroup of borderline candidates. Instead, it uses all candidates’ data to find a stable, evidence-based score.
-
Summing Across Stations: Each station’s pass mark is established in this manner. The pass marks for all stations are then summed up to derive the overall pass mark for the exam. This ensures that each station’s standard is set in a fair and consistent way using the regression-derived borderline point.
The Key Differences and Advantages:
- Reduced Subjectivity: Instead of individually categorizing candidates as borderline, the regression uses continuous data (scores and ratings) from all candidates, reducing the influence of any single examiner’s subjective classification.
- More Stable and Consistent Standards: Statistical modeling tends to smooth out minor fluctuations that occur due to examiner variation. Over time, this approach should yield more stable pass marks.
- Transparent and Defensible: The regression line and resulting pass marks are based on a clear statistical relationship. It provides a more mathematically defensible standard than relying solely on examiner-chosen borderline groups.
What Does This Mean for Candidates?
For candidates, the borderline regression method means that the pass/fail decisions are guided by a robust, data-driven technique. Performance is measured against a standard that is less likely to shift due to subjective factors. Candidates can be more confident that the standard set each diet of the exam is fair and reflective of true clinical competence.
In Short:
- The old method: Identify borderline candidates and use their average scores to set the pass mark.
- The new method: Use statistical regression to link examiner global ratings with actual scores and identify a precise borderline score. Then, combine these station-level standards to form the overall pass mark.
For more information visit GMC - Understanding Your Results page. We are glad to inform you that we have already adopted this marking technique in our mocks.