New analysis links doctors’ exam scores to patient survival

How do we know if new doctors have what it takes to prevent patient deaths? After completing residency training, graduating physicians typically take exams upon entry into practice, but surprisingly little is known about the ability of these standard tests to predict the things that count most in a physician’s performance, such as how likely their patients are to survive or avoid a return trip to the hospital.

A new study, published Arrived on May 6 JAMAStudy found that internal medicine patients of newly trained physicians with top scores on the certification exam — a comprehensive test usually administered after a physician completes residency training — had a lower risk of dying or being readmitted to the hospital within seven days of hospitalization included.

The analysis was led by researchers from Harvard Medical School and the American Board of Internal Medicine (ABIM), the body that developed and regularly updates the exam that qualifies a doctor as an internal medicine specialist. Some of the study’s authors, including lead author Bradley Gray, are employed by ABIM.

The findings, the team said, provide assurance that internal medicine exams reflect physicians’ future performance on critical indicators of patient care and outcomes.

“These results confirm that certification exams measure knowledge that directly translates into improved patient outcomes,” said senior author Bruce Landon, professor of healthcare policy at HMS and physician of internal medicine at Beth Israel Deaconess Medical Center.

Landon and colleagues said this is one of the first known attempts to measure the reliability of test scores in predicting patient outcomes.

The researchers also compared patient outcomes to “medical milestones” assessments developed by the Accreditation Council for Graduate Medical Education (ACGME). These assessments are based on periodic evaluation of trainees’ knowledge and skills throughout the residency. The analysis showed no correlation between patient outcomes and physicians’ scores on the milestone assessments.

Test doctors in training

Anyone who has watched a television medical drama knows that earning a medical degree is just the first step on a long journey of medical training and education. After students graduate from medical school, they proceed to residency training in a specialty of their choice before taking their board certification exams and becoming fully qualified independent physicians in their specific medical field.

In recent decades, new ways to test the knowledge of these novice doctors have emerged.

In the field of internal medicine, physician competency is assessed in two ways. The ACGME milestone assessments are administered at various times during residency training. The idea is that periodic milestone tests can provide the trainee with feedback on how well he or she is doing.

At the end of their training, the length of which varies by specialty, almost all internal medicine trainees take the ABIM, also known as medical board, certification exam. This latest comprehensive test gauges whether they are qualified to join independently practicing physicians, who are able to care for patients without supervision.

Scientific approach to improving medical education

Most internal medicine specialists begin their careers as hospital physicians, providing care to hospital patients. For the study, researchers analyzed patient outcomes from nearly 70,000 newly trained hospitalists who treated Medicare beneficiaries during 455,000 hospital admissions that occurred between 2017 and 2019.

The researchers compared the outcomes of patients at the same hospitals who were cared for by doctors with different exam scores. This allowed the researchers to eliminate or at least minimize the effect of differences in patient populations, hospital resources, and other variations that could affect patients’ likelihood of death or readmission, independent of a physician’s performance.

Board exam performance was strongly linked to the patient’s risk of death or hospital readmission. For example, there was an 8% reduction in the odds of dying within seven days of hospitalization among patients of physicians who scored in the top 25% on the exam, compared to the patients of physicians who scored in the bottom 25% on the exam . , which was still a passing grade.

Although physicians’ milestone assessments did not appear to predict patient outcomes, the researchers said using them as a periodic assessment to help determine where a physician trainee could still be a valuable tool for those conducting training programs.

“This type of evidence-based assessment of our own testing instruments provides valuable insights into which types of tests work for what purpose, indicating how they should be deployed in educating our future practitioners and leaders of medicine,” said Landon.