Rationale: Medical students have few validated predictors of USMLE performance. We hypothesized that USMLERx Step 1 Qmax could predict USMLE Step 1 performance.
Methods: In this prospective cohort study, 26
Results: See table below. Participants had higher than average scores for the USMLE Step 1, MCAT, and USMLERx. USMLERx scores strongly correlated with USMLE scores. Kaplan and MCAT scores had moderate correlation. In single linear regression modeling, USMLERx, Kaplan Qbank and MCAT scores were highly significant predictors of USMLE performance with high T-scores. USMLERx scores were able to account almost 80% of the variance in USMLE scores.
|
Test |
Mean score (SD) |
Correlation ( |
T-score |
p-value |
Variance (R2) |
|
USMLERx |
68.9 ±7.7 |
0.89 |
7.95 |
<.001 |
0.79 |
|
Kaplan |
67.6±7.2 |
0.76 |
5.17 |
<.001 |
0.67 |
|
MCAT |
31.7 ±0.5 |
0.73 |
4.80 |
<.001 |
0.59 |
|
USMLE (3-digit) |
234.7 ±19.8 |
– |
– |
– |
- |
Conclusions: USMLERx, Kaplan and MCAT scores appear to be significant predictors of USMLE performance. In this sample, USMLERx was highly correlated with the USMLE Step 1 performance and was able to predict approximately 80% of the variation in USMLE scores.
You can also use the following formula:
USMLERx(%)*2.64 + 47 = USMLE 3-digit score (+/- 20 for the 95% CI)
Subscribe (
March 6th, 2008 at 4:54 pm
Peace,
I’d be far more satisfied and convinced of the data and inferences if the sample size were larger. At least 100. Is there not potentially a selection bias, as well? Perhaps, you can build in a form into the USMLErx.com website wherein users of your system can volunteer their results of the USMLE if they choose to do so. At least, this will increase your sample size. At the same time, there is strong potential for bias wherein only people who score well would feel more compelled to report their results. Nonetheless, it is helpful having this information available to us. Thank You!
God Bless,
shree
May 20th, 2008 at 7:46 pm
Getting scores from people after they already know their score would more than likely produce a huge bias also, seeing as those who are likely to give out their scores are the ones who did very well on their step1 and would actually be comfortable with reporting their scores. Also you would have to sample from people who actually did >80% or so of their tests timed and didn’t look up answers, b/c correlating from people who take them on tutor mode and then look up answers would be worthless also. The only way to get this done would be to give free subscriptions to people who would only be allowed to take timed tests and who were obligated to report their score due to a binding contract from receiving free USMLE Rx. Although this could also produce bias from the fact that people under financial stress would be more willing to take this offer as opposed to those with readily available financial means, but it seems on the surface the best way to assess the real correlation of the Qbank to your step score?
July 4th, 2008 at 1:43 pm
Agreed - many things wrong with their “study”. Giving free exams is good way to do it. But one other way would be to select a random sample (much bigger than 26). This would the obvious selection bias. Another important part is that they should have selected these participants AFTER they’ve written the real USMLE in order to avoid the observation bias.
But most of all i’d love to see an ‘audit’ of all their users and how they score eventually - screw sampling… go straight to the population!