Mosaic Planning and Controls

Understanding your PMI-SP test scores

x

What Score is needed

There is no implication in so far as the course is concerned – you have completed the work and we issue the completion certificate.

  • As far as the exam goes – below 75% you are not ready and will probably fail – significant revision is needed.
  • 75% to 80% you are on track for a pass with some focused revision on the weak areas.
  • Above 80% you are probably OK for the exam – just need to keep the information ‘fresh’ until you get into the exam room.

Scores achieved during the course work are important, but the  100 question 'benchmark test' which should be taken as an exam (closed book, 2 hours max) is really the key. 

 

Understanding your PMI-SP Score

PMI-SP (as well as PMP and CAPM) scores are all based on how well you answer multi-choice questions in your exam.

Unfortunately, the information provided to candidates at the end of their PMI exam is minimal. You are told if you have passed or failed based on the number of questions correctly answered (but the passing score is a secret) and you are told if your score is below average, average or above average against the five ‘domains’. But you are not told what average represents, how it relates to the passing score, or how wide the average band is. In reality, the only option is to be happy if you are in the 70% to 80% who pass and talk to your training provider if you have failed. You cannot get anything additional out of PMI.

During your training, the situation is (or should be) very different!

Almost every training organisation offers its PMI exam candidates sets of questions to help consolidate their learning – we build our courses around lots of regular tests. If the tests are well designed they use the standard 4-option multi-choice question format used by PMI and provide feedback to the trainee on the number of questions correctly answered and detailed information about the answers to all of the questions. We do, and so does everyone else I talk to.

However, what many people don’t fully appreciate is the effect of the four options in the PMI multi-choice format and why a score of 55% is definitely in the ‘fail’ category! The cause is the fact that one of the options will always be correct, therefore a set of random guesses will score 25% on average, and the more questions you answer the more the 25% will come into play. The challenge is working out what proportion of the 55% can be attributed to ‘knowing the right answer’ and what proportion is likely to be a consequence of the ‘lucky 25%’.

As a starting point, any score you achieve will have a certain number of questions answered correctly because you knew the correct answer and a certain number of questions that you answered incorrectly but fluked the right answer for the wrong reason (the ‘lucky 25%’ effect).

One way to assess how many of each are included in your score is to assume a known number of answers are correct because you ‘know the right answer’ and then add 25% of the rest as likely to be correct based on the ‘lucky 25%’ factor. Therefore in a set of 100 questions, if you ‘know the right answers’ to 40 questions and add 25% of the remainder (60 / 4) = 15 you get a score of 55 (or 55%). So whilst superficially a score 55% may seem to be ‘not too bad’ knowing this represents a real score of around 40% gives a very different picture. And the reason increasing your score is quite hard work is because as your real score increases, the remaining number of questions subject to the ‘25% luck’ factor decreases. The situation is further complicated by the fact on a 'lucky day' the number of correct guesses will be higher than 25%, on a 'bad day' less than 25%..... the 'luck factor' its self is subject to a range of possibilities.

This chart is one way of assessing what you really know from any score achieved in a 4-option multi-choice question exam. If you don’t know the answer to any question and guess the lot, you should still score around 25%. Conversely, if you know the correct answer to every question, there are none left to be affected by the ‘25% luck’ factor.

PMI%.jpg

In the chart, your total score is represented by the Pale Blue line and the questions you can assume you ‘know the correct answer to’ is represented by the Dark Blue line.

Between these two lines, there are a steadily decreasing number of questions that you did not know the right answer to but got a correct answer to based on the‘25% luck’ factor. This 25% factor is represented by the Green Line (the two dotted green lines show the probable error range).

 

To use the chart

See what your score is in a test and convert it to a percentage. This is your ‘Total Score’ on the vertical axis. Project this across to the Light Blue line and then look down the chart to see your real score on the horizontal axis. The Dark Blue and Green lines will give you an approximate split between the questions you 'knew the right answer to' and the 'lucky guesses'.

For example, if you scored 62.5% in total, you probably know the answer to around 50% of the questions and the random factor added another 12.5% (the probable range being between 15% and 10% attributed to 'luck' the rest to knowledge). 

 

Limitations & Observations

This chart explains why PMI has set a passing score above 60% in the past (when we knew this information) and seem to have maintained this policy into the present. A score of around 65% would assure PMI the candidate probably achieved a score of 50% or better from knowledge. But there are a number of issues with the chart.

  • The first major problem is the chart does not have variability built in. On average there will be a ‘25% luck’ factor affecting the questions you don’t know the right answer to, but some people will be luckier than others. 
  • The next major problem is very few questions will be answered randomly. Most trainers will tell their candidates to make sure you answer every question, but the majority of questions will have a considered answer, some considered correctly and other considered incorrectly. This factor is impossible to graph. 
  • Statistically, there is no way of knowing which specific questions were answered correctly for the ‘right reason’ and which were in the 'lucky 25%'. In your coursework, you should be given enough information to assess this for yourself - this is why it is important to review all of the answers, not just the ones you got wrong.
  • The last consideration is the potential to remove some variability by correctly deleting ‘obviously wrong’ answers. If you can remove one option the probability of your guess being correct rises to 33%, if you can knock out two options it is up to 50%. This shifts the odds into your favour but reduces the ‘known to be correct for the right reason’ value. Again impossible to graph.

 

Summary

Next time you take a practice test use this chart to assess your real level of knowledge but make sure you check all of the answers you got right so you can identify the questions you got right for the wrong reason, the ‘25% luck’ factor.

One of the reasons we set our trainees a benchmark score of 75% in our coursework is to help overcome the factors discussed above. By the time you are scoring around 75% the effect of the 25% random variability has reduced well below 10% and we know you know enough to score around 65% in the exam without the luck factor. This means most people will pass most of the time; if you score below 75% you are starting to rely on the 25% luck factor and there is always a chance you may not have an average amount of luck on exam day.

To help you practice your PMI-SP answering skills we Tweet a new question every day, for today’s see: http://www.mosaicprojects.com.au/Training-PMP-Q-Today.html

Test Scores

There is no implication in so far as the course is concerned – once you have completed the work we issue the completion certificate.

As far as the exam goes:

Below 75% you are not ready and will probably fail – significant revision is needed.

75% to 80% you may be on track for a pass with some focused revision on the weak areas.

Above 80% you are probably OK for the exam – above 85% you are there! You just need to keep the information ‘fresh’ until you get into the exam room.

Within this the key topics for the exam are scheduling, cost (EV) and risk – you need to be scoring well (80%) in all of these.

The course work is important, but the Module 12 100 question test which should be taken as an exam (closed book, 2 hours max) is really the key.