Important Information About Your CFA® Exam Results - Calculating the MPS

Well monkeys, it sure was fun not worrying about the CFA® exam for a couple weeks. But, I'm sure that like me, you all got that special email last night telling everyone to restart their freak out until July 23rd. Thanks, CFA® Institute, that's just what my summer needs.

I remember last year, after finishing level 1, I freaked out fairly continuously until the date the results came out. Primarily, I was concerned with the question, "so, how the hell do they actually grade these things?" I wouldn't suggest this particular course. Luckily, I did learn quite a bit about how the grading process works and how the MPS is decided upon.

Hopefully everyone finds this illuminating or if you're inclined to freak out over the eventual MPS, that this eases your anxiety.

Let me start off with an observation. The first things the email states is to make sure that the email is going to the correct email address. Okay, but if it went to the wrong email address, how would I have gotten the email? Just sayin'. But I digress.

As I'm sure many of you have figured out, the CFA® does have a document online that describes how the grading process works. But, I'm also sure that many of you found it wanting, and this is for good reason, the CFA® is generally pretty hush-hush about their grading process, and the provided documentation can be a little cryptic if you're not familiar with what they're saying. The key method that is utilized is mentioned on page 10, right at the beginning of the "Standard Setting" subsection, which reads:

[The] methodology for arriving at the minimum passing score (MPS)—the modified Angoff Standard Setting Method

This is only mentioned once, and yet is the most important fact in the entire section. So, the real question is, what the hell is the Angoff Method? Knowing how this process works is key to understanding how the CFA® comes up with the MPS and why it changes from year to year. Here's the basic process for standard setting via the Angoff Method:

  1. Assemble Subject Matter Experts: The Angoff Method requires the input of subject matter experts in order to set a given standard. Some key points from the CFA® demonstrate the first step of the process:

     

    Another quality control during this period is the investigation of all comments and complaints related to the examinations.

    Any of you remember filling out a survey or being asked about your opinion on a given question? No, of course not, we're just candidates, which means that the comments and complaints are being provided by their subject matter experts during the first step of the Angoff Method.

  2. Round One of Grading: The Angoff Method's premise is that with a sufficient number subject matter experts, a uniform decision on what makes a minimally competent candidate can be discerned with enough iterations of "grading rounds." The rounds change slightly from one to the next, and based on what was mentioned previously, it seems likely that the first round of grading is to figure out which questions fall into one of these categories:

    Special focus is given to reviews under any of the following circumstances: multiple similar complaints, exam results suggesting more than one correct answer, or exam results that otherwise suggest that a question was confusing or unfair.

    If I had to guess, I'd say that this step is what makes the CFA® method a "modified Angoff Method" as opposed to simply an "Angoff Method". Don't quote me, this is just a guess. But, good news everyone, if you thought there were some questions on the exam that were totally unfair, don't worry, the CFA® states explicitly:

    If, after investigation, a question is determined to be confusing or unfair, results are adjusted to credit all answers. If more than one answer was correct, then all correct answers are credited.
  3. Round Two of Grading: After round one, we're now working within the traditional methodology of the Angoff Method. This is noted by the CFA®, which states:

    Each participant reviews the entire examination, question by question, and makes an independent judgment on the expected performance of a just-competent candidate on each question on the examination.

    What this means in practice is that each subject matter expert takes each question and makes an independent judgment as to the probability of a minimally competent candidate getting a question correct. The data is then collected and compiled and averages are calculated for each question, resulting in a probability for each and every question found on the exam. For example, if every question is assigned a probability of 70%, meaning that a minimally competent candidate has a 70% chance of getting the question correct, then the passing score recommendation for the round is 70%. Obviously, it's a little more varied than that, but the process is the same.

  4. Round Three, Four, Five,... , Whatever of Grading: Herein lies the bulk of the process found within the Angoff Method. After each round of grading, new information is provided to the subject matter experts, and the process begins anew utilizing the new information. The most common type of information that is provided to subject matter experts is what is known as "impact data". The CFA® explicitly states that they do, in fact, provide impact data to those doing the grading:

    Participants review the entire examination a second time after reviewing general impact data and overall actual candidate performance on the exam.

    But, of course, the CFA® is a little coy with the details about what "general impact data" actually represents. Typically, impact data is the percentage of candidates who would pass based on the previous round's median or average passing score recommendation. So, if in the previous round, the score recommendation ended up being, let's say, 65%, the impact data associated with this recommendation would be the percentage of candidates who would have passed had the final recommendation been 65%. Or, put more simply, in the above quote, replace the word "and" with "which is" (I have no idea why they wrote it the way they did). This process may be repeated as many times as necessary to arrive at a consensus recommendation. Once a consensus is reached, the recommendation is sent up to the CFA® Institute Board of Governors who then sets the MPS based on this recommendation. If I had to guess, I doubt the board significantly changes the recommendation, if at all.

From a high level, this is how the CFA® exams are graded. So, if someone tries to tell you that the MPS is 70% of the top 1% of scores, they're wrong. Maybe that's how they used to do it, but it isn't done that way anymore. Also, this should clear up why it takes so damn long to grade the exams!

Now that everyone is armed with the knowledge of how their exam is being graded, I hope we can all return to not caring about the CFA® results until July 23rd!

 

CFA® Institute does not endorse, promote or warrant the accuracy or quality of Wall Street Oasis. CFA® and Chartered Financial Analyst® are registered trademarks owned by CFA® Institute.

 

Great post. Lots of good info in there-- thanks for putting in the time and quashing a few rumors. I took Level 2 in June. It's tough to know whether to look forward to July 23 or hope it never comes. I think most, if not all, of the 8 full PTs I took were harder than the real thing, but I historically did not gauge how well I did on any.

Thanks for the info.

 

Enim eveniet architecto et ut autem vero. Ipsum dicta iste nesciunt qui. Repellat odio esse neque adipisci velit repellat cupiditate. Dolores dolor totam veritatis fugit est dolores. Molestias praesentium possimus nulla nulla sequi aspernatur ex eveniet.

I'm on the pursuit of happiness and I know everything that shine ain't always gonna be gold. I'll be fine once I get it

Career Advancement Opportunities

March 2024 Investment Banking

  • Jefferies & Company 02 99.4%
  • Goldman Sachs 19 98.8%
  • Harris Williams & Co. (++) 98.3%
  • Lazard Freres 02 97.7%
  • JPMorgan Chase 03 97.1%

Overall Employee Satisfaction

March 2024 Investment Banking

  • Harris Williams & Co. 18 99.4%
  • JPMorgan Chase 10 98.8%
  • Lazard Freres 05 98.3%
  • Morgan Stanley 07 97.7%
  • William Blair 03 97.1%

Professional Growth Opportunities

March 2024 Investment Banking

  • Lazard Freres 01 99.4%
  • Jefferies & Company 02 98.8%
  • Goldman Sachs 17 98.3%
  • Moelis & Company 07 97.7%
  • JPMorgan Chase 05 97.1%

Total Avg Compensation

March 2024 Investment Banking

  • Director/MD (5) $648
  • Vice President (19) $385
  • Associates (86) $261
  • 3rd+ Year Analyst (13) $181
  • Intern/Summer Associate (33) $170
  • 2nd Year Analyst (66) $168
  • 1st Year Analyst (202) $159
  • Intern/Summer Analyst (144) $101
notes
16 IB Interviews Notes

“... there’s no excuse to not take advantage of the resources out there available to you. Best value for your $ are the...”

Leaderboard

1
redever's picture
redever
99.2
2
Secyh62's picture
Secyh62
99.0
3
Betsy Massar's picture
Betsy Massar
99.0
4
BankonBanking's picture
BankonBanking
99.0
5
dosk17's picture
dosk17
98.9
6
DrApeman's picture
DrApeman
98.9
7
kanon's picture
kanon
98.9
8
CompBanker's picture
CompBanker
98.9
9
GameTheory's picture
GameTheory
98.9
10
Jamoldo's picture
Jamoldo
98.8
success
From 10 rejections to 1 dream investment banking internship

“... I believe it was the single biggest reason why I ended up with an offer...”