A multiple choice exam is looking for a sweetspot of a certain percentage passing, not too high and not too low.
The multiple choice question may be one of the most despised games ever conceived. The purpose of a multiple choice exam is to exclude people in a quantitative manner, be it for admission into schools, licensing professionals, or limiting the number of high grades in a class. Assessing a person on their individual merits is a time consuming process, and once a school or class hits a critical mass of students, it isn’t economically reasonable to scrutinize all of them. Let’s say you’ve got 1000 participants and five people reading their results. You can cut time and costs by figuring out a way to neatly get rid of 500 because they scored under a certain amount. A multiple choice exam cannot be so difficult that you exclude an excessive number of applicants. Most law schools, for example, have a minimum LSAT score that you must score below for automatic denial and a high score for automatic acceptance. Applicants in between those scores are then addressed on an individual level and other factors are introduced. The problem with such a system is that to ensure a multiple choice test produces the right number of passing scores, you have to keep changing the questions.
An article by the National Center for Fair and Open Testing explains, “multiple-choice items are an inexpensive and efficient way to check on factual (“declarative”) knowledge and routine procedures. However, they are not useful for assessing critical or higher order thinking in a subject, the ability to write, or the ability to apply knowledge or solve problems” (“Multiple-Choice Tests”, Education.com). It’s for that reason that a multiple choice question is always limited in scope: it can only be about basic knowledge of a topic. The formula is to have two answers that are blatantly wrong, one that is kinda right, and one that is the most right. One instructor during a review session for the BAR pointed out that on average a student will know the correct answer immediately to 25% of the problems, have no clue on 25%, and be able to boil it down to the right and kinda right answer for the other 50%. So the way that you evaluate the difficulty of a multiple choice question is how similar the right and kinda right answers are. A person who can’t boil it down to those two doesn’t know the basic material and shouldn’t pass.