14 Reasons Multiple Choice Tests Suck #edreform

The kids are testing right now, filling out bubbles and every part of me wants to burst the bubble and set them free. It’s a factory-transmission-mind-dump and even though I know they can articulate their learning, they have to prove it quantifiably on a kill-and-drill test that is both easier and harder than it should be.

And me? I’m feeling more like a drill-sergeant-text-proctor today than a teacher. I am bothered by the silence, by the tension and by the unnecessary adversarial relationship it creates.

Want to find the cheapest way to reform education? Get rid of the incessant multiple-choice testing. It won’t fix everything, but it would be a great first step. Here are fourteen reasons why we should abandon our obsession with multiple choice tests:

  1. Multiple choice is shallow. I know people try to create analytical, evaluative and creative questions. However, the medium itself is one of recall and recognition. 
  2. Knowledge is often connective and deeply rooted in context. Multiple choice takes away connective thinking and puts it into empty silos, where knowledge is reduced to lowest common denominator. I want students to know information. However, I also want them to know applied information. I want them to put the information together.
  3. Multiple choice is unreliable. As long as a student can guess a correct answer, every question is somewhat suspect. I get it. Teachers can look for overall trends and take out the statistical probability of guessing. However, how does that help anyone figure out if a child needs additional support with a concept. 
  4. There is rarely a chance to explain why something is true. Students should be able to articulate what they know and give a defense for why it is true. 
  5. Multiple choice kills the desire to learn. It might not sound like a big deal, but every time my students take a test, they are less likely to enjoy what they learn. 
  6. If you try for critical thinking, multiple choice tests become subjective. Students end up with questions like, "Which of the following best describes . . ." and the test becomes meaningless.
  7. Multiple choice does not allow for nuance, paradox or mystery. 
  8. Students need to see knowledge as contextual and personal. Multiple choice tests are standardized, impersonal and void of any real context. Students internalize the idea that learning is something irrelevant. 
  9. It reduces self-efficacy. Multiple choice tests fail to allow students to find information themselves and make decisions about their learning. To do so would make tests "unreliable." Too many variables. Unfortunately, life has multiple variables and "learn to be a critical thinking citizen" cannot be easily measured.
  10. Multiple choice pushes students toward a narrow, celebral definition of learning. Multiple choice does not allow for social learning. Instead, students must prove what they know in isolation. It doesn't allow for multiple modalities or differentiation, either. 
  11. Google has replaced multiple choice. In an era when knowledge is instantly available, we should be seeing whether students can find information, ask deep questions, engage with sources, curate what they find and find the bias in a source. Multiple choice doesn't allow for any of these necessary skills.
  12. It's way too easy to cheat, both for teachers and for students. 
  13. Learning is often a creative endeavor. However, multiple choice tests do not allow students to be creative, divergent or innovative. 
  14. Multiple choice tests are often not valid. For example, if a standard is concept standard, it often cannot be assessed in multiple options. If a standard is a process or a product, they probably should be assessed by actually doing the process and making the product. So, we're left with the assessment of skills, and only those that fit nicely into multiple options.
So, that leaves me thinking about the Common Core Standards (proper noun there -- very, proper, from very proper people). I embrace the fact that the new standards push for critical thinking. I am glad we're finally talking about innovation. I'm impressed that digital literacy now matters. 

But how will we assess it? 

My guess is that in order for the tests to be reliable and in order to aggregate the data efficiently, we're going to see more multiple choice questions. And here's the thing: the format and the content influence each other. Trying to push critical thinking and creativity into multiple choice is like trying to put Jesus onto a shallow device like a television. People do it, but they probably shouldn't.

Some Nuance:
I'm not opposed to the tool entirely. Multiple choice makes sense when looking for large data trends and efficiently measuring certain (though a very limited number) of standards. My issue is this: we are spending weeks (at least six) taking multiple choice tests. We are using the tool wrong the vast majority of the time.
SHARE

John Spencer

John Spencer is a teacher, author, speaker, and incessant doodler. He is the co-author of Wendell the World's Worst Wizard and the co-founder of Write About. He is passionate about helping students develop into better writers and deeper thinkers.

  • Image
  • Image
  • Image
  • Image
    Blogger Comment
    Facebook Comment

35 comments:

  1. I'm with you completely. Let's get rid of rigorous tests and replace it with circle time and kite-flying. That's the best way to build a global economy.

    ReplyDelete
    Replies
    1. I'm glad you were able to read the post carefully and make sure it was all about circle time. After all, that's exactly what I'm advocating. Get rid of tests and replace them with circles.

      Delete
  2. Really "circle time" and "kite-flying", are you serious?

    ReplyDelete
  3. @anonymous what does building a global economy have to do with education?

    ReplyDelete
    Replies
    1. You can try and make education into some humanistic religious quest where the purpose is to become a better person or whatever. However, the reality is that our children aren't growing up with the necessary skills to make it in a competitive economy. This isn't heartless. This is a reality. The sooner we realize it, the faster we'll be able to improve and catch up to the rest of the world.

      Delete
    2. What proof do you have that children aren't growing up with the necessary skills? Who are you trying to catch up with?

      Delete
    3. Rule number one is never start a land war in Asia. Rule number two is never start an argument over standardized tests with Joe Bower.

      Delete
  4. Nice list John. Being critical of multiple choice tests is not a popular stance to take. The tyranny of the status quo is a powerful force to overcome. Many will call you lazy, some will call you crazy and only a few will even bother hearing you out.

    School has been the same way for a very long time. If school ain't changing, then it ain't improving. Keep encouraging people to rethink their basic assumptions about what school should look like, feel like, smell like, taste like and sound like.

    I wrote a post about multiple choice tests here: http://www.joebower.org/2012/05/folly-multiple-choice.html

    ReplyDelete
    Replies
    1. True. It's unpopular, though not entirely unpopular. There are definitely a fair number of teachers who are outraged by the sheer amount of testing.

      Delete
  5. You think passing a multiple choice, standardized test is helping kids develop necessary skills for the economy?

    How about developing interpersonal and problem solving skills?

    ReplyDelete
  6. Replies
    1. I don't have an account, I was led here through a link and saw a comment I found ignorant.

      Delete
    2. I allow anonymous posts, because there are times it really helps with the conversation. Other times, it invites trolls. I've tried it both ways.

      Delete
  7. John - Thanks for connecting on Twitter. It's always a pleasure when conversations move from static to dynamic. I know you shared on Twitter that your focus is more on an over-reliance on MC and their abuse but I'd like to put forth the same scenario I pose to Joe on his blog:

    Please consider the following: A chemistry teacher (a well-respect, well-liked woman) I know sees every Junior, and Senior who failed the course the year before, for a Chemistry course. That's almost 90 students. According her content standards, all of her students need to be able to identify the difference between "density" and "particle size". All 90 students take a chem lab. All 90 students engage in authentic assessment where they explore the essential question: "Are laws always considered truths?" All 90 of her students take a test in the midst of that unit with 15 multiple choice questions that requires them to correctly identify the meaning or concept behind different, yet seemingly similar, chemistry terms and concepts. While she could have gone through 90 science labs to ensure all of her students correctly used the terminology correctly, she elects to focus the time she spends reading labs on giving quality feedback on their processes. Her standards say her students need to do x, she assess and documents their ability to do x.

    Perhaps she has too many students. Perhaps her standards are too specific. But, she has the students she has and her standards are what her standards are. Well-designed multiple choice questions can provide information about student learning. And while I recognize we are both entrenched in our views, a cursory view of the assessment texts I have in arms reach (Nitko, Brookhart, Jaeger, VanBlerkom, Stiggins, etc.) all speak to caution of using multiple choice and recognizing when they are appropriate and inappropriate. I was unable to locate any research or literature against them - beside Mr. Kohn's warnings that you cited. Fairtest.org openly addresses what they can and cannot do: http://www.fairtest.org/facts/mctfcat.html

    So my question - Is she a bad teacher because she's using a tool that sucks? What should she be doing instead?

    ReplyDelete
    Replies
    1. I'm not saying she's a bad teacher. I added some nuance to the post. There are times when multiple choice makes sense. However, in the example you mention, I think identifying the difference between density and particle size is something that might work better with a short answer test rather than multiple choice. I would also look at a lab write-up, a discussion, etc. and either check if the terminology is correct or have the students self-assess their word choice.

      Delete
    2. The point I tried to stress in my example is that it's 90 students. She's decided her time is better spent giving students feedback on the quality of their labs rather than skimming for their use of terminology. Time is finite. If there's a tool than can measure the things that can easily be measured, its strategic use can free up time to measure the complex, the beautiful, and the powerful.

      Delete
    3. I see that point, but I'm just saying that with finite time, it's still pretty doable to ask students to do a short-response vocabulary piece, offer a graphic organizer or listen to the discourse in a classroom.

      Delete
    4. It's doable, but I think it's making more work for the sake of avoiding multiple choice :) And I hope you know, all of this being said, 90% of the assessment work I do with teachers is performance-based assessment design. But when a HS Physics teacher asks for support in designing quality, complex challenging multiple choice questions for his course-load of 105 students, I'm not going to tell him no. One good round of standards alignment (i.e. checks for validity), he's likely to get there on his own. :)

      Delete
  8. Why not mix it up a bit. I am currently teaching Year 7's a unit on Ancient Egypt. The class is broken up into small groups to become Experts on a small topic. Two groups on each topic.... which becomes important below. They then have to Teach that topic to the other groups in the class.
    In order to test their knowledge and their teaching ability they will then develop a short Multiple Choice Test on their topic which the other students will complete. Of course there is an expectation that they will ACE their questions, but they are also expected to ACE the other questions on their topic from the second group.We have talked about how to construct good assessment items and they will create the test as a Google Form, thus increasing ICT skills. They seem excited and it seems much more educationally souns that Can you Guess!

    ReplyDelete
    Replies
    1. While I see this as potentially valuable, I wonder if it might work better to have them use Google Forms to do surveys and get opinions on the subject rather than test basic knowledge. What if they created higher-level discourse questions? I could be off-base here and I admit that it varies from class to class. But I'm just wondering if multiple choice is the best route.

      Delete
    2. I dont disagree..... higher order questioning is far more valuable, but we were looking for something quick, easy and engaging. Some of the class were not happy with MC and wanted to do things which were more visual or better showed relationships. At the end of the day, as you would agree, we want a range of different assessment strategies (Of, As and For Assessment)to elicit the greatest understanding of our students learnings

      Delete
  9. I love the time and thought you put into this post John. I have been feeling this way for quite a while now. As I have gathered teams to review student data and determine which students need interventions or not, we have seen numerous instances where multiple choice tests have given us poor data that conflicted with what teachers observed in daily performance. It may be easy for an outside of education anonymous person to belittle this post, as they do not have the hands on experience to lay witness to how testing data can be misconstrued and worse yet, not see how an unending and relentless flow of multiple choice tests simply sucks the enjoyment out of learning. The emphasis put on the mandated standardized tests is enough to turn many kids off, let alone the pressure put on secondary students to perform well on these tests so they can get into college's that ask them to perform completely different types of assessments.

    While faulty data from these types of assessments is not your focus, multiple choice does not adequately assess student learning, especially if our desire is not to race through a set of state (or federal now) mandated standards, but rather to assess deep learning and mastery of a skill. Many teachers are under the assumption that they must load up a gradebook and even worse, many of them still use the paleolithic 100 pt scale with the F cut-off being 65-69%. Even scarier is the neanderthal that still strongly endorses the "0" for no homework or missed tests policy. We will pound responsibility through their skulls with the biggest hammer we can find. Ahhhh....it is so hard to argue with such logic. Sorry to wander of topic a bit, but I love your post and thanks for sharing and enlightening.

    ReplyDelete
    Replies
    1. Tom - I think your comment highlights the tension between multiple choice tests and their use. I disagree with John's supposition that multiple choice tests suck and appreciate his nuance revision. Multiple choice tests don't suck any more than blood work sucks. What sucks is when people trust them more than other measures because they erroneously believe they are more objective. When teachers rely on them more than other measures or lack the training in how to design quality multiple choice items. I feel uncomfortable whenever people bash multiple choice tests as everyday, teachers make the strategic decision to use them. Sometimes it's the right decision, sometimes it's not. But to claim a tool in a teacher's assessment toolbox sucks or somehow kills student creativity by virtue of existing is misleading and unfair. Bad policy is bad. Bad assessment is bad. And not to belabor the point, but isn't the act of voting basically just taking a one question multiple choice test?

      Delete
    2. True, but voting is different. So are surveys. Multiple choice tests measure knowledge while voting and surveys measure opinions. Nobody claims that someone is knowledgable on a subject based upon an opinion survey or a choice preference.

      Delete
    3. Agreed. And no one should be claiming someone is knowledgeable on a subject based on a single test. No one measure can ever give us a full picture of skill and ability. Passing the Global Regents does not mean you are a historian; it means you've passed the Global Regents. (I keep wanting to add emoticons as it's getting late and I'm worried about my tone)

      Delete
  10. I get the point on both sides, but I have to say tests in general suck (especially the blood tests), or at least a majority of the tools we currently use. While assessment is a necessary part of education and it can be a timely one if done correctly, I am still more supportive of deeper, more hands on formative assessment, that can be looked at cumulatively to drive the determination of student performance levels. Multiple choice has its place, but what I would hate to see continued is the 50 to 100 question test that is all multiple choice. In those instances (and they still do exist, even in high performing schools) I think we have just supported the notion that test scores are the important measure and that quick data collection and analysis is what learning has become about.

    ReplyDelete
  11. I read this post and the comments this morning and fell roughly where @DataDiva does on the issue. Then today I re-read Lorrie Shepard's "Assessment in a Learning Culture" and I think that we could be a bit clearer about multiple choice tests (in the way they're usually designed and administered) being a symptom, not the disease. I've seen some well-designed multiple choice items (see Briggs, et al's "Diagnostic Assessment with Multiple Choice Items," if you have access) and I don't think Shepard would say multiple choice is inherently bad. Instead, the way we often use multiple choice reflects how assessment practices haven't kept up with the advances in the ways we think about student learning, and that frustrates us.

    ReplyDelete
    Replies
    1. That's why I added some nuance at the end. I don't see it as inherently bad, but something that should be used sparingly.

      Delete
  12. I completely agree that they area waste of time. Spending 6 weeks on preparing kids for the test and having them take the test is road kill. I wish I had a solution to this problem.

    ReplyDelete
  13. I prefer multiple because even though I may know the answer very well, due to my ADD, Dyslexia and Aspergers I have a hard time explaining it in my own words (I can but it requires a lot more attention than most people, just so I can make sense to the neurotypical people). Like I KNOW but I can't always SAY. If you ask me to do it, I will do it (and often better than a lot of people). Ask me HOW I did that? That's different. I would rather show you. That is how my brain works.

    ReplyDelete
  14. I think that you bring up some interesting points, and as you mention others have raised similar issues. I find myself aligned with much of your thinking. To add further complexity to this issue, I will briefly comment on a few of your points that I find controversial.

    3. Multiple choice is unreliable. As long as a student can guess a correct answer, every question is somewhat suspect.

    I would suggest that you look at the literature on item response theory (IRT). I don't have time to explain this to you now, but we can model latent traits as a function of item difficulty while accounting for guessing parameters. This is the basis for computer adaptive testing which tailors item content to each student.

    The other thing I would mention, is that it doesn't appear to me that you have an adequate understanding of "reliability". I will let you do your own literature review on this, but for now I will simply say that "tests" cannot be reliable or unreliable. Instead, "scores" are reliable or unreliable.

    Reliability of teacher judgements (scores provided for alternative assessment practices) are also highly suspect. Empirical evidence should be established for the reliability such "judgments" prior to accepting them as a legitimate assessment.

    14. Multiple choice tests are often not valid.

    First, interpretations and entailed uses of test-scores are valid or invalid. I admit that this stance is controversial in contemporary validity theory. Nevertheless, if we accept this position, then it is obvious that for a single test there may be multiple interpretations and entailed uses of these scores. For each interpretation or use, validity evidence is necessary. Consequently, validity evidence may be strong for one interpretation and weak for others. This is true of all "tests" or "assessment" processes. So this issue is not unique to multiple choice tests.

    I unfortunately do not have time to comment on many of your other points.

    I agree with much of what you have to say, but unfortunately many of your points appear unfounded. Once again, I think that you should consult the literature before making broad generalizations about multiple choice tests or any assessment procedure.



    ReplyDelete
  15. Is it possible to find a school where Multiple Choice Tests are not practiced?

    ReplyDelete

Please leave a comment. I enjoy the conversation.