In Search of Excellence, Creating My First Test of the Semester

I would have hoped after doing this for 29 years I’d know how to make up the perfect test. But I’m teaching a new prep this semester, and so I am filled with the usual doubts.

Will the test be fair, will there be enough time for the students to complete the test, will the test be at the right level of difficulty. My hope is that the test will be a useful learning experience for the students.

I think the fairness issue is the easiest one to deal with. As long as the test covers what we have discussed in class and the students have read in their textbooks, then I should be OK. But I’m guessing that at least one student will comment that something on the test wasn’t fair, or that I was trying to trick them.

The time issue is a little harder to deal with. The students will have 75 minutes for the test. It’s not a good sign if a few students are done such a test in 20-30 minutes, probably a sign that it was too easy. On the other hand, it’s also not a good sign if no one has handed the test in when the 75 minutes have passed. Usually a lot of stressed out students when that happens. I’ve also learned over the years that many students will take as much time as is given. They may have completed the test in 60 minutes, but they will double-check and triple-check every answer, and always be the last ones to turn in their tests.

The level of difficulty is the one I struggle with the most. I think it is fairly easy to make up a really hard exam, as well as a really easy exam. I’d like to be somewhere in between, to find that sweet spot.

I know many teachers believe that  difficult tests motivate students to perform at their highest level possible. That approach is consistent with a good deal of the goal setting and motivational research available. I’ve heard about many engineering tests where the class average is a 40, and I’ve never understood what the point of such tests were. Is it for the teacher to prove how much smarter he or she is than the students? Is it to remind the students how difficult the material is? Is it designed to motivate students to try harder on the next test. To me, such a test does not reflect poorly on the students as much as it does on the teacher. The test may not have been designed properly in terms of fairness, or time, or level of difficulty, or it may indicate that the teacher is not very effective in teaching the material. I’m also thinking I would never want to drive across a bridge that was designed by one of these students…

I’ve also heard about tests where the average score is in the upper 90s. I think that’s too easy, and does not serve to motivate students to push themselves at all.

I view designing a test that is in the sweet spot as a two-part process. First, I need to decide what the sweet spot should be. Is a 65  average the sweet spot, is it an 80, is it having at least half the students score in the 90s?

While much of the goal setting literature talks about setting goals that are relatively difficult to attain, I am more aligned with the approach that IBM used with its salespeople, discussed in what I consider to be one of the best business books of all time, In Search of Excellence.

In the book the authors talk about how IBM (one of the excellent companies) sets the standards for its sales people. Rather than setting the quotas at level where perhaps only 40% of the sales people would achieve that level, IBM sets the quota so that 70 to 80% of its sales people make quota.

Such a finding was consistent across the “excellent” companies, and the authors concluded that the reasoning was fairly simple; people like to consider themselves winners. And the excellent companies designed their systems to continually reinforce that notion; most of the people are made to feel like winners. So even though its workers are distributed around a normal curve like any large population, the excellent companies reinforced degrees of winning rather than degrees of losing. Thus, most of its people make their targets and quotas because such goals are set to allow that to happen.

In a company where only 40% of the employees are hitting their goals, such an approach means that at least 60% of the people think of themselves as losers, leading to dysfunctional, unpredictable, and frenetic behavior.

So I have long been a believer in the IBM approach, and try to set goals for a test where the majority of the students do well, which I translate to having test averages in the low to mid 80s. My sense is that this is a good bit higher than what most teachers strive for, but I am quite comfortable with this approach.

So all the above is really the first step, deciding what I want the sweet spot to be. Once I have settled on that (low to mid 80s), the second part of the process is now designing a test where that is the outcome. That is the hard part, and usually takes some trial and error.

Since this will be the first test for a new course, I will likely err on the side of being a little easy. To me, it’s helpful to build some confidence, to get an early win. I think that creates motivation to want to continue at that level for the rest of the semester, or at least until the next test.

So while I may have made it sound like designing a test is a lot of work, I’m still glad that I am on the designing side, and not the taking side. It truly is better to give (a test) than to receive.