Why All Grades Are Subjective

It's been quite a long time since I posted a blog entry. My life has been very busy with a full-time job, a dissertation to write, and a household to maintain. Today I'm going to share a reply I sent to a student email about why all grades are subjective.

Image Source: Alexander Russo. "This Week in Education: Cartoons: 'Climb That Tree.'" Scholastic. Accessed 24 April 2019. Note: The attribution of the above quote to Albert Einstein is almost certainly false. See its entry on Quote Investigator for more information on its probable origin.

For context, Monday was the first day of BYU-Idaho's Spring semester. As usual, we went over the syllabus, and as I explained my somewhat heterodox grading policy (which I implemented in part to address the challenges of subjectivity and grade disputes), I declared that grades have nothing at all to do with learning, and that all grades, not just English grades, are subjective. At this point, I paused. I told them that one of my grad school professors, James Paul Gee, often says that "Academics is an evidence game," which means that when we make claims, we provide reasons and evidence to support our claims, and in so doing, we subject our judgment to scrutiny. However, there was insufficient time at that juncture for me to provide evidence to support my claim that all grades are subjective. I said if any of them would like me to do so, they could email me, and I would be happy to oblige. One student did (the first to take me up on my offer in three years, hurrah!). Below, I copy my reply:

Dear Student,

I'm so glad you asked!  

All grades are subjective because teachers (or administrators, or state and national standards boards) have to make choices about what to measure, how to measure it, how to weight each thing they are measuring, and so on. Let’s take math as an example.

Most people consider math to be the least subjective of all academic disciplines, because, at least at the level of arithmetic and simple algebra, it is clear whether a student got the answer correct or not—whether their final calculation “adds up.” However, math teachers still have to decide whether to grade solely on whether students calculated the correct result, or whether to include the student’s process of calculating their result. In other words, they have to consider whether students should get partial credit depending on how well their calculations demonstrate that they are grasping the concepts, even if they make errors along the way and ultimately may not get the correct result. If teachers decide to grade on both process and result, they have to decide how to weigh process vs. result in determining a score.

One problem with grading based only on getting the correct result is that any student with a calculator and an understanding of how to use it can get a correct result, even if they do not understand the underlying mathematical principles. If we only care about whether students can use calculators correctly, then why teach math at all? The reason is because we need people who understand mathematical principles in order to conceptualize and solve difficult quantitative problems that machines cannot do all by themselves—we need mathematicians who can think holistically and creatively. That requires that we measure process—but standardized tests, which measure outcome, do not measure process. In fact, if you ask a professional mathematician whether process matters less than, as much as, or more than outcome, I guarantee they will say that process matters as much as or more than outcome. Some ways of formulating a calculation are better, “more elegant” than others, even when they both get the same result. Thus process matters, and expert judgment, which is to some degree always subjective, is required to evaluate students’ answers to set problems. But as any builder or engineer can tell you, getting the right answer to a mathematical calculation matters a great deal! So we can't grade solely on process, either.

In creating exams, teachers have to decide not only what sorts of problems to set, and in what form (i.e. written out, multiple choice, etc.), but also how to weight different kinds of questions. They have to decide how to prioritize the importance of different mathematical concepts in determining how well students are demonstrating learning the core objectives of the class. They have to decide what those objectives are. They have to decide whether to “curve” their grades or not. Furthermore, even when a test and its scores are “standardized,” the teaching itself may not be. Different teachers will naturally emphasize different aspects of a standardized curriculum, and will be better at teaching some concepts than others. That will likely affect student outcomes on standardized tests—so then, how much are the tests measuring student outcomes vs. teacher performance? This is one (misguided) reason why some national school standards programs have tried to penalize teachers when their students underperform on standardized tests. But that, too, is problematic, because teachers control very, very little of what our students come into our classes with and take away from our classes.

So let’s consider some aspects of the student half of the equation. Going back to the question of results vs. process, some students start out “ahead” of others. The students at the top may make very few gains over the course of a semester—in other words, they did not learn much. In contrast, students closer to the bottom may make lots of progress. Yet if grades are based on outcome, the students who started out ahead and learned little would get an A, while the students who started near the bottom and learned much might still only manage a C. On the other hand, if we measure process, then the student who learned the most but still has a poorer grasp of the subject would get an A, and the student who learned little but has a better grasp of the subject would get a C. That also seems unfair, doesn’t it? Would it be fair to measure both process and outcome, and give both students a B? I don’t know—that’s why it’s subjective.

But wait, there’s more! Evidence demonstrates that students who get a good night’s sleep and eat a good breakfast before an exam will score much better than those who don’t. And students who experience greater stress in their environments tend to struggle more in school—it’s hard to stay focused on math when you’re worried about whether your older brother is going to get killed by someone just because they think he “looks suspicious,” or whether your unemployed dad will be drunk when you get home, or whether you mom will have managed to save enough money from being spent on alcohol to buy you and your siblings some fast food for dinner (because your electricity has been turned off and you have no way to cook meals at home). It’s hard to get enough sleep when you get woken up by the sounds of gunfire. It’s hard to stay focused when your stomach is gnawed with hunger because your parents can barely afford to provide one meal per day, and the hard classes you have to take are all scheduled before you get to eat your school lunch. And so on.

Thus, if a student who lives in a secure neighborhood, in a secure home, with parents who are financially secure enough to provide regular meals and other kinds of support gets an A on the exam, and a student who lives in a dangerous neighborhood, who has had to move three times already this year (changing schools along the way), who sleeps on a mat on the floor in an apartment with thin walls through which she can hear the neighbors fighting until well after midnight, and whose mom is working three jobs just to make ends meet and cannot afford to provide breakfast gets a C on that same standardized exam—does that really reflect the academic merit of each student? If a teacher takes such obstacles into account, though, then those grades are obviously subjective. They are tailored by the expert judgment of a particular teacher about the needs, circumstances, strengths, and growth of a particular student. If the teacher does not take any of these environmental factors into account (perhaps even relying on a blind grading mechanism to ensure they don’t know which exam was marked by which student), then the grade appears more objective, but as I said: it measures outcome, not learning; and that is itself a subjective judgment call about what matters.

All of these factors influence the subjectivity of grades. Nevertheless, we must have a way to measure students’ learning and their grasp of core concepts. We must have a way to give them feedback about their progress. Administrators and school officials like grades because, as numbers, they seem objective. Furthermore, they’re easy to add up and track over time (which is advantageous to teachers as well as administrators and school boards). They’re scalable in a way that more specific written feedback is not. These administrators and school officials rarely stop to ask what those letters and numbers mean—what they are actually measuring. Learning is far more complex than can be measured by any set of numbers, let alone a cumulative course grade or GPA.

And speaking of GPA, let’s talk about the way it’s calculated. Here’s a standard* grading scale:

Percent Grade	Letter Grade	4.0 Scale
97-100	A+	4.0
93-96	A	4.0
90-92	A-	3.7
87-89	B+	3.3
83-86	B	3.0
80-82	B-	2.7
77-79	C+	2.3
73-76	C	2.0
70-72	C-	1.7
67-69	D+	1.3
65-66	D	1.0
Below 65	E/F	0.0

Scores for all graded assignments are totaled up (using weighted algorithms that vary from one class to another) into a final percentage, which is then converted into a letter grade. This reduces the complexity of the data, because it’s the letter grade that gets converted into a GPA. Note that in some cases, a difference of only 1% on a final grade (a score of 89% vs. 90%) results in a loss of .4 points in the calculated GPA—the same as a difference of 5% (an 87% vs. a 92%). The final grade and its attendant GPA tell us nothing about the relative difficulty of the class, what specifically the student actually learned and can implement outside the context of a regimented classroom, or how much progress they made from the beginning of the class to the end. It’s just a letter. It’s just a number.

Anyway, that’s why all grades are subjective.

Sincerely,

Sister Robinson

*Note that I said a standard grading scale, not the standard grading scales. There are variations from one school, academic department, and even one course to another. 

This is my new Betta, Irving Braxiatel. He earns an A+ in Being A Fish. This is the only grade that is completely objective.

Guilt-free Writing Time

This update is a little late because I was in Utah over the weekend celebrating my grandmother's 90th birthday. Fortunately, I was able to use my tablet to keep to my writing schedule while riding in the car on the way there and back. Without a WiFi connection, I couldn't access many of my notes, drafts, and reference articles, because they're all stored on Dropbox and apparently it didn't all sync to my tablet because I haven't been using my tablet nearly as much since I finally got a smartphone. That made it a little harder to put my writing time to as good a use as I would have liked, but I still wrote something. This potential problem with my writing infrastructure will have to be addressed in the near future.

The other thing I had to deal with last week was guilt-induced writer's block. See, I had this pile of student papers to grade, and I should have had them finished by the end of the previous week. So on Monday morning when I was trying to write, all I could think about was those papers that I still needed to grade. After 45 minutes of non-productivity, I gave into the guilt and went back to grading. The same thing happened Tuesday afternoon. On Wednesday morning, however, although I still wasn't done grading (it was a particularly arduous grading process for various reasons), as I sat down at my computer, my internal voice was telling me "I have to write now, even though the grading isn't done."

That "I have to write now" didn't feel good at all. Then it occurred to me that no, I don't have to write during my writing time--or any time, actually. But the whole point of having writing time set aside is that it means I get to write then, totally guilt-free, no matter what else I have to do that day, or that week, because writing time is writing time. Not grading time. Not answering frantic student emails time. Not helping friends' daughters with their Language Arts homework over the phone time. All that other stuff is what the rest of my time is for.

For two hours a day, I get to work on my own writing, without worrying about anything else. Writing time is guilt-free writing time. 

That was a tremendous realization for me. It probably shouldn't have been, but I'm just thick like that sometimes. The other thing that I had to do this past week was turn off literally every notification I could find on my computer, my tablet, and my smart phone. Notifications are my enemy. They make me feel bad about the things I'm not paying attention to right now, as opposed to being fully engaged with what I am doing right now.

Anyway, I'm still well behind schedule, but I'm still making progress. I've reached a point where I need to go back and re-read some Kenneth Burke, to clarify my definition of his comic frame and figure out how it maps onto the concepts of techne and phronesis. Reading Burke is fun, in the same way that opening clamshell packaging is fun. It's ridiculously hard but the goods inside are usually worth the challenge.

In other news, my housemate Stephanie got left behind this weekend as the rest of us were traveling. Apparently, she got a little lonely, and couldn't stand to wait any longer to buy my Christmas present. But she got attached to him herself over the weekend, so we're going to have joint custody of this beautiful little guy (I can't blame her. He's so relaxing to watch). He needs a name. Right now I'm thinking about Kazran, Rory, or Jim the Fish.

Jock the TA Octopus

My friends and I have been grading a lot lately. It's that time of the semester. Reading students' papers can be very interesting and rewarding, but it also (as I think most teachers would agree) frequently tends to be a frustrating experience, when it's not simply mind-numbing. Often, we fantasize about ways to make grading easier. Tonight, an exchange about grading between fellow Comp instructors on Facebook somehow led us to imagine sea slugs writing papers, and thence (naturally) to the idea of an octopus TA. 

It's actually not that far-fetched. I recently read some articles about octopus intelligence. They use tools, play, solve puzzles, are amazing escape artists, can crawl around on land, and have awesome camouflage skills. Then Abby shared this little article about Jock, an octopus in Scotland who has taken to cleaning his own tank. Feeling that an intelligent cephalopod like Jock might be interested in switching from janitorial to clerical work, in a moment of stress-relieving silliness, I invented Jock the Octopus TA. Below, I share some of Jock's recent assessments of student writing. Also, apparently octopuses refer to themselves in the third person. They are solitary creatures who evolved at the bottom of the ocean; their brains are not like our brains. Don't question it.

"Jock says your source is invalid. Try JSTOR next time. Jock loves peer-reviewed articles almost as much as he loves rearranging his tank furniture to be more feng shui." 

"Jock wishes to use this paragraph to scrub the scum off the side of his tank." 

"Jock is not interested in how comprehensive sex education is the solution to the failure of our entire educational infrastructure. He thinks you should write a paper about cephalopods' right to privacy." 

"QUOTING ACTUAL SCHOLARS! Jock approves."

"Jock finds your mastery of academic style over substance simultaneously impressive and depressing." 

"Jock is intrigued by your thesis statement, but finds your argument's credibility hampered by poor paragraph organization and weak source attribution."

 image source: dailyvisits.co.uk

 When Jock is not grading papers or cleaning his tank, he practices playing his bagpipes. He and Paul II, a German octopus guitarist, are hoping to start an international Octopus band. Paul II's predecessor was allegedly psychic, but Paul II is an empiricist and doesn't believe in such pseudoscientific nonsense. Inspired by his high-achieving bandmate, Paul II thinks he may have a future in academia as well--perhaps as a research assistant.