Sunday, November 22, 2009

Subjectivity In Grading

Phoebe recently posted about objectivity in grading:
There is a rather widespread idea amongst college students - and I remember this from college myself - that a good grade on even an undeniably quantitative assignment means 'the teacher likes me', whereas a bad grade is assumed the result of a teacher extracting cathartic revenge on a student who parts her hair on the wrong side or has otherwise unintentionally offended her instructor. Now having done a bit of grading at this point, I'm well aware how very untrue this assumption is - grading is necessary for providing feedback and all that, but is the least interesting aspect of teaching, falling well behind lesson-planning and giving a class. Objectivity in grading is not only the right thing to do, but the default.

I never really worried about whether teachers liked me or not affecting my grades. But what I do doubt is whether grading is truly objective.

For example, if I were in Dance's class and turned in a paper about how abolition was simply a function of vast economic and technological changes that replaced energy provided by man and beast with inanimate sources of energy, then I'm going to have to produce far more evidence than somebody who simply attributes it to the abolition movement and Abraham Lincoln. Now, my thesis is more sweeping and less direct, so more evidence is required. But to the extent that Dance personally agrees with a thesis I assume it must influence her view of how compelling the evidence is and consequently the strength of the argument itself.

Let's say for the sake of argument that Dance views European colonialism as deeply flawed, perhaps outright immoral. If two students submit papers, one praising colonialism and one decrying it, then doesn't the one who disagrees with her have an uphill battle to even get to the same point where she would find the evidence and argument equally compelling?

I choose Dance largely because I feel I owe her a thorough and detailed examination of the economic and technological changes across countries that support my conclusion on abolition, but I'm always too lazy to gather the data and all that. But this isn't her failing. It's a human failing. If I submitted a paper arguing in favor of the Palestinians' Right of Return to Withywindle, then I'm going to be held to a higher standard than somebody who argues the opposite.

I'm not saying that simply because nobody can be perfectly objective that professors shouldn't try, but I guess what I'm wondering is how a professor deals with that? Do they take their own biases into account? Do they downplay the actual content and persuasiveness of the argument in favor of more objective things related to the execution, such as grammar, organization, correct use of citations, etc?

I'll say this. It wasn't hard for me to know where professors' sympathies rested, and it is my perception that my papers that adhered to their sympathies got better grades than ones that did not. And this wasn't an isolated incident, but across the board. It wasn't a huge difference. Perhaps a A- instead of an A. Or B+ instead of an A-. However, the worst offenders were history professors. Every history paper I wrote at Georgetown that didn't adhere to the professor's ideology got a B. Every one that did got an A or A-. I have to say that might be why I am so skeptical of the academic discipline of history.

14 comments:

dance said...

to the extent that Dance personally agrees with a thesis I assume it must influence her view of how compelling the evidence is and consequently the strength of the argument itself.

True, but not actually the predominant factor in grading.

A student once turned in a paper arguing slavery was kind and gentle (he felt the need to assure me personally that he didn't really believe that). It got the highest grade in the class. Within the parameters of the assignment (develop an argument based on a single primary source), it did an excellent job.

A situation I really did have---a student argued early modern Chinese withdrawal from the world was based on inherent cultural factors, and yes, I vehemently disagreed, and yes, he got a lowish grade (B-?) because I held it to a higher standard of proof, and yes he came to see me, and I told him exactly that. EXCEPT, that I had just spent the previous half of world history (which he had taken with me), and the previous 8 weeks of his class arguing *against* culture as the essence of cause and effect. So I dinged him not because I was not convinced, but because I knew that he had access to lots of counterarguments that he had ignored. How did your B papers address the counterarguments?

Conversely, I have written a lot of comments along the lines of "this is true/I agree, but you haven't proved it." If professors *don't* write those sorts of comments, then that's much more of a problem than the previous China example, in my view. It's true that I'm guaranteed to see the logical flaws in things I disagree with, but I don't think I'm overlooking flaws in things I agree with---in fact, I frequently see sentences that make me think "don't write that just because you think that's what I think."

I have also written comments such as "good interpretation, although *I* happen to know it's not true." And those papers receive no penalty for that.

So compelling evidence and strength of argument is judged only by what I know they have access to.

With the slavery paper you describe, there's no way it would be a good paper in my class, regardless of whether I agree. Because I teach a class that is heavily based on primary-sources, and I simply don't give my students the tools to make that sort of argument, because proving something like that requires reading about 300 books on the world history of slavery, labor, and production. What were your B papers trying to do?

Subjectivity comes in the way the class is set up, more than in how grading standards are applied.

Withywindle said...

Proper grammar, as you know, is really big with me. Ah, how nice it would be to have students who had grammatical opinions of any sort.

History profs, according to the survey I blogged about a year or three back, may be even more Democratic (if not more radical) than any other discipline. They may be peculiarly politicized. Shame.

dance said...

PS. I think European colonialism was deeply flawed, but also a natural human response to the context. I think "immoral" is irrelevant to teaching history, and my students should be making arguments about why things happened or how things worked, not passing judgment. Neither "X was good" or "X was immoral" is a suitable argument, in my view/classes. So both your hypothetical papers would have a serious uphill to climb.

To use semi-real, half-remembered, examples:
"Turkey was enlightened in its treatment of women" is a bad thesis, even though I agree that freedom of choice and movement for women is an enlightened stance--"Upper-class women in Turkey enjoyed significant amounts of freedom and agency" is a fine thesis. Both are supported with the same readings and examples. Conventional wisdom may offer a setup for the thesis ("We think of women in Muslim countries as oppressed, but the reality is more complex..."), or students may end their paper with an epilogue-style sentence or two that speaks to morality or contemporary judgments, but such should not be part of the essay proper. Judgment may only be passed on hypocrisy, using historical standards, not today's standards. (Luckily, there's a TON of hypocrisy in European colonialism to attack)

FLG said...

A paper:
"The perception by Europeans of the other peoples in the Atlantic World as uncivilized and inferior provided a necessary rationalization for the subsequent conquest and exploitation of the native populations of the Americas and Africa."

B paper:
"While the Portuguese trade-based expansion was less coercive than the Spanish method of conquest, it was also less effective in spreading Christianity over the long-term."

B paper:
"The Cold War, with its bipolar political order and ideological struggle, was the defining factor in the formation of the aggressive American foreign policy in the Pacific realm from 1950 until 1975, and the policy choices of the United States inevitably had enormous effects across the region."

dance said...

None of those strike me as particularly ideological? What was the disagreement you knew you were making, and why?

("Defining factor" and "inevitably" can be hard to back up within the confines of an undergrad class, depending on the readings assigned)

FLG said...

With the Spanish-Portuguese. paper I was disagreeing with the less coercive = better vibe I was getting from the prof, which I understood at the time, but can't really remember specific evidence for, to be part of the general Leftie aversion to power thing.

It's not simply political ideology, but various academic ideas as well. The Cold War disagreement was largely with the prof not on political grounds, but on the idea of the relative importance of the actions of nation-states as opposed to various multinational sociological, cultural, political, and economic forces, memes, and organizations. A bias that is particularly strong in the Georgetown dept, IMO.

I didn't limit myself to the readings assigned in class. I utilized interview transcripts from The Foreign Affairs Oral History Collection of the Association for Diplomatic Studies and Training at the Library of Congress. I dug up a letter from President Kennedy to the President of South Vietnam. Just to name a few of the sources. In fact, I only used two sources from the class itself because of the non-nation-state bias. Not that I have anything against that per se (I have my own multinational economic theories of history), but it didn't offer much info on what I was arguing.

And before somebody questions whether the paper was on the topic, the assignment was something like, write about what had the most impact in the Pacific realm between 1950-1975.

Miss Self-Important said...

On the other hand, I am probably more conservative than FLG and never received any B's from my history professors. (I did, however, once receive a C- on an exam, but mostly b/c I had a studyfail of extreme proportions.) So, how does FLG account for evidence counter to his thesis?

dance said...

I should think with general lefties "worse at mental colonization" would support "better", not challenge it. That's a good thesis, though.

Cold War---yes, could be. But "most", as well as the thesis you chose, requires you to discuss why other things aren't so important, and it doesn't sound like you did that, if you only referenced 2 readings from the course. Because we commonly assume that the state has all the power, students writing for e.g., indigenous Filipino Nationalism, as the biggest factor likely automatically recognized they needed to defend a counter-intuitive argument and actively dismissed the obvious state power of army, law, etc. And "attack counterarguments, if they exist" is a pretty fair standard. I mean, your prof is giving you real information. It's not unfair to expect you to take it into account and see how it connects to things. Your outside sources also read a bit like "have a thesis and cherrypicking to prove it" rather than "building an interpretation from what I've got", and the former is a bit anathema to historians.

Anyhow, it's not hard to see how the essay might have fairly deserved a B. The real issue---might weak essays get an A because they agree with the prof? Your A thesis reads a bit like pandering---but it's also a much less complex and easier argument to prove than the other two, and students who know their limits *do* benefit in grading. I also see a lot of opportunity for nuanced reading of sources and intriguing sub-arguments that could warrant an A even with an just-okay thesis, in that essay.

To really assess this, I think you'd need to look at comments. When you wrote a thesis that disagrees with the prof, did you get different comments? Did the comments spend more time than usual focusing on content of the argument? Or did you get called on structural things you did without notice in other essays?

FLG said...

Dance:

Just to be clear. I said this was my perception, but I'm open to the idea that could have been something I was doing/not doing. Also, I'm not saying that my teachers were out to get me because I'm conservative. Just to be clear on that.

On the Cold War issue, I did make an argument that the Cold War was the dominant international conflict in the years concerned and that the actions of the two superpowers were subsequently extremely important. Then I proceeded to the United States' direct role in two major wars in the region and primarily about how the United States perception of anti-colonialization in the Pacific region changed from generally supportive, as a hold over from Wilson's 14 points, to hostile when communism became the driving ideology of anti-colonialization. But perhaps I assumed that the Cold War between two superpowers was self-evidently more important than, as you put it, indigenous Filipino Nationalism.

Miss Self-Important:
As I wrote above, I'm not saying profs are out to get conservatives. Just that I think they are held to a higher standard. Since you are far smarter than me, as evidenced by your current status as a Harvard PhD candidate, you obviously had no trouble overcoming the somewhat higher bar while intellectually mediocre yours truly did.

FLG said...

Also, I was thinking about how I accused Georgetown history of bias toward "various multinational sociological, cultural, political, and economic forces, memes, and organizations" over nation-states. Part of my perception is do to self-selection bias of both myself and my professors. I took history of the Atlantic and Pacific worlds. Obviously, these are going to focus on the factors that occurred across and around those oceans to explain what brought people together, into conflict, apart, etc.

Just wanted to put that out there, but even accounting for that I think it was still biased.

fabius.maximus.cunctator said...

FLG,

Highly interesting.

From my own experience of grading legal papers: gradings sucks. It is on a par with sitting in a dentist`s chair except you have to work at it and it takes a full day to correct 40 papers.

Anything that is readable, grammatically correct and not obviously false gives me a feeling of post-traumatic relief which is difficult to understand unless you have "done" 200 sloppy papers yrself. From that experince I wd argue that many teachers are probably soooo glad to get anything reasonably good to read that ideology fades into the background.

Caveat: my stuff used to be so technical that it wd have been difficult to get ideology into it in any shape or form.

As a schoolboy / student in Germany in the early, peacenik ridden 80s, faced with ex- or active peacenik teachers I got away with a very hawkish, conservative attitude.

Absence of bias ? Probably not. Other biases are stronger: Teachers are incredibly biased in favour of active (in class), articulate students which enabled me to get away with vy little homework among other things.
They are also strongly biased in favour of their own kind i.e. children of other teachers, academics etc. whose parents know too much, earn little and "do" even less. My best pal at the time got 9 sorts of hell every day for being the son of a businessman while I belonged to the right caste via my parents.
Finally, teachers are herd animals like brokers or bankers. If a student has been consistently A and A- there is a certain reluctance to change. I have occasionally written stuff for friends in need and it was not often graded as high as under my own "brand".

Btw. history is just the same around here - no way cd I have studied this subject at university altough I took a great interest in it.

FLG said...

FMC:
"From my own experience of grading legal papers: gradings sucks. It is on a par with sitting in a dentist`s chair except you have to work at it and it takes a full day to correct 40 papers."

That does sound awful, but the type of job that requires one to grade legal papers presumably has a lot of good things about it as well.

Flavia said...

This question isn't really relevant for my literature classes (politicized readings of literary texts--even Renaissance ones--are certainly possible, but rare at the undergraduate or even M.A. level), but it does apply when I teach freshman composition.

I don't think comp papers that I disagree with, politically, are subjected to a higher standard, but that probably has to do with the kinds of skills freshman have and the kinds of papers they're capable of--I think I'm at least as likely to give a low grade to an essay taking a position that I basically agree with as one taking one I disagree with, simply because what I'm looking for are basic skills: acknowledging that counterarguments exist, and attempting to address them; using a tone appropriate to the subject matter or assignment; not making straw-man arguments or assuming that one's audience already knows the debate, etc.

I'm willing to believe that I can't always see my own prejudices. . . but I actually like a student who mounts a really strong argument for a case I disagree with (basically, I like a smart, passionate student, regardless of his or her politics!). My best/favorite comp students have often been fierce libertarian boys, who a) like being pushed to defend their beliefs, and b)tend to assume I'm acting in good faith when I do so.

fabius.maximus.cunctator said...

FLG

Yes ! Absolutely. I was offered a very small teaching assignment "on the side" cple of years back. Liked it enormously. Had to give it up when I took a new job this year.

Grading was the main downside plus occasional frustration about the cumbersome bureaucracy involved.

On balance it was great: I admit I do like an audience. The audience deserves thorough preparation. That means relearning the stuff you do daily and questioning how and why you do it.

The latter part cost one of our adversaries a 6fig sum in a court action because -inspired by teaching - I questioned the line everybody had been taking and suggested a new line which the court loved. The "click" actually came during a lecture.

Btw. I can`t say whether teaching full time gives the same satisfaction. Most fulltime lawprofs have extra jobs I notice.

 
Creative Commons License
This work is licensed under a Creative Commons Attribution-No Derivative Works 3.0 United States License.