Total Posts:5|Showing Posts:1-5
Jump to topic:

Test Marking Systems

Logic_on_rails
Posts: 2,445
Add as Friend
Challenge to a Debate
Send a Message
9/2/2011 6:52:45 PM
Posted: 5 years ago
Let's see if we can get a thread on specifics in education to generate some useful discussion...

This thread is about how tests should be marked. In particular, I'd like to focus on multiple choice testing and answers where an incremental criteria isn't essential. By incremental criteria I mean how in an essay one's marks rise the more accurate they are. I'm referring to situations similar to maths where there is a correct answer and an incorrect answer.

The main reason the current scheme is flawed is that random guessing is rewarded. This leads to mark inflation in scores which results in tests which do not accurately gauge student understanding. It also promotes a mindset of guessing, which isn't positive.

However, it isn't so easy to improve on the current system. I had a debate where my old belief was crushed - http://www.debate.org... . Finicky mathematics and such ruined my old scheme.

I'd ask those truly interested in the subject to read this short document -http://www.caacentre.ac.uk... . While this is on a slightly different topic, it addresses many pique points on test marking.

My favourite system currently is Confidence Assessment (2.3 in the document) . What this system does is allow students to choose a level of certainty they have in their answer (1,2,3 for example) . If the student is correct they get the number of marks they put in for their confidence (ie. confidence level 1, only 1 mark, level 3, 3 marks) . However, there are stronger penalties for mistakes on high confidence levels (ie. mark deduction) . The reason for this is simple. Imagine you're a doctor and you were 'very confident' (let's say a level 3) of your prognosis and ended up being wrong and severely injuring your patient. Not good! However, if you had little confidence in your prognosis (ie. level 1) , you'd likely go get help or do something before attempting the operation.

The document explains the system better. However, what's key to note is that the current system rewards guessing on level 1 answers when instead one should in real life not go ahead and instead improve their chances.

As for the question of different marks for different questions, that's also easily solved.

My question to others is what their preferred test marking system is. I also ask about whether they believe confidence assessments are an improvement on the current system.

Finally, since I may be doing a debate on this subject at some point, those interested should feel free to make a point about debating me in this thread (assuming holes aren't found in my proposed idea) .
"Tis not in mortals to command success
But we"ll do more, Sempronius, we"ll deserve it
000ike
Posts: 11,196
Add as Friend
Challenge to a Debate
Send a Message
9/2/2011 7:02:56 PM
Posted: 5 years ago
When it comes to grading tests you cannot use theoretic logic, as it holds absolutely 0 value. You use the results in practice. Theoretic logic will tell you that multiple choice rewards guessing, results in practice will tell you that multiple choice is equally as hard due to both "the guess temptation factor" and the "misleading factor" which provides answers based on common errors in computation luring the test-taker to see the result as an option and mark it.

My Biology Class last year:
ALL tests are scantron multiple choice: class average = C or B

ALL quizzes are question and answer, no help and no options: class average = B, A
"A stupid despot may constrain his slaves with iron chains; but a true politician binds them even more strongly with the chain of their own ideas" - Michel Foucault
Logic_on_rails
Posts: 2,445
Add as Friend
Challenge to a Debate
Send a Message
9/2/2011 7:29:53 PM
Posted: 5 years ago
At 9/2/2011 7:02:56 PM, 000ike wrote:
When it comes to grading tests you cannot use theoretic logic, as it holds absolutely 0 value. You use the results in practice. Theoretic logic will tell you that multiple choice rewards guessing, results in practice will tell you that multiple choice is equally as hard due to both "the guess temptation factor" and the "misleading factor" which provides answers based on common errors in computation luring the test-taker to see the result as an option and mark it.

My Biology Class last year:
ALL tests are scantron multiple choice: class average = C or B

ALL quizzes are question and answer, no help and no options: class average = B, A

I don't know exactly what a 'scantron' test is, but I'll assume it's a typical multiple choice test.

I understand your viewpoint, but disagree. Firstly, variety in test difficulty is key. Secondly, psychology is key. I know many people rush the multiple choice sections of test papers and then spend a lot of time on question and answer types questions due to the perception that multiple choice is easier. This results in lower marks in the multiple choice sections due to not spending adequate type on the questions. This surfaces also in purely multiple choice tests as candidates may finish early, not making optimal use of the allotted time.

Furthermore, multiple choice is designed, like you mention, to have various tricks to compensate for guessing. Ie. Misleading answers are put in. These tricks lower scores. That is a partial explanation of the lower multiple choice results.

What this doesn't mean is that confidence assessment shouldn't be used, and I believe it still is optimal for multiple choice testing. On a side note, I don't really support multiple choice testing and prefer more question and answer styled questions as I find they test higher order thinking more.
"Tis not in mortals to command success
But we"ll do more, Sempronius, we"ll deserve it
Man-is-good
Posts: 6,871
Add as Friend
Challenge to a Debate
Send a Message
9/2/2011 8:44:08 PM
Posted: 5 years ago
I would not necessarily agree with the Confidence Assessment system. According to the link that you provided, the system was tested on students in a medical university, and was descried as "requir[ing] candidates to assign a confidence level to each of their selections to reflect their degree of certainty." It might seem beneficial, though I believe the emphasis on confidence, and its correlation to the correctness of the answer to be superfluous...Tests are used, to assess "a test-taker's knowledge, skill, aptitude, physical fitness, or classification in many other topics (e.g., beliefs)" http://en.wikipedia.org... confidence, diffidence, and so on are not in those fields, and are not necessarily entwined with 'knowledge, skill, aptitude" and so on.

This leaves the question as which how can we truly assess the intelligence and acuity of a person. In fact, my point is that we should change the structure and the formation of the entire test...in order "to objectively assess cognitive, analytical and other such skills in addition to straightforward factual knowledge." As your paper states, there are valid objections, including:

1. Guesswork, which--while resulting in a correct answer--is in no way an indication of one's intelligence or cognitive faculties
in particular...
"Homo sum, humani nihil a me alienum puto." --Terence

"I believe that the mind can be permanently profaned by the habit of attending to trivial things, so that all our thoughts shall be tinged with triviality."--Thoreau
Just1Voice
Posts: 155
Add as Friend
Challenge to a Debate
Send a Message
9/3/2011 12:02:55 AM
Posted: 5 years ago
Why use multiple choice questions at all?

Unless you are creating some sort of "choose your own adventure" test, the simplest solution to the problem you are pointing out is to simply stop using multiple choice questions in tests.