Total Posts:14|Showing Posts:1-14
Jump to topic:

Potential Flaws of the Proposed Voting System

NiqashMotawadi3
Posts: 1,895
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 3:55:52 PM
Posted: 2 years ago
Three preemptions to make before I actually begin...

(1) I do support the new voting system, because it's certainly better than what we have now, and the "judge pool" is something I'm personally excited about.

(2) The official forum-post didn't go into technical details, and so I'm going to base this critical analysis on the brief information that was provided.

(3) I'm only concerned with figuring out the potential flaws and disadvantages, and not with the later phase which would weigh in the advantages and disadvantages for each feature.

1- Voting Leaderboard Update

With adding a new judge voting option, we are creating a "leaderboard" that would list out the members who have judged the most debates and the percentage of times they have selected the winner of the debate. This would provide newer members an opportunity to figure out who experienced judges may be to nominate."

I suppose that the "percentage of times they have selected the winner" would cause some problems. For instance:

In debates where one participant is winning by a considerable number of points, the supporters of his opponent can simply change their votes and RFDs afterwards . Not to mention other voters who might genuinely have changes of mind but would be discouraged from changing their votes (as they wouldn't want to be accused of abusing the system, although they might have read the debate well the first time, but then got an epiphany after they voted which changed their whole perspective). A possible solution for that would be to report such "weird changes" in RFDs, but someone can have such changes of mind genuinely and be wrongly accused of abusing the system, while another voter could recognize an accelerating difference in the points, and decide to change his vote accordingly in a deceitful manner where he is given the benefit of the doubt. Not to mention that this would create more cases of reporting votes, when the current moderators are already overwhelmed with reports on votebombs and misconduct.

In debates where one participant is unethically winning because of in-friend voting, say, 22 to 9 points, a hypothetical voter would be tempted (more than he could be in the current situation) to vote in favor of the participant with the highest number of points to stay on the safe side. In other words, this feature would discourage Iconoclasts, and encourage hive-mentalists, which is not always a good thing.

2- Choosing Judges.

The new voting system allows the initiator of the debate to choose the judges(only voters) they want on their debate.

The opponent is not provided with sufficient criteria to know whether those judges are biased towards the initiator or not, but instead he is only given that judge's "percentage of choosing the winner" and the "number of debates that judge has voted on", although I can easily imagine a self-acclaimed voter who mostly votes on noob-sniping debates and maintains a 99% "percentage of choosing the winner" followed with "voted on 100 debates," unfairly voting for his friends who would insist on him being a judge on their debates, because they know he is biased for them although his overall record seems to suggest otherwise.
...

Those are the tip of the iceberg. It's predictable that many new issues would arise when the technical aspects are implemented, and probably some of the flaws I suggested could be solved easily when the functions are actually punched into the machine. But those were my two cents based on the little knowledge I have.
Intrepid
Posts: 372
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 3:59:01 PM
Posted: 2 years ago
I actually completely agree with this. I think DDO Admins need to address these problems before implementing the new system.
Ragnar
Posts: 1,658
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 4:30:46 PM
Posted: 2 years ago
At 4/7/2014 3:55:52 PM, NiqashMotawadi3 wrote:
1- Voting Leaderboard Update

With adding a new judge voting option, we are creating a "leaderboard" that would list out the members who have judged the most debates and the percentage of times they have selected the winner of the debate. This would provide newer members an opportunity to figure out who experienced judges may be to nominate."

I intend on distrusting anyone with 90% or more in either direction.

Someone choosing all winners could imply either voting only on simple debates, or otherwise going with the bandwagon. A danger to it in increased Fluff Votes ("strategic" use of the hands), as fluffers may fear their leaderboard status and assign extra unearned points.
Someone choosing all losers could imply zealous bias, similar to what we've seen from certain Muslim voters.
Unofficial DDO Guide: http://goo.gl...
(It's probably the best help resource here, other than talking to people...)

Voting Standards: https://goo.gl...

And please disable Smart-Quotes: https://goo.gl...
NiqashMotawadi3
Posts: 1,895
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 4:56:40 PM
Posted: 2 years ago
At 4/7/2014 4:30:46 PM, Ragnar wrote:
At 4/7/2014 3:55:52 PM, NiqashMotawadi3 wrote:
1- Voting Leaderboard Update

With adding a new judge voting option, we are creating a "leaderboard" that would list out the members who have judged the most debates and the percentage of times they have selected the winner of the debate. This would provide newer members an opportunity to figure out who experienced judges may be to nominate."

I intend on distrusting anyone with 90% or more in either direction.

Someone choosing all winners could imply either voting only on simple debates, or otherwise going with the bandwagon. A danger to it in increased Fluff Votes ("strategic" use of the hands), as fluffers may fear their leaderboard status and assign extra unearned points.
Someone choosing all losers could imply zealous bias, similar to what we've seen from certain Muslim voters.

I don't think this would occur to many people, though. I'm now thinking of doing case-studies of you and Raisor's votes, as I think you have the best votes generally, in order to see the percentage of winner selection. I'm not sure if it is 90%, though.
Geogeer
Posts: 4,296
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 5:10:53 PM
Posted: 2 years ago
At 4/7/2014 4:30:46 PM, Ragnar wrote:

I intend on distrusting anyone with 90% or more in either direction.

Hmm... I tend to try to focus on debates that nobody else has voted on. That way more people get their debates voted on. Most are fluff, some require some chewing.
thett3
Posts: 14,381
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 5:16:37 PM
Posted: 2 years ago
Yeah, the aggregatation of the number of times someone has selected the winner raised a few red flags with me too. It will be an interesting statistic for sure, but it could easily create a perverse incentive where people vote for who is currently winning, not who they think won, to keep their ratio up. This probably won't happen as I don't think many people will consider it important but it seems kind of like an unnecessary risk.

I agree on the selecting judges things. I think the status quo works fine, if the debaters want it to be decided by certain judges than the mods will enforce that. It seems like it will happen in at least a few instances that an unscrupulous member will select all their friends who will vote them up as the judges.

I think allowing up/down voting by far outweighs all of that though
DDO Vice President

#StandwithBossy

#UnbanTheMadman

#BetOnThett

"Don't quote me, ever." -Max

"My name is max. I'm not a big fan of slacks"- Max rapping

"Walmart should have the opportunity to bribe a politician to it's agenda" -Max

"Thett, you're really good at convincing people you're a decent person"-tulle

"You fit the character of Regina George quite nicely"- Sam

: At 11/12/2016 11:49:40 PM, Raisor wrote:
: thett was right
bsh1
Posts: 27,504
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 5:24:12 PM
Posted: 2 years ago
Valid points.

I agree that being a dissenting vote does not mean that you are a bad judge. I've been in rounds where this was the case. For example, at CFL States, I was judging the first round of Policy on a panel of 3 judges. The other two judges had never judge a debate round in their life, nor had they encountered spreading--they were brought just so that the respective schools could fill their quotas. They ened up voting on two asinine things, dropping the 2nd seed against the 6th seed, when the 2nd seed clearly won the round. The 6th seed concede and dropped virtually everything on the flow--but when you fail to take notes, and are so gob-smacked at the speed of the round that you barely paid attention, you'd miss this. Not to disparage inexperienced judges, but they really should've been confined to PF and Speech, not LD and Policy.

Ultimately, the point is that out of three votes, I would (though of course I'm biased in favor of myself) posit that 2 of those votes were bad. Being a judge is not reflected by how accurately you predict the winner, but by how well you understood the round.
Live Long and Prosper

I'm a Bish.


"Twilight isn't just about obtuse metaphors between cannibalism and premarital sex, it also teaches us the futility of hope." - Raisor

"[Bsh1] is the Guinan of DDO." - ButterCatX

Follow the DDOlympics
: http://www.debate.org...

Open Debate Topics Project: http://www.debate.org...
Raisor
Posts: 4,468
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 6:59:39 PM
Posted: 2 years ago
At 4/7/2014 5:24:12 PM, bsh1 wrote:
Valid points.

I agree that being a dissenting vote does not mean that you are a bad judge. I've been in rounds where this was the case. For example, at CFL States, I was judging the first round of Policy on a panel of 3 judges. The other two judges had never judge a debate round in their life, nor had they encountered spreading--they were brought just so that the respective schools could fill their quotas. They ened up voting on two asinine things, dropping the 2nd seed against the 6th seed, when the 2nd seed clearly won the round. The 6th seed concede and dropped virtually everything on the flow--but when you fail to take notes, and are so gob-smacked at the speed of the round that you barely paid attention, you'd miss this. Not to disparage inexperienced judges, but they really should've been confined to PF and Speech, not LD and Policy.

Ultimately, the point is that out of three votes, I would (though of course I'm biased in favor of myself) posit that 2 of those votes were bad. Being a judge is not reflected by how accurately you predict the winner, but by how well you understood the round.

THATS WHY YOU ASK FOR JUDGE PREFS.

Sorry but seriously if you don't ask for or follow judge prefs on speed or you see your judges not flowing and do not adapt your style you deserve to lose.
Ragnar
Posts: 1,658
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 7:17:00 PM
Posted: 2 years ago
At 4/7/2014 4:56:40 PM, NiqashMotawadi3 wrote:
At 4/7/2014 4:30:46 PM, Ragnar wrote:
I intend on distrusting anyone with 90% or more in either direction.

I don't think this would occur to many people, though. I'm now thinking of doing case-studies of you and Raisor's votes, as I think you have the best votes generally, in order to see the percentage of winner selection. I'm not sure if it is 90%, though.

Since a massive chunk of my votes are null (some incredibly detailed, but mostly just cleaning up the unvoted debates list), I wonder how the system will handle that? But yeah, there's a decent chance I'll end up distrusting myself if I hold to the stated 90% standard.

A really interesting statistic on people, would be how often they vote directly against the "agreed with before/after" bias. Sadly as it's an underutilized feature, there would be little data to be gathered right now.
Unofficial DDO Guide: http://goo.gl...
(It's probably the best help resource here, other than talking to people...)

Voting Standards: https://goo.gl...

And please disable Smart-Quotes: https://goo.gl...
ClassicRobert
Posts: 2,487
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 7:25:40 PM
Posted: 2 years ago
At 4/7/2014 6:59:39 PM, Raisor wrote:
At 4/7/2014 5:24:12 PM, bsh1 wrote:
Valid points.

I agree that being a dissenting vote does not mean that you are a bad judge. I've been in rounds where this was the case. For example, at CFL States, I was judging the first round of Policy on a panel of 3 judges. The other two judges had never judge a debate round in their life, nor had they encountered spreading--they were brought just so that the respective schools could fill their quotas. They ened up voting on two asinine things, dropping the 2nd seed against the 6th seed, when the 2nd seed clearly won the round. The 6th seed concede and dropped virtually everything on the flow--but when you fail to take notes, and are so gob-smacked at the speed of the round that you barely paid attention, you'd miss this. Not to disparage inexperienced judges, but they really should've been confined to PF and Speech, not LD and Policy.

Ultimately, the point is that out of three votes, I would (though of course I'm biased in favor of myself) posit that 2 of those votes were bad. Being a judge is not reflected by how accurately you predict the winner, but by how well you understood the round.

THATS WHY YOU ASK FOR JUDGE PREFS.

Sorry but seriously if you don't ask for or follow judge prefs on speed or you see your judges not flowing and do not adapt your style you deserve to lose.

Agreed. Half of debating is being able to adapt to your judges.
Debate me: Economic decision theory should be adjusted to include higher-order preferences for non-normative purposes http://www.debate.org...

Do you really believe that? Or not? If you believe it, you should man up and defend it in a debate. -RoyLatham

My Pet Fish is such a Douche- NiamC

It's an app to meet friends and stuff, sort of like an adult club penguin- Thett3, describing Tinder
whiteflame
Posts: 1,378
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 7:55:01 PM
Posted: 2 years ago
I pretty thoroughly agree with this. These were the two things I found most problematic, especially the first. While I'm not a fan of setting a list of judges (except, perhaps, for tournaments and major debates), I'm more wary of the choice to place people on the voting leaderboard simply for voting with the winning debater. I think squirreling is often justified, and can often require more of a judge explanation. It seems to me that quality of votes, utilizing a rating system for each vote, would probably do a better job of assessing them than whether they voted with the winning team.
Enji
Posts: 1,022
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 8:23:45 PM
Posted: 2 years ago
A fairly simple solution to the first problem would be to hide the current winner of the debate from everyone except the debaters until the voting period closes. While the voting period is in progress anyone other than the debaters would see the number of votes cast, but not the votes themselves or who cast them. This will prevent voting for the current winner to boost your "correct" votes statistic and will keep judges' reasons for decision focussed on the debate content, uninfluenced by previous voters opinions.
bsh1
Posts: 27,504
Add as Friend
Challenge to a Debate
Send a Message
4/7/2014 9:19:03 PM
Posted: 2 years ago
At 4/7/2014 6:59:39 PM, Raisor wrote:
At 4/7/2014 5:24:12 PM, bsh1 wrote:
Valid points.

I agree that being a dissenting vote does not mean that you are a bad judge. I've been in rounds where this was the case. For example, at CFL States, I was judging the first round of Policy on a panel of 3 judges. The other two judges had never judge a debate round in their life, nor had they encountered spreading--they were brought just so that the respective schools could fill their quotas. They ened up voting on two asinine things, dropping the 2nd seed against the 6th seed, when the 2nd seed clearly won the round. The 6th seed concede and dropped virtually everything on the flow--but when you fail to take notes, and are so gob-smacked at the speed of the round that you barely paid attention, you'd miss this. Not to disparage inexperienced judges, but they really should've been confined to PF and Speech, not LD and Policy.

Ultimately, the point is that out of three votes, I would (though of course I'm biased in favor of myself) posit that 2 of those votes were bad. Being a judge is not reflected by how accurately you predict the winner, but by how well you understood the round.

THATS WHY YOU ASK FOR JUDGE PREFS.

Sorry but seriously if you don't ask for or follow judge prefs on speed or you see your judges not flowing and do not adapt your style you deserve to lose.

The way policy is structured nowadays makes it virtually impossible not to speed.

Judges who are put in policy should be able to handle speed--if they can't, then tab shouldn't put them in that event. Tab should do some prescreening, seriously...Yes, all debaters should ask for preferences, but debaters can only be asked to adapt to a certain degree. You could no more tell most policy debaters to not spread as you could turn lead into gold.
Live Long and Prosper

I'm a Bish.


"Twilight isn't just about obtuse metaphors between cannibalism and premarital sex, it also teaches us the futility of hope." - Raisor

"[Bsh1] is the Guinan of DDO." - ButterCatX

Follow the DDOlympics
: http://www.debate.org...

Open Debate Topics Project: http://www.debate.org...