| Flaw in the Rating System? [message #1375] Sat, 20 September 2008 15:28 UTC |
 |
|
I think there's a flaw in the rating system, especially when there is the incentive to "game it" during contests.
When I say "gaming the system" I mean the ability to win or benefit from a system by using it in a way that it is not intended.
For example, there are plenty of good products out there that deserve a 4 or a 5 rating. And there are a few duds that are either for testing or by people who are learning that deserve a 2 or a 3. With the stakes of the contest so high (getting your expensive object printed, $250 in printing coupons, exposure of your work at conferences...) the pressure is on contestants to review the competing products lower than you normally would.
I like the rating system in general. But I think maybe a different voting system should be in place for the contests. I have always like voting systems where you can choose more than one (ie. your top 3 or top 10 favourite items), and then the finalists are voted upon in the end.
What do you think?
-Whystler
Check out my website: http://tshawnjohnson.wordpress.com/
|
|
|
| Re: Flaw in the Rating System? [message #1384 is a reply to message #1375 ] Mon, 22 September 2008 14:32 UTC |
 |
|
Whystler,
Currently we are using the regular rating system and trusting people.
I do agree that we could come up with systems that are more airtight.
What, we would like to do most is encourage enough people to vote so that any person trying to game the system does not exert enough influence to change a rating of a model.
We would also really like to keep on assuming that everyone will keep their ratings as honest as possible. Is that naive?
I will check and run some numbers of people's average ratings and things to see if there is a noticeable difference in the ratings of contest models.
|
|
|
|
| Re: Flaw in the Rating System? [message #1389 is a reply to message #1375 ] Mon, 22 September 2008 18:31 UTC |
|
|
Ideally, contest submissions should be anonymous. Another solution would be to make ratings NOT anonymous.
But I agree with Joris that for now it's all just for fun and in good spirit.
Personally I didn't manage to come up with anything as magnificent as I'd hoped for - due to the usual lack of time and talent  - and I have no problems giving high ratings to other entries. I do tend to go for aesthetics more than functionalism though. Maybe that's not entirely in line with the purpose of the contest 
|
|
|
| Re: Flaw in the Rating System? [message #1391 is a reply to message #1389 ] Mon, 22 September 2008 19:22 UTC |
|
|
| Christel wrote on Mon, 22 September 2008 14:31 |
Another solution would be to make ratings NOT anonymous.
|
I like this idea * A LOT*. It really helps the transparency of rating.
I have also seen rating systems where you can add an accompanying short message. This is good for the person whose item is rated because then, rating becomes a learning experience.
-Whystler
Check out my website: http://tshawnjohnson.wordpress.com/
|
|
|
Search
