Home » Support » Bug Reporting » Flaw in the Rating System?
Search Search  
Show: Today's Messages    Show Polls    Message Navigator
Flaw in the Rating System? [message #1375] Sat, 20 September 2008 15:28 UTC Go to next message
avatar Whystler  is currently offline Whystler
Messages: 568
Registered: September 2008
Go to my shop
Senior Member

I think there's a flaw in the rating system, especially when there is the incentive to "game it" during contests.

When I say "gaming the system" I mean the ability to win or benefit from a system by using it in a way that it is not intended.

For example, there are plenty of good products out there that deserve a 4 or a 5 rating. And there are a few duds that are either for testing or by people who are learning that deserve a 2 or a 3. With the stakes of the contest so high (getting your expensive object printed, $250 in printing coupons, exposure of your work at conferences...) the pressure is on contestants to review the competing products lower than you normally would.

I like the rating system in general. But I think maybe a different voting system should be in place for the contests. I have always like voting systems where you can choose more than one (ie. your top 3 or top 10 favourite items), and then the finalists are voted upon in the end.

What do you think?

-Whystler


Check out my website: http://tshawnjohnson.wordpress.com/
Re: Flaw in the Rating System? [message #1384 is a reply to message #1375 ] Mon, 22 September 2008 14:32 UTC Go to previous messageGo to next message
avatar joris  is currently offline joris
Messages: 426
Registered: May 2008
Go to all my models
Senior Member
Whystler,

Currently we are using the regular rating system and trusting people.

I do agree that we could come up with systems that are more airtight.

What, we would like to do most is encourage enough people to vote so that any person trying to game the system does not exert enough influence to change a rating of a model.

We would also really like to keep on assuming that everyone will keep their ratings as honest as possible. Is that naive?

I will check and run some numbers of people's average ratings and things to see if there is a noticeable difference in the ratings of contest models.
Re: Flaw in the Rating System? [message #1385 is a reply to message #1375 ] Mon, 22 September 2008 16:05 UTC Go to previous messageGo to next message
avatar Whystler  is currently offline Whystler
Messages: 568
Registered: September 2008
Go to my shop
Senior Member
I guess my experience in IMVU is coming through here. Gaming the system in IMVU is rampant. I've been involved in testing that system on various levels, and I know that before a feature is even released, there is much discussion about how the feature can be gamed (so that people can get more credits/money).

Perhaps if you have a smaller testing base of users, gaming the system is not an issue here. It will be an issue in the future though, when you have a larger base of users if you are running contests.

Everyone does seem to be pretty supportive and honest here though at the moment Smile

-Whystler


Check out my website: http://tshawnjohnson.wordpress.com/
Re: Flaw in the Rating System? [message #1389 is a reply to message #1375 ] Mon, 22 September 2008 18:31 UTC Go to previous messageGo to next message
avatar Christel  is currently offline Christel
Messages: 23
Registered: August 2008
Go to all my models
Junior Member
Ideally, contest submissions should be anonymous. Another solution would be to make ratings NOT anonymous.
But I agree with Joris that for now it's all just for fun and in good spirit.

Personally I didn't manage to come up with anything as magnificent as I'd hoped for - due to the usual lack of time and talent Wink - and I have no problems giving high ratings to other entries. I do tend to go for aesthetics more than functionalism though. Maybe that's not entirely in line with the purpose of the contest Smile
Re: Flaw in the Rating System? [message #1391 is a reply to message #1389 ] Mon, 22 September 2008 19:22 UTC Go to previous message
avatar Whystler  is currently offline Whystler
Messages: 568
Registered: September 2008
Go to my shop
Senior Member
Christel wrote on Mon, 22 September 2008 14:31

Another solution would be to make ratings NOT anonymous.


I like this idea * A LOT*. It really helps the transparency of rating.

I have also seen rating systems where you can add an accompanying short message. This is good for the person whose item is rated because then, rating becomes a learning experience.

-Whystler


Check out my website: http://tshawnjohnson.wordpress.com/

 
   
Previous Topic:Selling to others
Next Topic:tutorials