rating system, again

I saw that the rating system's been discussed various times, but I don't think this angle was.

Thing is: at the moment, if your product has 10 ratings with an average of 4 stars, it is automatically below any product which has 1 rating of 5 stars (from, like, yo momma).

Now, almost any alternative I think of has some drawback, but I'm sure lot's of poeple smarter than me have thought about this - must be some fair solutions out there?

Cheers, M.

Oh, and once again - posting from my Mac/safari never works - can someone please solve that bug?

I've been trying to come up with something, but indeed, nothing seems to be better overall. Perhaps splitting stars in half (only on the final scoring) would diversify a little.

Luckily most ratings level out eventually. As long as enough people vote

I think it would be nice to increase the exposure of the four star graveyard (sooner or later most five-star models are voted down and end up there).
Perhaps a separate page/gallery link for each rating?
Or something like top rated models last week / last month / last year /all time.

4 star graveyard - that's a good word...Some options:

- Add up all stars, and that's your rating. So 5x4 stars is worth more then 2x5 stars.
- something like on digg
- 5=+2, 4=+1, 3=0, 2=-1, 1+-2; and then add everything up and that's your rating (I think the result is more or less the same as option1)
- you need a minimum of 4 votes to show up in the rating gallery

Well, something like that...

Counting stars .. hmmm
Pro : You wouldn't be able to vote objects down anymore, you would be forced to rate other objects up.
Con : I think it would make it very difficult to get objects anywhere near visibility in the top-rating items. One would need 50 stars at least.. And once they are there, they are almost impossible to ever get them out of there.. Unless it is split into all-time and latest tops.

Digg.. don't know, what do they do?

The third option indeed seems the same as the first, but unrated items would remain in the middle somewhere, whereas in the first, they'd be at the bottom.

I'm not sure, those options seem less tamper-proof
The current system at least has the power of mass-correction.
(if the masses would get into motion at least )

Perhaps the rating-sort could use/show the real average ?
(5+5)/2=5
(5+5+4)/3=4.67
(5+4)/2=4.5
Etc..
That is, if that is not already how its done now

Or some other variation of score correction like you suggested.
Perhaps, statistical weighing or something, like removing extreme votes before averaging.

For the last one, how about:
A clean showroom/gallery with at least a few votes and an uploaded picture..

And a work in progress-gallery where anything goes ?

Just thinking out loud..

I think the rating system should ensure that all stars are treated equally. If I vote something down by giving 1 star, that star should be worth just as much as a 5 star. Without that rule, a system where the total rating is defined by the sum of the stars de-values negative opinions. Then again, one might argue that people are more likely to complain about a product than praise it.

Perhaps something to discuss is at what point is 4 stars worth more than 5 stars? For that matter, when is 1 star worth more (or less) than 5 stars? We can talk about the extremes (100 4-star ratings versus 1 5-star) but it's the boundaries in between the would really define that system don't you think?

I like the idea of weighting the ratings based on the total number of votes (either per rating category or overall).

Then again, maybe the concept of stars is a lacking solution and ratings like seller ratings on ebay might be better?

Another way could be the median: all votes are ordered by value, 1 to 5, and the central item is used as result if odd number of cast votes, or the average of the two central ones if even number of cast votes. 1 2 2 5 5 gives 2, and 1 2 3 5 gives 2.5.

This is the best suggestion I've heard yet! It doesn't solve every issue with the rating system, but I'd really like to see two sections, for "All-time top-rated models" and "Top-rated models from the last [some time period]". Don't get me wrong, I own (and totally love) Bathsheba's Gyroid, and everything Whystler designs is pure gold, but I'm a little tired of seeing them every time I open the model gallery.

We're working on something similar to better expose models to people. I agree that we need to overhaul the ratings system. We're looking at this too.

One option that, I think, hasn't been considered would make the system much more balanced. The problem is that those items that have been there longest get the greatest advantage. How about a system that factors in the total views that the item has gotten.
Under this system an item with 5/5 stars and 100 views would be equal to an item with 1/5 stars and 20 views. this levels the playing field for newcomers while still respecting the wonderful products that have been here awhile.

Equal!?? Yikes, that is perhaps too drastic a skewing. Items that have been listed longer have more ratings, but also statistically should have a more realistic rating. There are a huge number of items (over half I would guess) with no ratings at all. You could also use days since published rather than views.

Most ratings are just people guessing based on a rendered image. If somebody actually BUYS a model, I would consider their rating as 10 times more relevant than someone who has not bought it. I hope that they could modify the rating system to include this.

Gibell, I think your closing remark is spot on; surely BUYING a model i the highest rating one can give.

I still preferred the old system of sorting uploaded models by date as the default. It meant that as soon as I opened the gallery, I saw the latest work. This is of much greater interest to me than how other people have rated models.

As for the notion that having high-rated models up-front will "look cool" and "get as many people enthusiastic about Shapeways as possible"(Pete, Sat, 12 December 2009), I have 2 comments:
1/ I believe most members of Shapeways are here because this is the way their dreams can be realized, (not because they were impressed by the AMAZING work that the top-raters share with us)
2/ There may be some newcomers to 3-D modelling who feel a bit intimidated by the wonderful artistry we now see whenever we open the gallery

Although it's been a while since I was there, I think perhaps a place like Flickr could be inspiring as well; they have the same issue where they need to offer various ways of showing their content; random, time-based, rating-based, theme based.

The thing that is perceived as unfair with the current rating system is that 4 5 5 5 5 5 ... 5 gives 4 stars average (4.999... is rounded to 4).
This means that only one 4 can defeat any number of 5s.
That's what you call the 4-star graveyard (or we could say the 4-star punishment).

The median caculation proposed by Stannum would be an elegant way to get rid of the 4-star graveyard: the 4s and the 5s in this sytem have the same strengh.
- 4 4 5 5 would be 4.5 (neither the 4s or the 5s win), which should be the same result as the current average calculation event if half star are not displayed
- 4 4 4 5 5 would be 4 (same as current average 4.4 that is rounded to 4)
- 4 4 5 5 5 would be 5, and once again this is the major improvement: two 4s cannot beat three 5s

I give 5 stars to the median!

Well - I'm not sure it works that way; my guess would be that an average of lower than 4.5 stars brings you in the 4 star category, which means page 56 in the gallery or sommin'

My point is that in my opinion, getting 10 x 4 stars (average) is worth more than getting 2 x 5 stars.
And yep, actually bought products should probably count for 10 stars...

Hey but I understand Joris is on the case so we'll see something good soon...!

Sorry Michiel, but I am not convinced.

If I follow your thinking, a model where 11 people took the time to rate it with 1 star would be sorted before a model that would have receive 5 stars twice?

For me the problem is more that a model with already nine times 5 stars that is then given once 4 stars is rated less than a model that was given only twice 5 stars: the strength of the 4-star mark is bigger than the strength of the seven 5-star marks.

Keep in mind that the 10 times 4 stars you are mentionning are perhaps actually 9 times 5 stars and once 4 stars.

And if you are not convinced it works this way (4.99 is rounded to 4), take a model that has 5 stars with tens of votes and give it 4 stars, and see what happens some hours later (I won't do it, that would be sabotage ).

EDIT: actually, after thinking about it again, I am not 100% sure of my last sentence. All what I know is that one model can go from 5 stars and 9 votes to 4 stars and 10 votes. But if you are right, it could be that it had 5 times 5 stars, 4 times 4 stars (average 4.55 displayed as 5) and someone gave it a 3-star mark (average 4.4 displayed as 4).
If it is the way it works, then it is not as unfair as I thought.
Shapeways team-members, can you please give us the secret of your average rounding?

Magic, I to think you might be wrong on this.
If a model has 5 stars (based on an average score of 4.6)
you can probably vote it down to a 4 star with one vote.

But I'm quite sure I saw models going from 4 to 5 stars..
Which would be impossible in the method you describe, which would apparently never allow a model to go up ?


Edit : never mind part above, the hivemind already fixed it


To test it I gave 5 stars (positive sabotage ? ) to
https://www.shapeways.com/model/89089/haiti_size_13.html
(previously had one vote of 4 stars)
Et voila : it now has 5 stars

And indeed, curious as to what Shapeways has in mind for changes.
But it never hurts to add some ideas

OK, I agree with you Virtox, I have probably be mislead by what I saw (bad interpretation :blush: ).
Nevertheless, I have a last proposal (and then I stop on the rating system discussion - until I see Shapeways new solution, of course ).

Have you seen, in the Olympic games, how the countries are ordered? First, your order by comparing the number of gold medals. For all those which have the same number of gold medals, you order this subset by comparing the number of silver medals and then, once again, in case of equality you check the bronze medals.

What about doing the same thing for the stars? The first models would be those having the most 5-star marks, independently of the other marks. For those models which have the same number of 5-star marks, they are ordered between them by comparing the numer of 4-star marks etc.

In this way, getting 5-stars twice and then 4-stars is more than getting 5-stars only once, which is fair according to me (but I can understand that someone could think different).

I concur !

There's something with that hive-mind, because I was pondering the same thing
That is to say, I only got as far as something like counting grades.

But with the subset/sorting you propose, it could make for a fair and transparent (supplemental) sorting system.

Yes - that olympic thing sounds about right to me!

Hey guys,

In the mean time, a lot of great models have been "banned" to the four-star-graveyard (again). While some models only got a glimpse of all the fame..
For example the pencil thingy by Michiel, I've never seen an object go down that fast, but one 5-star vote fixed that easily.

So I make a regular stop there, and give out 5-stars to any models I really like. I invite you to join me, it starts somewhere around here : https://www.shapeways.com/gallery?page=29&sort=rating&amp ;amp ;r=0

And yes, technically this is not in your own best interest, as you might vote other people's objects up beyond your own.. BIG DEAL!
I just think it is good for the community as a whole, then again, I'm probably a hopeless hippie

Stijn

Ps. Ever notice how objects featured by shapeways (frontpage for example) soon have only four-stars left ? Seems like kindergarten all over again...