Rating by popularity

     

Last week we introduced a new method to order the products in our galleries. We calculate each day the popularity of a product. In this blog post I will explain how the popularity algorithm works and the reasoning behind this decision. Up until this week our default sorting order in our galleries was based on rating. This led to quite static galleries since  the number of users which rate a product is quite low (relative to our visitor counts). There was also the “4-star graveyard” The perception that, a lot of products have a 4 star rating and it is almost impossible to get to 5 stars anymore. Models with 1 vote and 5 stars were ranked higher than a product with 28 votes and 4 stars. How do you compare? For these reasons we implemented a popularity algorithm and changed the default sorting of all our galleries to be based on popularity. The popularity is calculated based on both attributes of a products and the activity around a product. We use the following activity indicators and attributes:   

1. Number of times a product is bought (not taking into account the quantity of the order)   

2. Sum of all ratings   

3. Number of times a product is viewed   

4. Number of comments left on a product   

5. Bonus for co creator products   

6. Bonus for products for sale   

7. Shapeways editor’s pick (only to be used in specific cases – no worries)                

We calculate the popularity of a product within three timescales; all-time, month and week. The month and week timescales only look at the activity within that timeframe. I cannot disclose how we exactly calculate the popularity to avoid any abuse of the system. We did some extensive testing with the algorithm the last couple of months and we feel that is a major improvement in the way the products are sorted in our galleries.   

The major drawback is that is now more or less a black box system for you as a user. It is like Google PageRank. A valid question is how do I get on top of the gallery? My hope is that blog post will help you understand and give you pointers on how to achieve that. Make sure there is lots of activity around your products. Blog about them (for views), encourage others to rate and comment on your models and above all sell your goods.    

And to give you a hint. We favor number of times a product is bought or viewed higher than the sum of all ratings. Is it perfect? No not yet! But we will be working actively to improve the algorithm to provide better and more useful galleries.You can the new system in action here.

14 comments

  1. Felix

    I’m already seeing a flaw in your system. When a model is featured on the Shapeways frontpage, it gets a lot of votes and a lot of views, maybe some comments. Automatically, his ranking will be much better compared to another model uploaded at the same time.

    1. Robert Schouwenburg

      Yes, featured products on our homepage is very effective for your popularity. We also feature products in our blog posts or newsletters and this also has an impact. But so is putting your products on Facebook or Twitter. That really works. Take a look at Duann’s blog post “StumbleUpon Digg and Tweet your Designs” on this very subject!

      You can suggest products (even your own) to be featured to us via the forums.

  2. Mitchell Jetten

    yeah to be honest i don’t really like this system,

    i’ve sold a lot of models,, and have very happy customers,, but not a lot of them have voted,, so i’ve just got 4votes max for a model even if i’ve sold a lot and have happy customers!

    1. Robert Schouwenburg

      Actually buys are more important than ratings. For you this would mean that your products should be higher in the gallery than similar rated products (and sold less).

  3. George

    Why not take into account the quantity of the order? If someone orders two of a model seems to me it should count as twice two individual orders. They must really like it.

    1. Robert Schouwenburg

      Good question. We looked at this and it did not lead to usable results. A lot of products which are sold in quantities are products similar to screws, nuts and bolts.

    2. George Bell

      Maybe for items over $25 or something? These wouldn’t be nuts and bolts. Someone ordered 4 of one item of mine that are over $50 each!

  4. George Bell

    Does it count if the shop owner orders a model? I could go either way on this one …

    1. Robert Schouwenburg

      Yes, we count these in.

  5. Glenn Slingsby

    I much prefer this system. It is way better than the useless ratings system in place before. At least this takes into account many variables which has to be better than just using one…

    Glenn

  6. Jake Drews

    I like it (so far). Im sure overtime, it will level out and become a smoother system.

    Keep up the good work!

  7. Bathsheba Grossman

    I think this is an excellent feature. I agree with Glenn, it’s way better than sorting by raw ratings. Good for you!

    Important: This is the first way of sorting the gallery that produces shoppable results. You need customers to be looking at a shoppable list. Most of the time.

    I think you should get rid of the “editor’s pick” criterion. It’s better to be Google than Yahoo.
    I’d love to see a “Staff Picks” list, but it should be separate from this feature.

    Is there anything to compensate for bias toward older models, which will simply have higher absolute numbers for views, sales etc? I wonder if I’m overpopulating the top of the list because I joined early. E.g. a model with 10 sales per week which has been up for 20 weeks will outperform a newer one with 50 sales per week, for the first month of the newer one’s life. This problem solves itself as time passes, but a month is a long time.

    Maybe there should be a “trending” list as well, which is weighted more heavily by sales-per-week than by sales-ever?

    I don’t think you can have too many lists, so long as they’re full of models that people want to buy. (You can definitely have too many lists that don’t make sense. Come on guys, why would I ever sort by product name? If I know the name of a product I can search for it by name, but I’m never going to click through 97 pages to look up something that begins with J.)

    Carry on!

  8. Michiel Cornelissen

    Nice work! My comments:
    - Why the co-creator bias? If people like that as a feature, it will show up in their ratings, won’t it?
    - agree with Bathseba’s comment on the editor’s pick, I think.
    - Not so sure if the amount of comments are that relevant.
    - Instead of a bonus for products for sale, I would by default only show products for sale, and make it a checkbox like it was.
    - I still sort of would like the option to simply show a list of most views, as an option.
    - Maybe you should consider allowing any visitor to rate, not just people with an account. At the moment, most people with account are probably designers and modelers, and that could have a strange bias; such as, nice rendering, or wow, great modelling, or something – things that actual customer ‘in the wild’ would not care much about. What would be against letting any visitor rate?

  9. Gillian Posey

    I hope the number of comments excludes comments by the designer of the item? Many designers put the description of their item in the comments rather than in the “Description” box, and comments often get added more than once because of problems with the interface.

Comments are closed.