Even aside from the problems of reviewers who have unrealistic standards, who haven't read the book, or who hate the author - Goodreads would still have a hard problem.
Very interesting article. I very much agree that just looking at average number of stars for any product is pretty worthless. I always look at the written review that goes with the star rating, for anything I care about (and I won't bother looking at reviews at all if I don't much care about it). This goes for product reviews on Yelp and Google as much as for books. If there is no written review, I weight the star rating at zero. This is especially important for bad reviews. I've often seen very bad reviews for things I care nothing about. I would therefore be inoculated against review bombing from the books you indicate above, because the reasons for those bad reviews are nuts, IMO. Of course it is still annoying, because I need to read all those dumb reviews before I know to ignore them.
I don't think it works for me to judge reviews based whether I agreed with the reviewer on other books. There are just too many different parameters that are important to me for this to work, especially for fiction. In my experience, people that like a book I like also like books I abhor. It just doesn't work. In practice, I buy non-fiction based on reviews I read in Amazon, but buy fiction just based on liking the author's previous books and a short description of the book by the publisher.
Yes - having to read all the reviews (of whatever quality) does take a lot more time! I usually end up reading just a small sampling of each star level, or a larger sampling if it's a product I care about more.
And that's another factor I forgot to mention - liking the author's other books is probably the best recommendation! (*Eifelheim* aside.)
Yeah I agree, number of stars is mostly worthless. What I really like is being able to read a sample from the book. Reading the first 30 pages gives me a pretty good idea if I'm going to like the authors writing style. In the past I'd do this by picking books of the shelf at the bookstore. On a side note, I've been reading everything written by T. Kingfisher, well except the horror stuff.
Weighting review-scores by agreement is not actually all that computationally costly, since there are various tricks that are used to reduce compute needed (using only a random sample of all reviews; binning readers into categories; relying on particularly significant differences, etc), and it can be done ahead-of-time. There are a lot of people working very hard to optimize recommendation engines (for ad serving especially) and I'm sure Amazon has a bunch of them on staff. It's most likely that the issue is just Goodreads never having been intended to be so large.
Implementing a good recommender system would solve some of the review bombing problem too, since those bombers would be significantly weighted downwards, I think.
I toyed around with a different solution to this problem - adding more metrics. What if users rated a book with 1-5 stars, but also on its pacing, tone, style, etc? Then you could look for books that have a high rating AND the other elements you enjoy.
For what it’s worth, I built a prototype of this. Haven’t fully decided if the idea is working or not, but here you go: https://inkhrt.com.
Yes, more metrics like that is a great idea! You've got a good point that even if they don't get into the full overall rating, they still give nonzero information.
Very interesting article. I very much agree that just looking at average number of stars for any product is pretty worthless. I always look at the written review that goes with the star rating, for anything I care about (and I won't bother looking at reviews at all if I don't much care about it). This goes for product reviews on Yelp and Google as much as for books. If there is no written review, I weight the star rating at zero. This is especially important for bad reviews. I've often seen very bad reviews for things I care nothing about. I would therefore be inoculated against review bombing from the books you indicate above, because the reasons for those bad reviews are nuts, IMO. Of course it is still annoying, because I need to read all those dumb reviews before I know to ignore them.
I don't think it works for me to judge reviews based whether I agreed with the reviewer on other books. There are just too many different parameters that are important to me for this to work, especially for fiction. In my experience, people that like a book I like also like books I abhor. It just doesn't work. In practice, I buy non-fiction based on reviews I read in Amazon, but buy fiction just based on liking the author's previous books and a short description of the book by the publisher.
Yes - having to read all the reviews (of whatever quality) does take a lot more time! I usually end up reading just a small sampling of each star level, or a larger sampling if it's a product I care about more.
And that's another factor I forgot to mention - liking the author's other books is probably the best recommendation! (*Eifelheim* aside.)
Yeah if there are like 100 written reviews I won't read them all, but just a sample. But if say 20 I will.
Yeah I agree, number of stars is mostly worthless. What I really like is being able to read a sample from the book. Reading the first 30 pages gives me a pretty good idea if I'm going to like the authors writing style. In the past I'd do this by picking books of the shelf at the bookstore. On a side note, I've been reading everything written by T. Kingfisher, well except the horror stuff.
Weighting review-scores by agreement is not actually all that computationally costly, since there are various tricks that are used to reduce compute needed (using only a random sample of all reviews; binning readers into categories; relying on particularly significant differences, etc), and it can be done ahead-of-time. There are a lot of people working very hard to optimize recommendation engines (for ad serving especially) and I'm sure Amazon has a bunch of them on staff. It's most likely that the issue is just Goodreads never having been intended to be so large.
Implementing a good recommender system would solve some of the review bombing problem too, since those bombers would be significantly weighted downwards, I think.
I toyed around with a different solution to this problem - adding more metrics. What if users rated a book with 1-5 stars, but also on its pacing, tone, style, etc? Then you could look for books that have a high rating AND the other elements you enjoy.
For what it’s worth, I built a prototype of this. Haven’t fully decided if the idea is working or not, but here you go: https://inkhrt.com.
Yes, more metrics like that is a great idea! You've got a good point that even if they don't get into the full overall rating, they still give nonzero information.