With all this talk about Facebook and the value of networks and recommendations, it reminded me of an issue I have had for a long time with Amazon reviews.
The problem is the 'squeaky wheel' syndrome - the 'squeaky wheel' is the only one you hear. Or put another way, you tend not to care when things go right, only when they go wrong.
If you work with service companies, you see this phenomenon reflected in customer feedback. I've seen industries with anywhere from 10-to-1 complaints-to-compliments, to over 30-to-1! People are more likely to give you feedback (negative feedback) when something goes wrong. You can bet your house on it.
Amazon collects reviews from millions of people every day. If you take a look at the ratio of good reviews to bad reviews, the ratio is no where near 30 or even 10-to-1. But this isn't surprising. Amazon promotes reviewing as a way to help people make choices. People are glad to help, so they post positive as well as negative reviews.
However, I guarantee that if you look at the distribution of reviews (good vs bad ratio), it is still a bad indication of how likely you are to be happy with a product. Just as the 30-to-1 complaints-to-compliments ratio of a service company tells you nothing about how good the service is (that particular company had less 1% of all customers complaining).
Case in point, this is the review distribution on Amazon for a cheap USB wireless drive.
If you read the 8 reviews that gave it a "1" you would probably never buy this product. They are scathing. And those 8 reviews are 30% of all reviews given! If you take all 27 reviews as representative of the 'average' experience with this product, you've got a 30% chance to end up with something you will hate. Not great odds.
But it's all a big lie. There is no way Linksys is going to release a product to market that fails, completely, in 30% of cases. It's just not going to happen.
What has happened is Amazon has drastically improved the ratio of bad-to-good reviews, but not to the point that you can be confident that they are representative of the 'average' experience.
This may sound a little bit nit-picky, but it's important to know. It's the same phenomenon at work when CNN post the results of invitation polls - it's a self-selecting group giving their opinion. Something vastly different from actually polling a population.
It's kind of an obvious point. But it's amazing to see how persuasive Amazon reviews (indeed customer reviews in general) are on people's purchase decisions. When most of them tell you little about your chances of receiving a dud (they do have other uses of course).
I ended up buying that drive despite the bad reviews. I will be big and post my experience (good or bad) when it arrives and I get it working.
5 comments:
I give this post a 4 out of 5.
TW.
On the other hand, most of the time you are using the reviews as a relative measure (should I buy product a or b?), rather than an absolute (should I buy produce a or nothing). As long as the bias is roughly the same for all products, the relative estimate should be ok.
Yep, good point Hadley. All products do have the bias. However, you still can go wrong because the bias (if you aren't aware of it) skews your view of other relative comparisons. Price for instance - what if you are deciding between product a and b and the prices are quite a bit different. You want to buy the cheaper one (b) but the 8 bad reviews gives you an unrealistic idea of the probability of getting a dud so you get "a" instead. If you had known the actual probability of getting a dud "b", would you have made the same decision? The review bias in this case affects the price elasticity of the decision.
If you're not trying to make these types of decisions, your point stands - and it is probably valid for a vast number of decisions you make on sites like Amazon.
A related phenomenon is movie reviews - I'll often see a lot of bad reviews of a movie and decide not to go. However, if I look at the reviews of movies that I have enjoyed, there are often just as many negative reviews.
Another thing to watch out for is only a few praising reviews on either products or especially books. Typically, an author will get a few close "friends" to write glowing reviews about the book because they know people read them and make buying decisions based on reviews. Heck, I do. Now, I wonder if authors actually write the review and send it to someone to post, or if people charge to write positive reviews.
Another example is the phone company. You never pick up a landline and say, wow, it works! You only scream when it doesn't work. I try to go out of my way to let people know when they have gone above and beyond or just provided superior customer service or products. It can go a long way!
Post a Comment