Liberty Vittert: A statistician's guide to life -- What online 'average' customer reviews really mean
Anyone with an internet connection today can rate products, drivers, doctors, restaurants and more -- but, what do those stars and "average" reviews really mean?
A few days ago, I bought a dog bed on Amazon. (My dog Henry has managed to chew through many, many a bed.) I frankly don’t know why I bothered, but I looked at the dog bed reviews -- the average number of stars that other customers had given to rate their satisfaction with the dog bed.
Photo credit: Liberty Vittert
I was choosing between a dog bed that had 4 stars with about 6,000 reviews and another that had 4.6 stars with 200 reviews. How do I choose? Do I pick the one with more reviews, but a lower rating or less reviews with a higher rating?
I ended up buying the bed with more reviews and then got an email asking me to rate it. It was a conundrum -- how do I rate a dog bed? I can’t ask Henry how much he likes it.
I ended up buying the bed with more reviews and then got an email asking me to rate it. It was a conundrum -- how do I rate a dog bed? I can’t ask Henry how much he likes it.
Photo credit: Liberty Vittert
I mean, it looks pretty nice and it has his name on it. He seems to like it enough. He hasn’t chewed through it yet (just give him time, though).
Do I give it 4 stars? 5 stars? What does that even mean? I gave it 5 stars. I’m not sure why. I mean, it’s not that great.
'Average customer review' -- the fine print
But more importantly, the stars on Amazon are called an “average customer review” rating. So what exactly does average mean? Well, that very phrase “average” actually gives a whole lotta wiggle room.
Let’s look at a more important example than Henry’s dog bed.
It would be unequivocally true for one of the many Democratic presidential candidates to say, “Average income for the American household was $83,143 when President Obama left office, but under President Trump, the average income is only $63,179. Elect me!” Yes, that statement is absolutely true, but it isn’t correct.
There are three measures of “average”; mean, median and mode.
It can be said that the average household income in America in 2018 was $90,021, if the mean is used as the “average” (adding up all household incomes and dividing by the number of households). But the average income is only $63,179 if using the median as the measure of “average” (the income level where half of households are above and half are below). Or if you want to make it sound really dire, you can say the average income was $12,500, if you use the mode (the income that occurs the most often).
Does Amazon ... or Uber ... or Yelp use the mean to measure average -- which can be skewed by a few very negative or very positive reviews? Or, do they use the mode, the median, or something else entirely?
For Amazon, it is something else entirely. The company announced in 2015 it would begin using a new proprietary method under which some individual reviews carry more weight than others based upon how recently the customer purchased the product or whether the customer reviewer is “verified” (i.e. Amazon confirms the item was actually purchased by that customer.) There are many other factors, including whether the customer regularly leaves negative reviews, which can lower their impact.
So where does all this confusion leave you?
From my statistical viewpoint, your best bet is to go with the product with the most reviews. There will be the least amount of skew from a few very positive or very negative reviews. But, there is a catch; you may be giving up a spectacular product with few reviews for a “good enough” product with thousands of reviews.
GET FOX BUSINESS ON THE GO BY CLICKING HERE
The bottom line is -- whether it is a serious decision in your personal life, who you are going to vote for, or buying a dog bed -- make sure “the average” and the other stats that may be influencing you tell you what you really need to know. After that, the risk versus reward decision is one only you can make.
P.S. I’ll let you know how the dog bed holds up.
Professor Liberty Vittert is a professor of the practice of Data Science and an ambassador for the Royal Statistical Society.