How much trust do you put in online reviews?

Borla · Aug 20, 2011

I just read this article: http://www.nytimes.com/2011/08/20/technology/finding-fake-reviews-online.html?hp

In tens of millions of reviews on Web sites like Amazon.com, Citysearch, TripAdvisor and Yelp, new books are better than Tolstoy, restaurants are undiscovered gems and hotels surpass the Ritz.

Or so the reviewers say. As online retailers increasingly depend on reviews as a sales tool, an industry of fibbers and promoters has sprung up to buy and sell raves for a pittance.
“For $5, I will submit two great reviews for your business,” offered one entrepreneur on the help-for-hire site Fiverr, one of a multitude of similar pitches. On another forum, Digital Point, a poster wrote, “I will pay for positive feedback on TripAdvisor.” A Craigslist post proposed this: “If you have an active Yelp account and would like to make very easy money please respond.”
The boundless demand for positive reviews has made the review system an arms race of sorts. As more five-star reviews are handed out, even more five-star reviews are needed. Few want to risk being left behind.
Sandra Parker, a freelance writer who was hired by a review factory this spring to pump out Amazon reviews for $10 each, said her instructions were simple. “We were not asked to provide a five-star review, but would be asked to turn down an assignment if we could not give one,” said Ms. Parker, whose brief notices for a dozen memoirs are stuffed with superlatives like “a must-read” and “a lifetime’s worth of wisdom.”
Determining the number of fake reviews on the Web is difficult. But it is enough of a problem to attract a team of Cornell researchers, who recently published a paper about creating a computer algorithm for detecting fake reviewers. They were instantly approached by a dozen companies, including Amazon, Hilton, TripAdvisor and several specialist travel sites, all of which have a strong interest in limiting the spread of bogus reviews.
“The whole system falls apart if made-up reviews are given the same weight as honest ones,” said one of the researchers, Myle Ott. Among those seeking out Mr. Ott, a 22-year-old Ph.D. candidate in computer science, after the study was published was Google, which asked for his résumé, he said.
Linchi Kwok, an assistant professor at Syracuse University who is researching social media and the hospitality industry, explained that as Internet shopping has become more “social,” with customer reviews an essential part of the sales pitch, marketers are realizing they must watch over those opinions as much as they manage any other marketing campaign.
“Everyone’s trying to do something to make themselves look better,” he said. “Some of them, if they cannot generate authentic reviews, may hire somebody to do it.”
Web retailers are aware of the widespread mood of celebration among their reviewers, even if they are reluctant to discuss it. Amazon, like other review sites, says it has a preponderance of positive reviews because of a feedback loop: Products with high-star ratings sell more, so they get more reviews than products with poor ratings.
But they are concerned about the integrity of those reviews. “Any one review could be someone’s best friend, and it’s impossible to tell that in every case,” said Russell Dicker, Amazon’s director of community. “We are continuing to invest in our ability to detect these problems.”
The Cornell researchers tackled what they call deceptive opinion spam by commissioning freelance writers on Mechanical Turk, an Amazon-owned marketplace for workers, to produce 400 positive but fake reviews of Chicago hotels. Then they mixed in 400 positive TripAdvisor reviews that they believed were genuine, and asked three human judges to tell them apart. They could not.
“We evolved over 60,000 years by talking to each other face to face,” said Jeffrey T. Hancock, a Cornell professor of communication and information science who worked on the project. “Now we’re communicating in these virtual ways. It feels like it is much harder to pick up clues about deception.”
So the team developed an algorithm to distinguish fake from real, which worked about 90 percent of the time. The fakes tended to be a narrative talking about their experience at the hotel using a lot of superlatives, but they were not very good on description. Naturally: They had never been there. Instead, they talked about why they were in Chicago. They also used words like “I” and “me” more frequently, as if to underline their own credibility.
How far a business can go to get a good review is a blurry line. A high-end English hotel, The Cove in Cornwall, was recently accused in the British media of soliciting guests to post an “honest but positive review” on TripAdvisor in exchange for a future discount of 10 percent. Nearly all the recent reviews of the Cove are glowing except for the one headlined, “Sadly let down by overhyped reviews.”
The hotel said it was a loyalty scheme that was being misconstrued. TripAdvisor, though, posted a warning about the Cove’s favorable notices on its page for the hotel. The site declined to say how often it has had to post such caveats.
Founded 11 years ago, TripAdvisor never expected to see so many positive reviews. “We were worried it was going to be a gripe site,” said the chief executive, Stephen Kaufer. “Who the heck would bother to write a review except to complain?” Instead, the average of the 50 million reviews is 3.7 stars out of five, bordering on exceptional but typical of review sites.
Negative reviews also abound on the Web; they are often posted on restaurant and hotel sites by business rivals. But as Trevor J. Pinch, a sociologist at Cornell who has just published a study of Amazon reviewers, said, “There is definitely a bias toward positive comments.”
Mr. Pinch’s interviews with more than a hundred of Amazon’s highest-ranked reviewers found that only a few ever wrote anything critical. As one reviewer put it, “I prefer to praise the ones I love, not damn the ones I did not!”
The fact that just about all the top reviewers in his study said they got free books and other material from publishers and others soliciting good notices may have also had something to do with it.
Even if you get a failing grade or two, all is not lost. Dot-coms like Main Street Hub manage the reputations of small businesses for a fixed fee.
“A courteous response to a negative review can persuade the reviewer to change their reviews from two to three or four stars,” said Main Street’s chief executive, Andrew Allison. “That’s one of the highest victories a local business can aspire to with respect to their critics.”
The result, he said: “It’s like Lake Wobegon. Everyone is above average.”
Click to expand...

It's not surprising, but it's disappointing how much of a business writing fake reviews has become. I would LOVE to have the chance to try to sift through the fake vs. real reviews as mentioned in the article to see how good I was at detecting fake ones. I find myself often reading through reviews on Yelp! or Amazon and discarding them because the tone or vocabulary of the reviewer makes me think they are trying too hard, or I get a gut feeling that they are the type who is either easily impressed or conversley always nit-picky. Maybe I'm being arrogant, but I think I could sniff out a good portion of the fake reviews. And that's why I rarely just look at a product or place and say "Wow, it has 4.5 stars, it must be great!" without reading several reviews in detail first.

On the other hand, I've really been trying to use Yelp! to review places I visit so that I can give feedback to others, since I appreciate the feedback I read there and use it often. I also figure that it's an additional way to support the businesses that I like.

amonkie · Aug 20, 2011

I am big into Yelp, partly because you can look at a poster's ratings and see how many 1,2,3,4, and 5 start ratings they give.

When I read reviews I look for points of consistency on specifics (Good or bad) across the different reviews. Hearing about characteristics of a specific waiter or staff person by name or appearance, patterns on specific days, etc and also how long ago the place was rated.

Amazon is a totally different animal though. It's hard to know what someone's expectations were going into a purchase, and whether the expectations were the same as mine.

Spiritsoar · Aug 20, 2011

To be completely honest, I almost always go to the bad reviews first. I look at them carefully (one bad experience doesn't mean a horrible place), but I've found that consistency in bad reviews shows a lot more than consistency in good reviews.

Borla · Aug 20, 2011

amonkie said: ↑

I am big into Yelp, partly because you can look at a poster's ratings and see how many 1,2,3,4, and 5 start ratings they give.

When I read reviews I look for points of consistency on specifics (Good or bad) across the different reviews. Hearing about characteristics of a specific waiter or staff person by name or appearance, patterns on specific days, etc and also how long ago the place was rated.

Amazon is a totally different animal though. It's hard to know what someone's expectations were going into a purchase, and whether the expectations were the same as mine.
Click to expand...

Agreed about checking out the poster's ratings. I especially like it if I can find their reviews on places that I frequent to see how much their opinions align with mine.

Spiritsoar said: ↑

To be completely honest, I almost always go to the bad reviews first. I look at them carefully (one bad experience doesn't mean a horrible place), but I've found that consistency in bad reviews shows a lot more than consistency in good reviews.
Click to expand...

Excellent point, both that one bad experience doesn't mean the place is bad, and that if everyone complains about the same thing it is likely to be true.

ratbastid · Aug 20, 2011

There's a little family italian place in my town here that we LOVE. It's small, cozy, a little offbeat.

The online reviews of it seem to be exactly 50/50 people who love it for all the reasons I love it, and people who hate it.... for all the reasons I love it. Very interesting.

Stan · Aug 20, 2011

I tend to look at negative reviews and compare them to my intended use of the product. A lot of times, people whine about things that I couldn't care less about. One of the criticisms on my new bike was that the muffler bolt tended to vibrate loose. Nice to know, I have Loctite, and I know how to use it. If that's the biggest issue with the bike, I'll be happy.

Ice|Burn · Aug 20, 2011

Spiritsoar said: ↑

To be completely honest, I almost always go to the bad reviews first. I look at them carefully (one bad experience doesn't mean a horrible place), but I've found that consistency in bad reviews shows a lot more than consistency in good reviews.
Click to expand...

This. Looking for a common problem that a bunch of people share is a excellent warning sing.

Devoid · Aug 20, 2011

Stan said: ↑

I tend to look at negative reviews and compare them to my intended use of the product. A lot of times, people whine about things that I couldn't care less about. One of the criticisms on my new bike was that the muffler bolt tended to vibrate loose. Nice to know, I have Loctite, and I know how to use it. If that's the biggest issue with the bike, I'll be happy.
Click to expand...

Yep, same here. If people mark down something because of some stupid shit I don't care about, it won't affect my spending. Also, people just love to bitch.

BadNick · Aug 20, 2011

I see it as similar to "news" now adays aka "show business" in many cases. I get input from multiple sources and many points-of-view before I believe what they say.

Same with reviews and almost anything online: take what you read with a grain of salt and healthy skepticism and get confirmation from other sources.

Ourcrazymodern? · Aug 20, 2011

"...the secretary will disavow any knowledge of your actions."

Zen · Aug 20, 2011

I use the criteria mentioned above. I also google '*product* problem" and "*product* forum"

I feel I have hit the gold mine when a sister or brother geek does a 10 page thread on a tech forum, complete with process-photos, of how they mended or modded the item I am interested in.

Craven Morehead · Aug 25, 2011

I'd trust this one....

Zen · Aug 26, 2011

I tried Yelp to see if it picked up my local Indian restaurant, and yes ... the review was uncannily accurate.

A decent, slightly upmarket Indian restaurant in an area with many Indian people. I am sure we could have got better food had we known the area, but this was certainly passable. As usual too much food! Kingfisher on draught helped.
The weird chutney that went with he pappadum dragged the rating down somewhat
Click to expand...

That 'weird chutney' clinched the deal. No one who hasn't been there would have thought to say. It is not even really a chutney, but a micro-chopped concoction of onion and all manner of strange non-meat entities which refuse to sit on a pappadum.

A different track: 'Insist on precise measurements' - I have just bought two ladies' watches of identical design to the gents' ones I thought I was getting. My mother will be very happy, and I will, henceforth, check for size when reading on-line reviews.

Daval · Aug 26, 2011

I tend to read as many reviews about a place as possible - both bad and good. Everything has to be taken with a grain of salt. I realize there is a lot of fake reviews out there but I figure if I can get a basic trend for a place then I will have a good idea.

wolf · Aug 26, 2011

I like Yelp for places to go. I also definitely read and contribute to the reviews on Newegg.com and TigerDirect.com for technical products I purchase or about to purchase.

Baraka_Guru · Aug 28, 2011

I always keep in mind that social aggregate reviews are probably based mostly on what people like or dislike, rather than any attempt at objective criticism.

You sometimes get conscious reviewers who have good write-ups with a balanced approach, but they are only useful if you can determine if their judgements are in line with your own. It helps if they can draw comparisons or otherwise indicate they have a benchmark for review.

I often find aggregate review scores to be disagreeable. A recent example I came across was at Goodreads, where nearly 25,000 ratings gave G. R. R. Martin's Feast for Crows an average of 4.08 out of 5. That's a score of 82%, which I think is ridiculous. The book was a mess. I gave it a score of 2 out of 5.

I think the issue there is that people are ranking it based on it being a part of a well-loved series. It has characters and references that exist outside of the book itself. It's a part of a whole. However, I looked at it as a book in itself because that's what it is, and I think it failed on a few counts. I nearly didn't finish it. I wasn't going to make excuses for it because it was a part of a puzzle I had been enjoying.

Tully Mars · Aug 28, 2011

Craven Morehead said: ↑

I'd trust this one....

Click to expand...

Sick, twisted... awesome. I love it.

I've written a ton of Trip Adviser reviews. They started sending me gifts after about 50 or so. First it was some crappy back pack, seriously really poor quality. The more I worte the more swag they sent. The last gift I got from them was a pair of Bose noise cancelling head phones. Then I moved to Mexico and stopped posting reviews for them. I should have kept it up.

I've written bad reviews and good. Sometimes if you write a bad review the owner will challenge your review. TA gives, or used to give, owners the chance to respond. I wrote a couple bad reviews for a chain of hotels in Oregon and Washington, they did not like it at all. They tried to suggest I worked for their competition. I've never had anyone offer me anything for writing a good review. But I could see how that could be exploited.

Ourcrazymodern? · Aug 28, 2011

I don't review anything unless I like it. Of course, then I review it obsessively.

snowy · Aug 28, 2011

Tully Mars said: ↑

Sick, twisted... awesome. I love it.

I've written a ton of Trip Adviser reviews. They started sending me gifts after about 50 or so. First it was some crappy back pack, seriously really poor quality. The more I worte the more swag they sent. The last gift I got from them was a pair of Bose noise cancelling head phones. Then I moved to Mexico and stopped posting reviews for them. I should have kept it up.

I've written bad reviews and good. Sometimes if you write a bad review the owner will challenge your review. TA gives, or used to give, owners the chance to respond. I wrote a couple bad reviews for a chain of hotels in Oregon and Washington, they did not like it at all. They tried to suggest I worked for their competition. I've never had anyone offer me anything for writing a good review. But I could see how that could be exploited.
Click to expand...

I really like TripAdvisor. I use it regularly to find cheap hotels that are cheap in price but not in quality or service. I'm going to have to start writing reviews, especially after our stay in the EconoLodge in Bend. Highly recommended. 2-star price, but the service was good, the breakfast was decent, the bed was comfortable, and damn, it was clean and quiet despite being right on BUS 97. It's also just a few blocks from downtown, and half the price of a downtown hotel.

Xazy · Aug 29, 2011

I have found fake reviews on Yelp, ones that were obviously posted by a competitor of the company.

Log in or Sign up

How much trust do you put in online reviews?

Borla Moderator Staff Member

amonkie Very Tilted

Spiritsoar Slightly Tilted

Borla Moderator Staff Member

ratbastid New Member

Stan Resident Dumbass

Ice|Burn Getting Tilted

Devoid New Member

BadNick Getting Tilted

Ourcrazymodern? still, wondering

Zen Very Tilted

Craven Morehead Very Tilted

Zen Very Tilted

Daval Getting Tilted

wolf Evil Grin

Baraka_Guru Möderätor Staff Member

Tully Mars Very Tilted

Ourcrazymodern? still, wondering

snowy so kawaii Staff Member

Xazy Vertical

Log in or Sign up

How much trust do you put in online reviews?

Borla Moderator Staff Member

amonkie Very Tilted

Spiritsoar Slightly Tilted

Borla Moderator Staff Member

ratbastid New Member

Stan Resident Dumbass

Ice|Burn Getting Tilted

Devoid New Member

BadNick Getting Tilted

Ourcrazymodern? still, wondering

Zen Very Tilted

Craven Morehead Very Tilted

Zen Very Tilted

Daval Getting Tilted

wolf Evil Grin

Baraka_Guru Möderätor Staff Member

Tully Mars Very Tilted

Ourcrazymodern? still, wondering

snowy so kawaii Staff Member

Xazy Vertical

Useful Searches