Feedback, stars and the wisdom of the crowd

m_sharman · ‎05-26-2019

Interesting op-ed piece in the NY times today about online reviews/ratings. I found this quote interesting and wondered whether it applied to Upwork:

Finally, it’s hard to know what the stars even mean. Often times, whether it’s a mattress or can opener or an Uber driver, a five-star rating means “nothing disastrous happened,” said Nikhil Garg, a doctoral candidate at Stanford University. A recent study he co-wrote reported that 80 percent of people gave freelancers hired from an online platform five stars. But when he asked people to choose from different words (“terrible,” “mediocre,” “best possible,” etc.), at least half of the freelancers earned the equivalent of a two-, three- or four-star review.

For those interested in the article, here is a link to read the full text.

prestonhunter · ‎05-26-2019

This relates to why Upwork made the change a few years back to start using Job Success Scores, which vastly diminish the importance of feedback star averages.

wendy_writes · ‎05-26-2019

Miriam, exactly! This is why the words buyers take the time and initiative to write are so much more indicative than the stars.

On a side note, JSS is a way of life here - but without knowing the content of private feedback, FLers have no confirmed knowledge of their shortcomings ... and can't correct / address them.

Sharing a compilation of say 6 months worth of private feedback - shared blindly with no buyer names or dates included - would benefit everyone.

luce-neidert · ‎05-28-2019

Wendy C wrote:
Miriam, exactly! This is why the words buyers take the time and initiative to write are so much more indicative than the stars.

On a side note, JSS is a way of life here - but without knowing the content of private feedback, FLers have no confirmed knowledge of their shortcomings ... and can't correct / address them.

Sharing a compilation of say 6 months worth of private feedback - shared blindly with no buyer names or dates included - would benefit everyone.

That information would be rather interesting. As Upwork seems to love finding new ways to make money, the platform could even sell the information to the freelancers.

To get back to the topic of 5 stars, it's interesting to note that there is a cultural side to them. As you know, the typical French person always has something to complain about. Giving 5 stars is hardly compatible with the French culture. I had a French customer for a long time who would always write extremely positive reviews, but never give 5 stars.

sergio-soria · ‎05-29-2019

Luce N wrote:
Giving 5 stars is hardly compatible with the French culture. I had a French customer for a long time who would always write extremely positive reviews, but never give 5 stars.

Good to know that French customers can take your JSS down!

luce-neidert · ‎05-29-2019

Sergio S wrote:
Luce N wrote:
Giving 5 stars is hardly compatible with the French culture. I had a French customer for a long time who would always write extremely positive reviews, but never give 5 stars.
Good to know that French customers can take your JSS down!

They do, but at the same time, they tell you that you're great.

purplepony · ‎05-28-2019

Wendy C wrote:
Miriam, exactly! This is why the words buyers take the time and initiative to write are so much more indicative than the stars.

On a side note, JSS is a way of life here - but without knowing the content of private feedback, FLers have no confirmed knowledge of their shortcomings ... and can't correct / address them.

Sharing a compilation of say 6 months worth of private feedback - shared blindly with no buyer names or dates included - would benefit everyone.
__________________________________________________________________________
Wendy, I definitely agree. However, IMO, more than 6 months would be more beneficial as many of us oftentimes have projects which continue for 6 months and beyond,

wendy_writes · ‎05-29-2019

Pat, if U would do this every six months we would have a rolling record - longer than 6 month jobs would still show. It would just be a matter of tracking your semi-annual 'review'.

Less than 6 months takes more effort than needed and longer than every six months is not fair to FLers who did one-off jobs due to the nature of their expertize. Thinking of resume and LI profile writers and some translators -

lysis10 · ‎05-29-2019

I'm getting annoyed at companies nagging me to review. I apparently forgot my password to yelp, and I'm not interested in resetting or getting my account back so I don't review.

Got an email from the moving company that moved me over a year ago to review them. I call a place for CS support and they want a review. Everything I do people want a review. Leave me alone, people. No I don't want to review you. Send me an email ONCE, and then leave me alone about your stupid reviews if I choose to not answer.

mwiggenhorn · ‎05-29-2019

I agree, Jenn. I did write a review for an Amazon product yesterday because it just might be the best thing I ever bought but they didn't ask me. I did it on my own.

luce-neidert · ‎05-29-2019

Mary W wrote:
I agree, Jenn. I did write a review for an Amazon product yesterday because it just might be the best thing I ever bought but they didn't ask me. I did it on my own.

You're being too kind to Amazon. They don't deserve that kind of cooperation.

gilbert-phyllis · ‎05-30-2019

Luce N wrote:
Mary W wrote:
I agree, Jenn. I did write a review for an Amazon product yesterday because it just might be the best thing I ever bought but they didn't ask me. I did it on my own.
You're being too kind to Amazon. They don't deserve that kind of cooperation.

Reviews of products sold on Amazon mainly help other customers. I consult them a lot. (I think Amazon is one of several big entities that need to be broken up, but I live in a very rural area and depend on it heavily.)

Today, I took grim pleasure in writing a 1-star review of a slide-out keyboard tray I purchased back in January. It's never functioned quite as promised and yesterday, it fell in my lap (and nearly onto my bare feet, which would have surely fractured bones).

luce-neidert · ‎05-31-2019

Phyllis G wrote:
Luce N wrote:
Mary W wrote:
I agree, Jenn. I did write a review for an Amazon product yesterday because it just might be the best thing I ever bought but they didn't ask me. I did it on my own.
You're being too kind to Amazon. They don't deserve that kind of cooperation.
Reviews of products sold on Amazon mainly help other customers. I consult them a lot. (I think Amazon is one of several big entities that need to be broken up, but I live in a very rural area and depend on it heavily.)
Today, I took grim pleasure in writing a 1-star review of a slide-out keyboard tray I purchased back in January. It's never functioned quite as promised and yesterday, it fell in my lap (and nearly onto my bare feet, which would have surely fractured bones).

Well, Phyllis, using Upwork has made me aware of how feedback can be completely fake. Have you noticed all those "jobs" were you are sent a product then you are paid to write a review about it?

Also, I think it's time customers stopped using that company, which I do as much as I can.

gilbert-phyllis · ‎05-31-2019

Luce N wrote:
Phyllis G wrote:
Luce N wrote:
Mary W wrote:
I agree, Jenn. I did write a review for an Amazon product yesterday because it just might be the best thing I ever bought but they didn't ask me. I did it on my own.
You're being too kind to Amazon. They don't deserve that kind of cooperation.
Reviews of products sold on Amazon mainly help other customers. I consult them a lot. (I think Amazon is one of several big entities that need to be broken up, but I live in a very rural area and depend on it heavily.)
Today, I took grim pleasure in writing a 1-star review of a slide-out keyboard tray I purchased back in January. It's never functioned quite as promised and yesterday, it fell in my lap (and nearly onto my bare feet, which would have surely fractured bones).
Well, Phyllis, using Upwork has made me aware of how feedback can be completely fake. Have you noticed all those "jobs" were you are sent a product then you are paid to write a review about it?

Also, I think it's time customers stopped using that company, which I do as much as I can.

I believe it's pretty easy to tell the fake reviews from authentic ones on Amazon. For the products I buy, anyway.

Boycotting Amazon is a luxury that people in deeply rural areas do not have. I am 35 miles (each way) from the nearest office supply store, for instance.

petra_r · ‎06-01-2019

Luce N wrote:
Well, Phyllis, using Upwork has made me aware of how feedback can be completely fake. Have you noticed all those "jobs" were you are sent a product then you are paid to write a review about it?

Such job posts and jobs are in violation of Upwork's terms of service and should be flagged and taken down.

colettelewis · ‎06-01-2019

Petra R wrote:
Luce N wrote:
Well, Phyllis, using Upwork has made me aware of how feedback can be completely fake. Have you noticed all those "jobs" were you are sent a product then you are paid to write a review about it?
Such job posts and jobs are in violation of Upwork's terms of service and should be flagged and taken down.

__________________________________

Like so many other violations of UW's terms of service - some are taken down and some aren't. I think it depends on the dollar value of the client.

prestonhunter · ‎06-01-2019

re: "Well, Phyllis, using Upwork has made me aware of how feedback can be completely fake. Have you noticed all those 'jobs' were you are sent a product then you are paid to write a review about it?"

Amazon.com doesn't allow this for non-book items.

And Upwork doesn't allow job postings which violate Amazon.com ToS.

Upwork does not have any general prohibition against writing reviews. The rule is about violating other sites' ToS.

Anonymous-User · ‎05-29-2019

I'm sick of stars, thumbs up, likes, scores...

Artificial intelligence and judgement.

sergio-soria · ‎05-30-2019

Cairenn R wrote:
I'm sick of stars, thumbs up, likes, scores...
Artificial intelligence and judgement.

I feel guilty giving a thumbs up to your comment. LOL