🐈
» Support » Product Release Notes » Replacing 5-Star Average Feedback with Job Su...
Page options

Replacing 5-Star Average Feedback with Job Success Score

garnorm
Community Member

Hi all,

We remove 5-star average feedback entirely, and display only your Job Success score throughout most of the Upwork site. 5-star feedback given per contract will remain displayed on your Work History.

 

We’re making this decision because we’ve seen Job Success scores help top performing freelancers win more contracts. This means clients are having better experiences and coming back for more, which in turn benefits the marketplace and generates more opportunities for freelancers overall. The score helps freelancers at all levels stand out from the competition by presenting clients with a complete picture of their differences. This is an opportunity to help freelancers continue to grow their businesses that we can’t pass up.

 

Additionally, you can learn more about Job Success score and how it's calculated in our help article.

540 Comments
researchediting
Community Member

@Stephen D wrote:

....stable and bug-free Elance platform.

 

 No. False. Not ever. In your dreams.

 

....Most ridiculous is that obviously nobody listens to seasoned professionals here. Instead of considering their opinions, you simply defend your point blindly and at all costs.

 

Gee, that complaint sounds familiar. Where have I heard it before? Could it have been...Elance?


Best,

Michael

iaabraham
Community Member
I don't like this one bit and there's no use explaining why because the decision has been made.
prestonhunter
Community Member

re: "I don't like this one bit"

 

Isabelle,

You have a very high job score. This can only help you.

 

Although contractors with lower job scores don't yet realize it, moving from feedback stars to job scores will help them as well.

 

Why exactly do you not like this?

I'm having a hard time understanding why people object to this when it will help make Upwork better and will help them earn more money?

 

Is it simply because the Job Score is a newer measure than feedback stars, and more complex and opaque in terms of its formulation?

stephenbrightman
Community Member

Preston

You've questioned myself and Isabelle Anne now on why we don't like it even though our JS scores are high. Well. maybe it's because we care about the site and the people on it and not just ourselves.

 

"I'm having a hard time understanding why people object to this when it will help make Upwork better and will help them earn more money?"

 

How do you know that?? With respect, you're beginning to sound like one of them now. Relying totally on a dark, secret metric that punishes freelancers but never clients, while making clients less and less acountable and their past history less visble, is not IMO making Upwork a better place.

iaabraham
Community Member

Preston please - usually your posts are rational but come on. Should I keep quiet about something that's wrong just because I'm not negatively affected by it?

 

Besides, my JS can drop significantly at any moment and we all know that. 

 

My complaints about JS are the same as those we've read in countless threads these past months.

 

Is the star-feedback system without its flaws? Of course not, but at least it's more transparent/straightforward and less subject to the whims of the Upwork team and its crazy algorithims.

 

And not to mention the fact that instead of fixing the bugs that are making some of us tear our hair out, we're once again saddled with an unasked-for "improvement" to the site.

prestonhunter
Community Member

Stephen and Isabelle, I appreciate your comments and think your motivations are very admirable because this change is very likely to be advantageous to you.

 

I, too, want Upwork to prosper and do things that are beneficial for all of its contractors. I think where we have a disagreement is that I genuinely think this will benefit all (or nearly all) contractors.

 

I don't think the Job Score is "wrong." I see it as something different which is designed to solve a problem. I just don't think the star feedback works for differentiating contractors. I think the job score really improves the client experience and brings more money to contractors at all levels.

lenaellis
Community Member

Hi John,

 

To answer you question,  Clients' 5-star average feedback when viewing a job post will remain. 

 

Thanks,

 

Lena

droleary
Community Member

@Preston H wrote:

I don't think the Job Score is "wrong." I see it as something different which is designed to solve a problem. I just don't think the star feedback works for differentiating contractors. I think the job score really improves the client experience and brings more money to contractors at all levels.


You keep saying nonsense like this, but you don't ever back it up by a reasoned argument.  There is nothing "right" about my 82% JSS, especially the fact that I have no idea how it got calculated compared to my 5 star feedback.  The problem with star ratings is a statistical one (skew), and that is definitely not a problem that is solved by the JSS.  Basic statistics knowledge would also fix your incorrect conclusion that "all levels" are benefiting.

 

prestonhunter
Community Member

re: "There is nothing 'right' about my 82% JSS"

 

Darrin, I agree that an 82% job success score does not accurately reflect your quality as a contractor. Your score should be higher.

 

If you continue working on Upwork, I have no doubt that your score will go up. I realize that this is little consolation now.

prestonhunter
Community Member

For those of you wondering why I believe this change will be beneficial, here is an explanation:

 

* There are many excellent contractors on Upwork. There are mediocre contractors. There are bad contractors.

 

* The feedback star average rating is heavily clustered around 5 stars across the board for contractors. There is little spread. When clients see this, they either feel that it is not very helpful or they feel that they are looking at clients who are all of similarly high quality.

 

* Using average star feedback, clients have a difficult time discerining between bad, poor, mediocre, good, great contractors. Whether they realize this before hiring anybody, or after hiring a poorly-performing contractor who has a high-star rating, this diminishes the client's experience using Upwork.

 

* Clients who have a poor experience using Upwork because they hired a poorly-performing contractor who they mistakenly thought would be high-quality due to their near-5-star feedback average are less likely to return and post additional jobs and they are less likely to recommend Upwork to other potential clients.

 

* With the Job Success Score, there is a much greater spread of scores. A client may see applicants applying to his job posting who represent a wide range of job success scores. They might see, for example, contractors with scores ranging from 60% to 100%.

 

* The client seeing this wider range of scores feels that Upwork is a more useful tool because he sees candidates representing a range of possible quality levels. This is a good thing in and of itself. Moreover, the client is more likely to be able to hire contractors who represent a targeted quality level. Sometimes a client may want to hire the very highest quality contractor and pay more for doing so. Sometimes a client may have a job that he gladly entrusts to a lower-scored contractor due to the contractor's excellent cover letter or lower hourly cost or other reasons. If the client ends up with a more predictable experience with the contractor, he will be more likely to be satisfied, more likely to post additional jobs, and more likely to recommend Upwork to others.

 

* If clients have a more positive experience and are more likely to recommend Upwork and more likely to use it themselves, this means there are more jobs on Upwork. Having more real jobs on Upwork benefits all contractors. It is not possible for ONLY the contractors with the highest possible job scores to do all jobs. Other contractors will be hired as well. Personally, I have a relatively high job score, but I do not have time to apply to all jobs that I might be qualified for. On average I apply to only one or two jobs per month. I turn down most invitations. These jobs are from real clients who hire other contractors, at all ranges of Job Score.

 

So, to those of you wondering what I'm thinking, there it is.

 

Note that none of what I have said here addresses whether or not the Job Success Score is transparent, understandable or fair, or whether it can or should be improved or not.

 

I am definitely NOT saying that the Job Score provides a perfect and/or fair ranking of all contractors on Upwork. I am not asking any contractor to like this change or to refrain from criticizing the job succes score and how it is calculated.

 

The move to using primarily Job Succes Scores to rate contractors is primarily client-centric. This is not intended as a service to all contractors. The intention is to please clients and increase client spending on Upwork.