Reply
cancel
Showing results for 
Search instead for 
Did you mean: 
Reply

OFFICIAL PROPOSAL TO UPWORK: Failure in Job Success Score algorithm

Highlighted
Active Member
Arto K Member Since: May 11, 2019
1 of 21

Hello fellow freelancers,

 

Before saying that "this topic should be messaged to admins/devs", let me say that I want to make this public to raise some discussion about it: Job Success Score and how it's calculated.

 

Let me get this out of the way: I sincerely believe that Job Success Score is a needed feature. This post is not against it.

However, there is one thing that I find very especially problematic in it. The lack of feedback can count negatively towards your JSS.

 

I have several completed jobs (usually small scale, one small job), where the client has disappeared or does not return my messages. Asking for a review is something that I might do once, but will not keep spamming an unresponsive client.

 

My arguments on why this policy is faulted:

1. On certain fields, there are a considerable amount of clients who use UpWork only for one project or have a minimal amount of projects on a large timespan. I cannot speak for other fields, but mine (copywriting/translation) I feel like it's an issue. They receive the work, release the payment and leave never to return again.

 

2. The sole fact that the payment was released with no disputes, speaks for the fact that job was completed succesfully.

 

3. The clients are not forced nor penalized for not leaving feedback, so if someone doesn't care enough to dedicate a small time, they can do that.

 

The lack of review does not directly equal that Job Success was bad. Thus, it makes no sense to count it negatively towards it.

 

 

My proposed solution to this problem:

 

Forcing the reviews for the clients. I propose an update in a following manner: when client has an option to release a payment, a simple window could pop up, kindly asking to leave an honest review to release the payment. The "Release payment" button cannot be pressed before at least the star-based reviews are left, text review can still be left optional. This will not take more than 20 seconds of clients time.

 

Same system can be applied to freelancers, too. Before withdrawing payments, system makes sure that all clients have received review from the freelancer. Does not take much time either and if freelancer has already left reviews to all clients, no problem the money can be withdrawn without any stops.

 

Job Success Score is great, but it really is obscure om how it's calculated. I have made deduction on why mine has gotten bumped down consistently by reading all the documentation and comparing it to my profile and work history.

 

As I said before, I chose to post about this because I want to see am I alone with this or is it a more widespread problem. If it's indeed a problem, maybe community engagement will encourage UpWork to review this policy, which does not directly reflect the input and quality of work.

 

Please let me know do you agree or disagree with this, and why?

Are you satisfied with JSS calculation system?

 

All the best to all of you

 

Active Member
Damon B Member Since: Apr 11, 2017
2 of 21

I think this is a really good suggestion.

 

Job success score has been a point of contention for some freelancers and it can be a barrier for some people who are still starting out. Your suggestion legitimately helps people to improve their score.

Community Guru
Petra R Member Since: Aug 3, 2011
3 of 21

Arto K wrote:

 

However, there is one thing that I find very especially problematic in it. The lack of feedback can count negatively towards your JSS.


"No feedback" contracts, provided money was paid and you don't have a really high percentage of such contracts, have absolutely no effect  on your JSS at all.

Nor do idle contracts (provided money was paid and you don't have a really high percentage of such contracts.)


You are asking to eliminate something that isn't even happening. Huge waste of time.

 

Before you say "But why is my JSS 83% when I have only 5 star feedback?" - go to your "My Stats" page and look at the private feedback those clients who did leave feedback (only) left for you (under "clients who would recommend me.")

 

JSS is mainly calculated from private feedback, because nearly everyone leaves 5 star public feedback.

 

 

Community Guru
Rene K Member Since: Jul 10, 2014
4 of 21

Arto K wrote:

The lack of feedback can count negatively towards your JSS.


 

Your Job Success Score on Upwork

 

(...)

A pattern of contracts that end without feedback can reduce your score


When it happens occasionally, your score is not impacted by contracts ending without feedback. But when this occurs in excess for a freelancer, it is flagged and counted against your score. Requesting that the client close the contract and leave feedback can prevent this situation. Contracts paused due to poor client behavior are not considered in your score.

(...)

 

also:

 

(...)

What else do I need to know about the Job Success score?


It’s intentionally different from your 5-Star rating
The scores differ because Job Success contains more than just public feedback. It also includes private feedback, long-term contracts, and repeat contracts. The collection of these inputs helps freelancers to better differentiate themselves on Upwork.

(...)

-----------
"Where darkness shines like dazzling light"   —William Ashbless
Community Guru
Will L Member Since: Jul 9, 2015
BEST ANSWER
5 of 21

Arto,

 

If the purpose of the JSS system were to be to provide true feedback for freelancers it would be transparent and only be calculated using elements the freelancer can affect. Of course, a freelancer cannot force a client to provide feedback, so we're told it's not a big deal for the JSS if the freelancer doesn't have a lot of "no feedback" completed projects. There is no known precise definition of "a lot," so non-feedback clients are just a fact of life for many freelancers.

 

But the purpose of the JSS system is to a) make it easier for clients to wade through the sometimes overwhelming proposals they receive and b) to make clients feel they are choosing "the best of the best" when they select a freelancer for their project. This is why the system clients use to post new projects defaults to a minimum JSS requirement of 90, despite the fact there is no evidence there is any real difference between a freelancer with a JSS of 89 and another freelancer with a JSS of 90.

 

There are other ways to deal with a), but they would require a little more work by the client. (We can't have that.) Upwork has provided no information that addresses point b).

 

Upwork needs a lot more clients spending a lot more money in order to eventually turn a profit. That is particularly important now that the company's shares are publicly traded. The big drop in the company's share price last week was an unavoidable reminder of this for management.

 

So, transparency is not a primary goal of the JSS system. (And if it were, there is no way Upwork could prevent freelancers unhappy with a client's less-than-stellar feedback from hastling the client.)

 

Upwork has made no apparent significant change to the JSS system during the few years I have used Upwork. Management has much bigger fish to fry and there is no reason for them to listen to freelancers' complaints about the JSS's weaknesses and inconsistencies as long as Upwork has far, far more freelancers than are needed to serve the far fewer clients who use Upwork. There is no reason to expect any change to the system is likely for the foreseeable future.

Active Member
Arto K Member Since: May 11, 2019
6 of 21

Hello Will,

 

Thank you for your in-depth answer, I truly appreciated it.

 

I understand what you said, to be honest I knew it already before posting, but I did not wan't to seem hostile towards UpWork as a company, as it is not my objective.

 

Obviously, as a for-profit company, UpWork will keep concentrating to the client side, as that is where the money is coming from. With the rise of remote worker/digital nomad movement, there will always be another freelancer, but clients are harder to obtain and keep.

 

This is the "not-really-my-problem" part, but with rise of new competitors on the market, it should be pretty important to listen to people who work for you. UpWork is to the date market leader and the best platform with large client base. However in my opinion they would need to look further into future, because if enough freelancers are gruntled enough, and better (or equivalent) option comes along, UpWork might experience migration, which will defintely be bad for stockholders.

 

What comes to the possibility of "what if the forced reviews are bad", then freelancer has not met the client satisfaction. To my experience, pretty much all clients first send a PM and ask a correction for bits their not satisfied with. Most of the clients know how important a good review is and will try the win-win way first, by asking the freelancer to correct their work.

 

Bad reviews happen when freelancers refuse to cooperate or are just not up to clients standards, which in my mind means that they have lied/boasted on their profile or application (the amount of clients that are really just "mean" is fractional).

 

In conclusion I agree with you that there is probably not going to be a change for this in foreseeable future. But if it's never discussed about there will never by a change and that's the reason why I started on this topic.

Community Guru
Will L Member Since: Jul 9, 2015
7 of 21
"gruntled"? I have always wondered what the opposite of disgruntled would
be. 😉

Active Member
Arto K Member Since: May 11, 2019
8 of 21

This is the time when I play the "not-a-native-speaker" card. Smiley Very Happy

Community Guru
Rene K Member Since: Jul 10, 2014
9 of 21

Arto K wrote:

I propose an update in a following manner: when client has an option to release a payment, a simple window could pop up, kindly asking to leave an honest review to release the payment. The "Release payment" button cannot be pressed before at least the star-based reviews are left, text review can still be left optional. This will not take more than 20 seconds of clients time.


I love the idea. Maybe worth exploring.

 

 

-----------
"Where darkness shines like dazzling light"   —William Ashbless
Community Guru
Petra R Member Since: Aug 3, 2011
10 of 21

Rene K wrote:

Arto K wrote:

I propose an update in a following manner: when client has an option to release a payment, a simple window could pop up, kindly asking to leave an honest review to release the payment. The "Release payment" button cannot be pressed before at least the star-based reviews are left, text review can still be left optional. This will not take more than 20 seconds of clients time.


I love the idea. Maybe worth exploring.


Pointless / unworkable / unfair:

  • Doesn't work with hourly or multi-milestone contracts.
  • It is only public feedback, which is just about meaningless as far as the JSS is concerned.
  • Leaves more feedback for contracts with many small milestones than big contracts, and none on hourly.
  • Clients can't always know how the contract will pan out after the first milestone
  • Often clients don't want to leave feedback. They are not happy, but don't want to ruin the freelancer's profile either. So they let the freelancer close the contract so they don't have to leave feedback. So forcing this does not do freelancers any favours...
TOP SOLUTION AUTHORS
TOP KUDOED MEMBERS