Reply
cancel
Showing results for 
Search instead for 
Did you mean: 
Reply

Upwork rating algorithm punishes hard work, rewards scammers

Ace Contributor
Anton A Member Since: Dec 12, 2017
1 of 69

Upwork's job success score is an insane metric that keeps good people off the platform and favors scammers and those who game the system.

 

I bust my ass for two years here editing videos for peanuts, have all positive public reviews, write tons of proposals and take on every reasonable client. However, after all this effort, I am barely above 90%. As far as I can see, I have close to zero chance of being Top Rated or getting Rising Talent.

 

If I get ONE MORE negative review, I will be relegated to the under-90% category that nobody even considers when searching or hiring. So for the next six month, I pretty much won't be working at all. I am the honest hard working freelancer you want to encourage, but instead, Upwork weighs the score insanely in client's favor. There are some clients that will give you negative review no matter what because they want to use you as a proving ground to see a rough draft of their idea, but have little intention of paying you. Why do I have to sweat every time that I may get a negative score from some a-hole? Why do I never get the benefit of the doubt?

 

Now, here's the best part: my friend, on my recommendation, comes to Upwork and  hires a Top Rated guy to write some Swift code. This jerk bills him over $2K, delivers nothing, and ends up being a complete scammer. This is a guy who has great reviews and comes up as one of the top results in a search with a friggin badge next to his name and a 100% success score.

 

Sorry Upwork, but this is complete and total BS. The score is easy to manipulate by people who want to game the system yet punishes those who try to be as engaged as possible. 

 

The algorithm needs to be re-worked if Upwork wants honest hard-working people to stick around.

Community Leader
Craig G Member Since: Jun 20, 2017
2 of 69

@Anton A wrote:

 

 

The algorithm needs to be re-worked if Upwork wants honest hard-working people to stick around.


What reworking would you suggest that would be equally or more accurate than the current algorithm, would reward honest hard-working people in the way you'd like to see, and would not be amenable to manipulation by dishonest people?

Community Guru
Melissa C Member Since: Jul 22, 2017
3 of 69
Yes, I, as an honest, hard working freelancer would also like to know the answer to this question.

Also, not naming anyone here, but let's say a freelancer has a good JSS and is top rated, would you, OP say that that freelancer must be gaming the system? Because I would disagree.
Ace Contributor
Anton A Member Since: Dec 12, 2017
4 of 69

Hi Craig - thanks for the reply, here are my suggestions to Upwork for changing the algorithm:

 

1. Don't weigh it so heavily in favor of the client's rating, negative OR positive. A negative rating for freelancer means less work as an immediate and direct consequence. A negative rating for a client means they may potentially end an up with a lesser quality talent for their project, but even that is questionable. So first is a blow while the second is barely a slap. I am not saying the two should be equal, just that client's word carries too much weight, especially when it's a first-time client. There are clients that are simply not fair or reasonable in their evaluations, I want Upwork to account for that possibility. However, if a client is a long-term client, THEN I would weigh their rating MUCH HIGHER. Let's give higher credence to the people that consistently use the system. There should be some protection for a freelancer against a client simply turning out to be nuts, unreasonable, a jerk - which does happen. 

 

2. Reconsider how the badges are given out to prevent people from gaming the system and to account for actual effort and hard work. Right now it's possible to fake a few jobs, get a badge and 100% success score and then just sit there waiting for a client to scam. After that, just change your login name and repeat the process. On the other hand, loosen the restrictions on granting the badges to freelancers that work hard on multiple jobs. Effort and hard work need to be quantified and rewarded by Upwork, at the moment these are accounted for only indirectly. I don't know how to implement this, but Upwork is BILLION-dollar company, which has the resources and the brains to come up with something equitable.

 

Community Guru
Hasmik V Member Since: Aug 12, 2017
5 of 69

Anton,

 

My JSS dropped down to 70% because of one misunderstanding, but I'm still alive and here) I kept getting jobs, not much, but enough to make my JSS 86% and I know I can make it higher and become Top Rated. Wanna know how? There ARE people who pay attention to your cover letter, your work samples and experience, your professionality MORE than your JSS.

Ace Contributor
Anton A Member Since: Dec 12, 2017
6 of 69

Hi Hasmik, thanks for the note.

 

I agree, a negative rating is not the end of the world, but it's darn near close. 

 

As a freelancer, I have little time to deal with a huge hit to my reputation because of some idiot that doesn't understand, appreciate, or respect the system and may never even come back here.  I need to keep making money, right now. So rather than clawing my way back to the more favorable range, I may choose to go elsewhere. 

 

Despite all my crying, I do like Upwork and I am still here. I am posting here because I would like to improve the system. 

 

Community Guru
Charles K Member Since: Mar 6, 2017
7 of 69

I've had a lot to say on this subject in the past. And Anton, I can tell you what I have learned.

 

You are not going to get Upwork to change the system. There have been literally thousands of complaints about JSS and to my knowledge nothing of any substance has ever been changed.


So, whether your arguments have merit or not is not even relevant. It doesn't matter. At best they will say they are listening, but they aren't.

 

So my first piece of advice is to stop getting worked up over it. I know it's easier said than done, but I had to do it myself.

 

Second piece: find any idle and open projects, contact clients and close them amicably. (See below for why.)

 

You have only two choices: learn to work the system, or let the system work you.

 

"...for the next six months, I pretty much won't be working at all."

 

This is exactly what you DO NOT want to do.

 

The JSS is a percentage, and that means the law of large numbers comes into play. The fewer jobs you do, the less the JSS reflects your actual success. The more jobs you do, the better it does.

 

The fewer jobs you do, the more power you give any one client to affect your JSS. The more jobs you do, the less power each client has. "Turtling" will simply make it even more likely that you have JSS problems in the future.

 

The best protection against a low JSS is dilution by successfully completing a good number of jobs on a regular basis. If you do a lot of jobs, and you are actually good, your JSS will naturally go up, you'll get Top Rated, and everyone will live happily ever after™.

 

Obviously don't take on "just anything" in order to get a good rating, but yes, you may have to take on more jobs just to get some points in the win column. I know some very good long-time freelancers who mostly do work for established clients and they have to do this anyway on occasion. It's gaming the system, but the system is a game.

 

Craig, your question is a little facile, because the details of the algorithm are hidden from us, which makes providing truly informed suggestions nearly impossible. But I do know of one change that should be made: stop considering open-but-idle projects as "unsuccessful" and counting them against freelancers.

 

1. This is NOT an accurate reflection of the success of a project, especially if a payment has already taken place. It usually means the client has occasional work and finds it easier to keep the job open. Unhappy clients IME do not pay freelancers and then leave the project open. They either leave the project open with no payment or they pay and immediately close the project. Regardless of the exact percentages, this is an unfair assumption to make.

 

2. This being counted against JSS is the cause of countless complaints and confused posts from freelancers. It costs Upwork money in support costs. It costs Upwork good people like the OP.

 

3. It leads to wasteful and unprofessional "busy work" that makes EVERYONE look bad. I am now forced to periodically go through my clients and ask the ones that have been idle for a period of time to close their projects and then reopen them later if they need to. And each time I have to explain that this is necessary because of Upwork's system. Given that there's no actual basis for assuming that an idle contract is "unsuccessful," this is a pointless waste of my time and the client's time. It also probably reduces client retention on the platform, and it makes Upwork look bad (and clients have told me so.)

 

Counting idle projects against JSS is lose/lose/lose. But they won't change it. So I play the game.

Community Guru
Richard W Member Since: Jun 22, 2017
8 of 69

Charles, I agree with a lot of what you wrote, but disagree with a couple of things:

 

 

 

The JSS is a percentage, and that means the law of large numbers comes into play. The fewer jobs you do, the less the JSS reflects your actual success. The more jobs you do, the better it does.

 

The fewer jobs you do, the more power you give any one client to affect your JSS. The more jobs you do, the less power each client has. "Turtling" will simply make it even more likely that you have JSS problems in the future.

 

The best protection against a low JSS is dilution by successfully completing a good number of jobs on a regular basis. If you do a lot of jobs, and you are actually good, your JSS will naturally go up, you'll get Top Rated, and everyone will live happily ever after™.

 

Dilution will only help if you currently have a proportion of poor feedbacks that is higher than you expect your long-term average to be. If you've already reached your long-term average, then you can't dilute it.

 

On top of that, if you attempt to do a lot of jobs, you are more likely to take on jobs that don't result in excellent feedback. It's better for your JSS if you're very selective about what jobs you accept.

 

The best protection for your JSS (apart from being very selective and doing your best to make sure your clients are happy) is probably to become Top Rated, so you have the TR perk. So actually it might be good for Anton to be particularly cautious for a while, in the hope of getting TR.

 

That's the tactic I adopted recently, when my JSS fell to 92%. I was only 4-5 weeks away from getting TR, and decided not to risk taking on any new clients until I got there. (In fact I relented and took on one, who looked very safe.) Unfortunately for Anton, he probably has longer to go to get TR, but 3 months will be enough if he can stay above 90 JSS and meets the other requirements.

Community Guru
Jennifer D Member Since: Feb 15, 2016
9 of 69

@Charles K wrote:

 

...

Craig, your question is a little facile, because the details of the algorithm are hidden from us, which makes providing truly informed suggestions nearly impossible. But I do know of one change that should be made: stop considering open-but-idle projects as "unsuccessful" and counting them against freelancers.

 

1. This is NOT an accurate reflection of the success of a project, especially if a payment has already taken place. It usually means the client has occasional work and finds it easier to keep the job open. Unhappy clients IME do not pay freelancers and then leave the project open. They either leave the project open with no payment or they pay and immediately close the project. Regardless of the exact percentages, this is an unfair assumption to make.

 

2. This being counted against JSS is the cause of countless complaints and confused posts from freelancers. It costs Upwork money in support costs. It costs Upwork good people like the OP.

 

3. It leads to wasteful and unprofessional "busy work" that makes EVERYONE look bad. I am now forced to periodically go through my clients and ask the ones that have been idle for a period of time to close their projects and then reopen them later if they need to. And each time I have to explain that this is necessary because of Upwork's system. Given that there's no actual basis for assuming that an idle contract is "unsuccessful," this is a pointless waste of my time and the client's time. It also probably reduces client retention on the platform, and it makes Upwork look bad (and clients have told me so.)

 

Counting idle projects against JSS is lose/lose/lose. But they won't change it. So I play the game.


 Charles, I generally agree with you but I've also had a lot to say about *this* part of your argument Smiley Happy

 

As I've said previously, Upwork's position is that a *pattern* of jobs with no feedback or idle jobs indicates poor client and contract management, and as a client I tend to agree. This is why JSS is negatively impacted if a freelancer has a *pattern* of jobs with no feedback or idle jobs.

There is more to being a successful freelancer on Upwork than just winning a job and delivering the finished product. Client and contract management is an important skill for all freelancers. It starts before you even bid on a job (does the client have a great history of feedback, both given and received) and doesn't end after you have delivered the finished product.

 

There's clearly a difference in the success of a freelancer with 10 jobs total, all of which have been closed and had feedback left, and a freelancer with 100 jobs total, but only 10 closed with feedback.

 

It's also extremely easy (and even easier than ever now) for a client to rehire you (or anyone else) with a click. Some freelancers also set up a "retainer" type system with their long-term clients on hourly contracts, where they bill 15 minutes a month to keep the job from going stale.

 

As for the OP...You have 2 jobs where clients did not give you 5 stars, including one where the client didn't give you any written feedback and marked you harshly on multiple categories - and your own feedback to that client indicates there was some problem with the contract. It's very likely that client gave you very low private feedback.

 

Also, your statement that you won't get hired if you drop below 90% is factually incorrect, as plenty of freelancers have already told you. There are freelancers in these forums who have recovered from far worse, with far worse profiles and skills than you.

Community Guru
Charles K Member Since: Mar 6, 2017
10 of 69

It was a number of months ago, Jennifer, so forgive me if I've forgotten some of the details.

 

But I do distinctly remember being told directly by an Upwork mod that JSS can start to be penalized when a contract is idle for as little as 30 days. That's ludicrous.

 

As for a pattern, it happened to me with only two idle projects out of literally dozens completed in the preceding months, and none of those for very long.

 

I don't think that's reasonable.

 

If they want to penalize people who have half their projects sitting around idle for 6 months, fine. But that's not what is happening. And the "cloak and dagger" / "we must protect our seeeeecrets" stuff doesn't help.

 

Every two weeks a bunch of people wander in here wondering why their JSS went down, and in a lot of cases it is this nonsense. And for every one of those, there is a larger number also bewildered that don't post.

 

And yes, clients can easily rehire. But IME, only a very small number of clients know about this. You are probably more savvy than the average client around here. Even clients that do, again IME, have the general reaction of: "Okay, but why do I need to bother, why can't I just leave this open since we've already had a successful transaction and we work well together?"

 

I have no good answer to that question because there IS no good answer to that question. There's only a bad answer: "Because Upwork wants it that way, even though it makes no sense and is extra work for everyone."

 

We can bat this around until the cows come home, but the bottom line is still that I have to choose between risking my ability to get new clients and having to (IMO, unprofessionally) bother existing ones to close projects for no valid reason. If a project has had an exchange of work and funds and the two parties have it open and there are no disputes or problems, I see no reasonable basis for counting this as an unsuccesful project, especially given all the deleterious effects that come about as a result.

 

There's no value add here, and lots of value subtract. JMHO.

TOP SOLUTION AUTHORS
TOP KUDOED MEMBERS