🐈 Community

Job Success score sudden drop

Hey, have any of you guys experienced a sudden drop in you Job Success score? Like, a really huge drop out of nowhere. I had an above 90% Job Success percentage and was Top Rated back when the website was still oDesk when suddenly, my score dropped to 79% after the Upwork transition.

 

The thing about it is that I haven't had any completed jobs before the Upwork transition so there shouldn't be any changes to my score. I only have an ongoing project ever since December last year, in which the client consistenly delivers payments (iirc longer than 3 months contracts with regular payments should even increase the score). I'm completely at a loss as to what caused this sudden drop.

 

Upwork's help center has been hugely unhelpful so far; the guy assisting me made me wait for almost a month for the "investigation" and in the end, he just keeps on posting sections from Job Success score section from the FAQ.

 

Anybody else encountered this?

83 REPLIES 83
setumonroe
Member

Hi Christian, and Welcome to the Community.

When did your score change? Now, or a month ago?
---- easy like Sunday morning ----

Hey Setu! Thanks for the welcome. It happened around a month a go, right after oDesk transitioned into Upwork. 

prestonhunter
Member

re: "Upwork's help center has been hugely unhelpful so far"

This type of question is NOT what you should use customer service for.

You should bring questions like that to the community forum and look for answers by searching in Upwork information resources and searching in the forum.

I have the same issue my Job success Rate was 91% and it drops to 85% for no Reason. I have all 5 stars and 1 4 star Only. 

I do not understand after so much success I am not able to have 90% or above success rate.

@ Muhammad Mobeen A,

Since all of your completed contracts are within the 2 year window. The drop in score has to come from poor private feedback on some of your recent jobs. Remember that a job completed months ago can and does eventually affect your JS score today.

 

Also you may want to review your profile picture.

---- easy like Sunday morning ----


@Preston H wrote:
This type of question is NOT what you should use customer service for.

You should bring questions like that to the community forum and look for answers by searching in Upwork information resources and searching in the forum.

I disagree. Questions like that should be really easy for CS to answer and only CS can actually answer them. It's true that the community's vaguest guess is usually more on point than CS, but that doesn't mean it's OK, or that we should accept it.

Krisztina,

We may not actually disagree, if we're looking at the same question.

 

I'm not saying that customer service, as an abstract concept, shouldn't answer questions like this.

 

But, as a practical matter, it is not currently productive to take a question like this to Upwork customer service.

 

Whether or not an ideal customer service center should answer questions like this is another question entirely.

 

My advice was to the contractor who spent a month trying to get a simple question answered and ended up with notes copied from a FAQ. This is not a wise way to spend one's time.

 

My advice to current contractors on Upwork is to never use customer support for any reason whatsoever, no matter what, unless there is absolutely no possibility that what they need done can not be accomplished in any other way.

 

In order to save contractors time and frustration, my advice is that if any contractor has a question, bring it here or search for the answer in Upworks' help section and community forum.

Thank you for your valuable insight Preston. The first thing that popped up in my mind after the problem appeared is that this might be technical error. A few people mentioned they lost tracked hours after the update, which is not surprising considering their site (and company) went through a major overhaul. So I did what I thought was the best; contact their CS since the problem is most likely on their side. I believe that when asking technical problems like this, it is more appropriate to contact the company itself rather than the community.

 

BTW, having a 20 point drop in the job success score is not a simple problem. That is not something that can be solved simply by talking to other freelancers, who might possibly be experiencing the same problem. If it is a 3-5% point reduction then that is perfectly reasonable; having a huge drop in your success rate like that means that something terribly went wrong, especially if you haven’t done anything to trigger it.

 

I'm still waiting on their "investigation" because ultimately, I have no choice. I’m just here to find out if there are any similar cases with mine so I can have someone to compare experiences to. If you can enlighten me on what should I do with my problem, then I’ll greatly appreciate it.

  

 

Christian,

It was never my intention to suggest your problem is insignificant or simple.

 

But I doubt that the problem is actually with your account.

 

I do not know details of your account history nor do I have access to Upwork back-end source code.

 

But from what I have read in the community forum and based on what I have read in information released from Upwork, I believe that you are facing the unfortunate results of how the Job Score system works generally.

 

By that I mean that if another contractor had a work history and measured back-end data metrics identical to yours, that person's job score at this time would be identical to yours. I don't believe there is an account-specific glitch associated with your specific user account.

 

This doesn't mean there is not a problem. But I don't believe the problem is isolated to you as an individual, but that the problem is with the Upwork Job Score system working producing results that may not even be expected by the system's creators under certain situations.

 

But I could be wrong about this.

 

And for the record, a technical error or problem or bug is more likely to be reported to actual Upwork programmers and administrators if it is discussed thoughtfully in the community forum than if it is mentioned to a customer support representative.

 

There are different levels of employees that we're talking about in these two different realms. The customer support representatives are low-paid contract workers working from remote locations, often struggling to understand any English sentences other than those which use the most basic words and grammar. They rarely have very much familiarity with Upwork, but they have a few tools and buttons at their disposal and are asked to handle all kinds of customer service requests. Not only do they not actually have any vested interest in Upwork's technical infrastructure and overall platform being improved, they do not actually know enough about computer progamming and information systems to realize that the Upwork system CAN be changed. As far as they know, Upwork is simply the monolithic, unchanging and unchangeable system that they are paid to work with.

 

In the community forum, on the other hand, there are managerial-level employees who have actually been asked to communicate with Upwork users, provide genuine help where possible, and report to Upwork management and technical leads about trends, problems, issues, etc.

I see, thank you for offering a possible explanation. I agree, the way the Job Success score works is far from perfect; it's just really weird (and alarming) seeing your Job Success score go from 90%+ to 79%  literally overnight. Another thing to consider is that all my previous projects were completed successfully, with good enough private and public feedback to make me top rated prior to the update - so this really came out of nowhere.

 

CS won't even offer an explanation like you did, even though they forwarded the issue to their engineering team.  I do hope that someone (like the employees you mentioned) might address this problem to the management directly, should no reasonable explanation or solution arise from this discussion. 

Christian

 

There are many cases like yours but no one knows how it works no one can help you even who made this alogrithem may can not help you.The simple answer will be a bad private feedback or your past 24 month job history.

 

Preston H wrote: "I don't believe there is an account-specific glitch associated with your specific user account."

 

Re: But why? UpWork has tons of glitches. This might be just another one in a row.

 

Christian A wrote: "As you can see, I haven't done anything recently to lower my JS score. My last completed project was in Mar, so if the client gave me a big negative private feedback, I wouldn't have been in the Top Rated program by Apr."

 

Re: Unless you got a bad private feedback from some of these successfully ended jobs. The job in past which had a great both public and private feedback fell out from calculation in the meantime, and now all is left are jobs with worse private feedback than you would expect. I know... this sounds impossible.

 

Christian A wrote: "As I have previously mentioned before, a drop of 1-5% is perfectly acceptable to me, even without a reasonable explanation." 

 

Re: If they don't explain little drops, well, such behaviour leads to what happened to you. My JS dropped three weeks ago from 100% to 95 %. Yes, it's not a big deal, except I don't have any clue what's the real reason behind. The next time JS changes it can be anything from 0 % to 100 %. And I won't get any explanation. It's not ok. It's not acceptable. It makes whole work place unsettling. We should not spend our time asking ourselves what went wrong. If something did go wrong a freelancer needs to know what exactly so he can improve. Otherwise, it's just time wasting on banging head against the wall. 

____________
Don't correct my grammar!

Hey Vesna,

 

Please don't get me wrong, I am not saying that what they did is right. I'm just referring to my personal opinion that a decrease of 1-5% is more tolerable than a whopping 20% deduction. 

 

All I'm saying is that I would've preferred to have a smaller deduction like yours and wait it out until it is resolved, rather than taking a huge penalty which reduced my JS score dangerously close to account suspension levels. At least that way I could've applied to jobs that requires a JS score of 90% and above, while waiting for Upwork to fix my issue.

 

Of course, in the end, unexplainable drops in the JS score shoudn't even happen in the first place. 

suznee
Member

Christian,

 

On a good note after July of this year your JS should go way up as the one job you have with a low score should drop off. I am not sure anyone can tell you here why it changed after it changed to Upwork. Private feedback and not completing jobs on time are a big reason a lot of JS scores change. We all have no idea how we were rated privately. And from my understanding if the Client choose lower then 9 in the rating on would you recommend .... It will affect your JS.

Christian,

 

You are suffering from the law of averages. When you have a small number of jobs, one bad feedback can ruin you JS score. Remember, private feedback is not always similar to public feedback received.

 

This is just a mathematical calculation to show how low feedback affects a small number of jobs. Of course the actual JS score takes in a multitude of other calculations unknown to us.

 

Microsoft Excel - Book1 (05-30-2015 13.21.52).png

---- easy like Sunday morning ----

Hey Setu, thanks for the detailed explanation.

 

The thing is, between my original JS score and the sudden drop, I had no completed projects whatsoever to lower my score. Can a client can suddenly change their previous feedback? 

Hey Suzanne,

 

I do hope that after July something changes for the better. On a side note; can Upwork deduct JS scores from you for any other reason than client feedback? Because I haven't had a new job since last year so I have no recent negative client feedbacks that can affect my JS score. I only have an ongoing project since December last year.


@Christian A wrote:

Hey Suzanne,

 

I do hope that after July something changes for the better. On a side note; can Upwork deduct JS scores from you for any other reason than client feedback? Because I haven't had a new job since last year so I have no recent negative client feedbacks that can affect my JS score. I only have an ongoing project since December last year.


Not that I am aware of.

 

 

Here are the things that affect JS

 

What metrics does the Job Success score include?

 

We look at many performance indicators, including:

  • public and private feedback
  • successful completion of work
  • client complaints
  • responsiveness levels
  • missed deadlines
  • disputes
  • long term client relationships

    Additionally, we consider all your Upwork jobs -- including whether or not work was performed. Jobs ending without receiving payment, or with a full refund, may be excluded from your profile history. But they are still an important factor in calculating your Job Success Score. The goal is to capture a breadth of information not necessarily reflected in your star-rating or profile.

 

@Suzanne

 

"On a good note after July of this year your JS should go way up as the one job you have with a low score should drop off. "

 

Sorry to go off track a bit Suzanne, but what is the significance of July? Are ratings on a six-month basis?

 


@Nichola L wrote:

@Suzanne

 

"On a good note after July of this year your JS should go way up as the one job you have with a low score should drop off. "

 

Sorry to go off track a bit Suzanne, but what is the significance of July? Are ratings on a six-month basis?

 

Nicola,


JS is based on 24 months and he has a low score showing for 2013 which will fall off his JS in July.  

Sorry to be the bearer of bad news, but after July his JS score wont go back up by much. Think about it, the job that everyone is looking at as the "bad" rating job was actually not that bad since he had a 90% JS score all along. It was not until the last one (job) took effect in early April (coincidentally with the Upwork rebranding) that the score fell to 79%. Therefore I would bet my money that's the culprit. If you look at the calculations, I simulated the jobs with the timings.

---- easy like Sunday morning ----

To everyone suggesting that it might be a single negative feedback that pulled down the score, here's a quick timeline to show what happened to my account for this year:

 

Jan - Job success score is above 90%. No Contracts Taken.

 

Feb  Job success score is still above 90%. No Contracts Taken.

 

Mar - At the beginning of the month, client finished a long-time job which started Feb 2014, Job success score is still above 90%. No Contracts Taken.

 

Apr - Received an email notifying me that I am eligible to be Top Rated, and after completing my profile I became one. Job success score is still above 90%. No Contracts Taken.

 

May - Around May 5/6/7 (not sure exactly when) I noticed that my JS score suddenly dropped to 79%. No Contracts Taken. Posted the problem to Customer Service by May 8, still waiting for solution or reasonable explanation. 

 

As you can see, I haven't done anything recently to lower my JS score. My last completed project was in Mar, so if the client gave me a big negative private feedback, I wouldn't have been in the Top Rated program by Apr.

 

The only probable explanation right now I could think of is if a negative feedback from the job in Mar arrived almost two months late in May. Can that even happen?

 

 

Christian

 

The only probable explanation right now I could think of is if a negative feedback from the job in Mar arrived almost two months late in May. Can that even happen?

 

May be its not from the job you have low rating may be its your first or second job you have a bad private feedback because the system count your last 12 month jobs as well to see if you got a bad private feedback.As i said no one can confirm you what went wrong there are some hidden fectors which are killing freelancers.

Hey Iftikhar,

 

I have also asked that to the CS guy I was talking to, and he mentioned that the way they calculate the JS score didn't change when they rebranded into Upwork. It's still the same system before when they were oDesk. So taking that into consideration, my JS score shouldn't have changed since it was 90%+ before the update.

 

That leaves us three possible explanations:

 

1. CS guy is wrong, and something drastic changed with how they compute after the update. 

 

2. A possible negative feedback from my last job in Mar was delayed for almost two months, and arrived early May, coinciding with Upwork's launch.

 

3. A technical problem on their end.

 

Also, to prevent confusion, my JS score isn't exactly 90%, I just keep saying 90%+ since I don't recall the exact number. Based on my personal recollection it's even above 98%, but until someone from Upwork verifies it i'll keep it rounded down. All I know is that it's at least 90% since that is the minimum requirement to become top rated.

 

 

We can understand you but the system dont.You should have to digg this forum and you will find lots of complaints like yours.As I said before there are many cases but no answers.

 

The system drops my JS two times without any reason.From the start I was at 94% after 2 to 3 days it drops to 92% even i did not do any thing after that i have finished 2 to 3 jobs with 5 star rating and my JS jumps to 97% and again after some days it went to 94% without doing any thing so what you will call it.When I asked for the reason they just tell me may be I have done a sin(Refund or bad private feedback) in last 12 or 24 months which effect my JS which dont make sense.

 

The system always count the bad things not the good ones.

Thank you for understanding Iftikhar.

 

As I have previously mentioned before, a drop of 1-5% is perfectly acceptable to me, even without a reasonable explanation. I can patiently wait for them to refine their faulty system if that is the case. However, a sudden drop of around 20% (without any rational explanations whatsoever) severely eats away your credibility here in Upwork. It brings my JS score dangerously close to 70% (and according to the FAQS, anything below it might warrant a suspension) so I can't easily ignore this issue.

 

 

 

 

Yes the algorithm is too dangerous so I will suggest you to do some small jobs as well so you can have some 5 star ratings to maintain your JS just dont depend on one client only.

Christian,

 

I understand that you are confused and in disbelief, but look at the calculations again. What is unique about your situation is that there are a small number of jobs spaced far apart enough, to understand a bit more than normal.

 

A CS rep confirmed twice to me that if there is not enough "data" change in your job history, then there will not be any change in the score - because the score will not be updated. Think about your situation; with jobs months apart it would be easy to identify which client gave a negative private feedback if it was updated immediately each time - therefore not "private."

 

Upwork is committed to privacy and of course preventing accidental elucidation of the JS score calculation. Therefore it is beneficial to mask the changes in groups of jobs at a time. Also remember all the Mods here have stated that long term contracts are weighted more heavily on JS score, but this swings both ways. If you get a negative feedback on a long term contract it may also affect your score more than normal.

 

What you need to do from here is maximize your private feedback on the current long term contract you have now. That is the only sure way to maintain 90%+.

---- easy like Sunday morning ----

Once again, thanks for a well written explanation Setu.

 

So what you're suggesting is that they intentionally delayed the negative feedback for two months in order to protect the identity of the client who gave it?  (who, ironically, was still revealed in this case) As you have mentioned, my other jobs usually last for months and are spaced far enough apart; yet I have never encoutered this situation before. My JS score usually updates within a day or two after the completion of a project. 

 

If you don't mind me asking, does this normally happen to you, or anyone you know? If this is a common practice (the two month delay) then it might bring a new perspective to my issue.

Quote:" So what you're suggesting is that they intentionally delayed the negative feedback for two months in order to protect the identity of the client who gave it?  (who, ironically, was still revealed in this case)"

 

I absolutely agree with this-it serves no purpose really.

 

Mods confirmed in some other threads that recommendation score is updated on every five jobs and JS (job success score)  is updated biweekly (every two weeks.) In their blog post they announced some upcoming changes to JS as well.

Thanks Natasa for clearing that up!

 

If the job success score really updates every two weeks (as confirmed by the mods), then something's definitely up. 

 

@ Christian,

 

Can I ask you what you see at "last updated" date on your stats page?

 

My Stats - Google Chrome (05-30-2015 19.52.46).png

---- easy like Sunday morning ----

Hey Setu! 

 

It says "May 31". Good news is, my previous Job Success score is back to 100%. Someone might have noticed my issue and reported it directly to Upwork. CS on the other hand, messaged me something about their engineering team (not exactly sure what happend, still trying to get some details), so it might also be a coincidence.

 

Anyway, thank you everyone for your input!

Happy to hear your JS score is at perfect 🙂

I doubt however that there was anything involving the engineering team, since everyone's score got updated today/yesterday. I have seen several cases that CS resorts to the "engineering team" when there is no evidence they got involved.
---- easy like Sunday morning ----

re: "It says "May 31". Good news is, my previous Job Success score is back to 100%."

Maybe you secretly were a horrible person yesterday but today you became a wonderful contractor, and maybe Upwork magically knows this.

That must be what happened. Because the only other explanation is that Upwork programmers had a terrible mistake in their calculating of job scores, which they have now fixed.
kugrin
Member

Question to the mods: Does oDesk delay the inclusion of recent ratings in order to mask them? If yes, how is this handled when someone only has one or two long term jobs at a time? Could there be an extreme case, where a poor rating from say 2013 would only be included in the JS of say 2016, because too few jobs closed in the meantime and anonymity of the private rating could not be ensured? 

a_kazleva
Member

This job success is ridiculous. I have opened my account maybe two years ago and never used it. I started working on Odesk at the beginning of this year. Up until yesterday under job success it said I don’t have enough work history for accurate calculation which is true because I have only 4 jobs that are completed. I have other 4 jobs which are in progress and at least 3 of them will be in progress for some time as I am still working on them. So how come now (although I haven’t closed any contract recently) I have enough info for calculation and it is only 80%?

@ Krisztina @ Aleksandra

You are assuming that data is collected only at the end of the contract. So..... If everyone only had long term contracts, then Upwork would have no data.?
---- easy like Sunday morning ----

I am very interested in this.

 

An unexplained fluctuation between more or less 98% client happiness and 79% client happiness.

 

This is not good.  Someone at upWork should have, and should still explain this.  While I can understand the powers that be probably don't want to set off a wave of panic or I told you so's this is unethical, and unprofessional.

 

I note this happened about a month after Customer Support told me that there was a huge issue with the client happiness score on the engineering side, only to retract that after Garnor had someone speak to him.

 

That said I've posted some observations and conclusions about the client happiness calculation here:

 

https://community.upwork.com/t5/Freelancers/Reduced-Client-Happiness-Score-So-It-Happened-to-Me/m-p/...

 

I welcome your thoughts, criticisms etc. 

We are all in this together after all.

Upcoming Events
Oct 27
Nov 29