🐈
» Forums » Clients » Re: "Top Rated" Freelancer Status = Complete ...
Page options
1ff3bc54
Community Member

"Top Rated" Freelancer Status = Complete Joke

What is the purpose of the "Top Rated" badge that some freelancers are given? 

 

Please see the attached screenshots showing an application from a recent job I posted.  The freelancer in question has "Top Rated" status (and even mentions it in her cover letter) with "100% Job Success".... but she has only completed 3 jobs on Upwork, receiving feedback scores of 1.95 and 3.00 on two of those jobs, plus another 4 jobs "in progress" with no client feedback.

 

How does someone with such a poor work history get a "100% Job Success" score and "Top Rated" status? 

 

These scores/satus is totally meaningless to clients like myself if it's not at all reflective of the freelancer's actual work history on Upwork.  Two poor feedback scores (out of 3 total) should not equal "100% success"....

25 REPLIES 25
holymell
Community Member

Are you sure those are all the reviews she has? She's worked 7,000 hours. Those are probably just the reviews you can see without hitting "view more". Did you try that?

It says she's worked eight jobs. Still, yeah, I also don't see why her score would be 100%. That is odd...

Edit to add- I would also like to see an explanation for that 100%. That should not be right.

I am 100% certain those are the only 7 jobs in her job history.  The screenshot shows everything her work history -- 3 completed jobs (5.00 score, 3.00 score, 1.95 score) and 4 jobs in progress.  She has worked nearly 3,000 hours at one of the current positions, and nearly 2,000 hours at another. 

 

As far as I can tell, the Job Satisfaction score is just completely meaningless. 😞

It's not meaningless. A lot of freelancers think it's unfair to them.

I have never seen a profile such as the one you posted here.

The ONLY way i can imagine her having a 100% JSS is if all those clients privately claimed they would recommend her.

HER JSS may be meaningless, but you'll see plenty of profiles like hers where the JSS is below 60%. I can only imagine this to be a bug of some sort.

It's not meaningless when you have freelancers sticking in contracts with clients who underpay them just because they fear the impact of that client on their JSS.

It's not meaningless when you have to refund thousands of dollars to keep an undeserved negative score from your profile. These are things that have happened to people here on the forums that I have seen. It means a lot to many of the freelancers who work hard to earn and keep a good score. We don't like the JSS. But it's for you, the client. Not for us.

So yes, in this one weird example HER JSS is absolutely meaningless.

For most of us a single poorly executed job would lower our score.

I can't imagine that this is not some kind of bizarre error.

Have you seen any more like this?

"HER JSS may be meaningless, but you'll see plenty of profiles like hers where the JSS is below 60%."
-- Sorry, but if someone can have terrible job feedback and still receive a 100% Job Success score and Top-Rated badge, then that means I can't trust those numbers and will have to ignore them when review applications.  

 

This is not a bug.  I've now come across multiple freelancers applying to my current job post who have several feedback ratings below 5.00 (all within the past 18 months) -- including one with very similar numbers to the applicant I shared screenshots of in my OP, and no agency associations -- that have a JSS score of 100% listed on their profiles.  

sam-sly
Community Member


Mark B wrote: 

This is not a bug.  I've now come across multiple freelancers applying to my current job post who have several feedback ratings below 5.00 (all within the past 18 months) -- including one with very similar numbers to the applicant I shared screenshots of in my OP, and no agency associations -- that have a JSS score of 100% listed on their profiles.  


 The profile you posted seems unusual, I've never seen that. Agency freelancers often have strangely high JSS but it sounds strange since her agency has no JSS.

 

About this comment... A freelancer's JSS has multiple factors including:

- Public reviews from clients

- Private reviews from clients (whether a client privately says he/she would recommend the freelancer)

- Long-term and repeat contracts with the same client

 

Also as mentioned, Upwork uses three windows:  6 months, 12 months and 24 months.

 

While some clients rate higher in public and lower in private, not all do this. Some may give a 4.something public and a 9 or 10 private. They are asking different questions. Such clients may even hire the freelancer on additional contracts.

 

Edited to correct a typo

cylver1z
Community Member

Hello Mark,


If a freelancer is exclusive to or the owner of an agency, the agency's JSS will be shown on the profile rather than an individual rating. Click here for more about agency's JSS. You may also know how the freelancer's JSS are calculated here.


Untitled

Thanks for the reply Ryan.

 

The freelancer in question is associated with an agency, but the agency has the following stats listed on its profile page:

 

  • Current Jobs: 0
  • Total Jobs: 0
  • Current Members: 5
  • Total Hours: 0
  • Total Earned: $0

There's no Job Success Score displayed on the agency's profile, 2 of the agency members listed have JSS scores below 80% (and two have no JSS scores at all), and the freelancer who applied to my job (the one shown in the screenshots attached to my OP above) is not listed as one of the five agency members.

When there's so much ambiguity about a Job Success Score -- and absolutely zero obvious relationship between a freelancer's job history and their score / Top-Rated status -- that renders the feature meaningless and unhelpful when reviewing applicants for a job post.

Ohhh. It's an agency profile. 

 

You might be better off hiring the independent freelancers, then. Their JSS scores will be a more accurate reflection of their job success.

 

This is certainly an odd divergence from the freelancer's end of things when our scores dip for reasons we can't figure out, and we think it's not fair. Interesting to see there's another side of this, where, apparently, Upwork just gives 100%JSS to someone who clearly didn't earn it.

 

I would actually like to see a logical explanation for this one.

 

Mark--perhaps you could consider hiring a freelancer who is not from an agency. Look at what their reviews say and ignore their score if you have to.

 

Or you could hire someone with a 50% JSS if you're so certain it doesn't matter.... 

 

"You might be better off hiring the independent freelancers, then. Their JSS scores will be a more accurate reflection of their job success."

-- No, the screenshots in my OP were from a freelancer's profile, not an agency.  I was simply responding to Ryan's post above, where he said that the freelancer's agency JSS would display on their individual profile.

 

I only hire indepdent freelancers, and as usual I specified on this job post that I would like to only receive applications from idependent freelancers, but there's no way to actually enforce that requirement so I end up with 50+ applications from agencies that I have to manually archive/reject.

 

(For the record, I'm not really new to hiring on Upwork.  I have made 117 hires, have 20 active contracts, and have spent over $350,000 USD on the platform in the past 3 years.)

Hi Mark,

 

Could you please pm me the profile in question so I could take a look if their score is displayed as expected?

~ Vladimir
Upwork

Thanks Vladimir, just sent you a PM with the link (plus a link to several other similar example profiles).  

 

Hi Mark,

 

Thanks for the pm and sorry for the late follow-up. I checked and can confirm the score displayed on the profiles you reported is correct and by design in all three cases. I can't share any specific details but can confirm the bad outcomes you pointed out happen either outside the two-year window or were removed from the freelancers' score calculation.

~ Vladimir
Upwork


 

(For the record, I'm not really new to hiring on Upwork.  I have made 117 hires, have 20 active contracts, and have spent over $350,000 USD on the platform in the past 3 years.)

Good! Then you should definitely know whether the JSS is meaningless or not. You've had a lot of experience with it, and it appears that you may be forgetting all of your vast,  prior experience because something touched a sore spot.

 

Seems that you viewed a single profile that is inexplicably odd, and then decided that the entirety of the JSS system must be meaningless because of that. 

 

That woman shouldn't even have a JSS. Clearly, something is wrong with HER profile, not with JSS in general. 

 

Is this the only profile you've seen that looks like this (with NO positive feedback and 100%), or is this an epidemic? I am interested in this, myself. I personally have not seen anything like this, so to me, this looks like a fluke. She really shouldn't even have a JSS based on how many jobs she's had and the time frames between them.

 

But I don't hire freelancers. You do.

 

For you to hire so many freelancers, and see only one of these, after all the profiles you've viewed, you would know better than I do if this is a fluke or if the entire system is flawed and meaningless. 

 

How many of these have you seen? Is one profile truly enough for you to say that all JSS scores are meaningless?

 

Edit- I am sorry, you did mention that you've seen several freelancers with lower than five stars with a JSS o 100%. Yes, that would be fine and normal so long as the freelancer has a long history on the site and has more five-star reviews than they do four or less. Private feedback also weighs more heavily than public. If a freelancer gets a three-star rating, but the client hits "10" on whether they would recommend that freelancer again, then their JSS probably won't be affected.

 

We as freelancers have been upset for a while that there are numerous aspects of our own Job Success Scores that are hidden from us, that we cannot know about. So maybe it will make you feel better to know that not every aspect of that freelancer's performance will be listed on their profile, but it will (usually, except in this case) reflect in their JSS? Several of us have had our scores dip without any logical reason. As a freelancer, I would love to agree with you that the JSS is meaningless, but I can't. It's not.

sam-sly
Community Member

Something does seem strange. But one thing, JSS uses three calculation windows: 6 months, 12 months, and 24 months. Upwork takes the highest score of the three. In the screenshot you posted, her poor feedback projects were older than two years (2014 and 2015). Those projects would be outside the calculation windows.

 

Ongoing, active long-term contracts count positively  towards a freelancer's score, otherwise, freelancers who work for one happy client longterm would lose their JSS. It looks like she was on elance or odesk? That was before my time as a freelancer, but I've noticed some of their profiles look different.

 

I admit I haven't seen anyone with such a high ratio of negative reviews with such a high score. Usually, the confusing thing is freelancers with all 5/5 and a low JSS (explained by lots of refunding and/or poor private feedback).


@Samantha S wrote:

Something does seem strange. But one thing, JSS uses three calculation windows: 6 months, 12 months, and 24 months. Upwork takes the highest score of the three. In the screenshot you posted, her poor feedback projects were older than two years (2014 and 2015). Those projects would be outside the calculation windows.

 

Ongoing, active long-term contracts count positively  towards a freelancer's score, otherwise, freelancers who work for one happy client longterm would lose their JSS. It looks like she was on elance or odesk? That was before my time as a freelancer, but I've noticed some of their profiles look different.

 

I admit I haven't seen anyone with such a high ratio of negative reviews with such a high score. Usually, the confusing thing is freelancers with all 5/5 and a low JSS (explained by lots of refunding and/or poor private feedback).


 

BUT: you need to have completed a certain number of projects within the past 12 or 24 months to even qualify for/maintain a JSS. One project won't do. 

This score is absolutely off and the (empty) agency profile doesn't explain it either.  I really hope Valeria comes along to shed some light on this mystery. 

 

FWIW, I had one disgruntled client 4 months ago and my JSS fell from 100% to 97%. More than 10 jobs later my score hasn't budged a single percentage point. That's how it works for most of us!


@Ela K wrote:

@Samantha S wrote:

Something does seem strange. But one thing, JSS uses three calculation windows: 6 months, 12 months, and 24 months. Upwork takes the highest score of the three. In the screenshot you posted, her poor feedback projects were older than two years (2014 and 2015). Those projects would be outside the calculation windows.

 

Ongoing, active long-term contracts count positively  towards a freelancer's score, otherwise, freelancers who work for one happy client longterm would lose their JSS. It looks like she was on elance or odesk? That was before my time as a freelancer, but I've noticed some of their profiles look different.

 

I admit I haven't seen anyone with such a high ratio of negative reviews with such a high score. Usually, the confusing thing is freelancers with all 5/5 and a low JSS (explained by lots of refunding and/or poor private feedback).


 

BUT: you need to have completed a certain number of projects within the past 12 or 24 months to even qualify for/maintain a JSS. One project won't do. 

This score is absolutely off and the (empty) agency profile doesn't explain it either.  I really hope Valeria comes along to shed some light on this mystery. 

 

FWIW, I had one disgruntled client 4 months ago and my JSS fell from 100% to 97%. More than 10 jobs later my score hasn't budged a single percentage point. That's how it works for most of us!


I agree it is surprising that she even has a JSS score. In another thread, a mod said that ongoing contracts somehow count if there are no closures. It sounded like in some cases, freelancers with active, long-term contracts don't have to complete 5 contracts within the two-year window as long as they have active long-term contracts. Hopefully, a mod will explain?

 

I think that makes sense when someone has a strong history then works for the same one or two clients continuously. I admit this doesn't appear to be the case in this profile since the freelancer had a higher than average ratio of negative feedback. But then, her negatives were outside the 6, 12 and if I recall even the 24-month windows. 

 

BTW, I agree with your comment about how usually one meh or negative review causes significant drops in JSS for most. I worry about that with every contract closure. So it is strange to see such a high ratio of negatives with such a high score.

"FWIW, I had one disgruntled client 4 months ago and my JSS fell from 100% to 97%. More than 10 jobs later my score hasn't budged a single percentage point. That's how it works for most of us!"

-- Well, then you should probably be very irritated at the fact that other freelancers with multiple sub-5.00 feedback scores have a 100% Job Success Score and Top-Rated badge, no?  I can't speak for other clients, but in the 151 times I've hired on Upwork in the past 3 years, I've almost always looked at JSS ratings as a key factor in screening applicants.

 

For the record, to those who are asking, I have now found at least 4 different applicants to my recent job post that have similar 100% JSS scores despite multiple sub-5.00 feedback ratings and no agency association.  It's not isolated to one profile/freelancer.  I've also seen applicants with several 5.00 feedback scores on past jobs, with no other reviews below 5.00, who have JSS ratings under 100% (ie. 88%, 97%, etc).  See screenshots showing one example attached to this post. 

(These screenshots are from an applicant to my job that DOES have an agency association, but the agency has no Job Sucess score listed on their profile, has only completed 3 jobs on Upwork, and 2 of those jobs have feedback scores less than 5.00 so there's no way that the 100% JSS on the freelancer whose profile is attached could be showing the agency JSS instead of her own...)


@Mark B wrote:


(These screenshots are from an applicant to my job that DOES have an agency association, but the agency has no Job Sucess score listed on their profile, has only completed 3 jobs on Upwork, and 2 of those jobs have feedback scores less than 5.00 so there's no way that the 100% JSS on the freelancer whose profile is attached could be showing the agency JSS instead of her own...)


If a client has a poor collaboration history then the client feedback will not count against the freelancer score.

 

In the screenshot you provided there is a 2.5 star given by a client with low given and received feedback on multiple jobs.

"If a client has a poor collaboration history then the client feedback will not count against the freelancer score.

In the screenshot you provided there is a 2.5 star given by a client with low given and received feedback on multiple jobs."

-- Thanks for the info.  Do you know what is considered "poor collaboration history"?  Because the clients who left those 2 negative reviews for the freelancer mentioned in my OP look pretty reasonable to me... not sure I'd consider their scores "poor".  See attached details of the two clients who left those 1.95 and 3.00 reviews on the freelancer's profile.  



@Mark B wrote:

"If a client has a poor collaboration history then the client feedback will not count against the freelancer score.

In the screenshot you provided there is a 2.5 star given by a client with low given and received feedback on multiple jobs."

-- Thanks for the info.  Do you know what is considered "poor collaboration history"?  Because the clients who left those 2 negative reviews for the freelancer mentioned in my OP look pretty reasonable to me... not sure I'd consider their scores "poor".  See attached details of the two clients who left those 1.95 and 3.00 reviews on the freelancer's profile.  



 As Samantha explained a couple of days ago, JSS only reaches back two years. The two negative ratings you pointed out were from 2014 and 2015, and so not a part of the calculation.


@Mark B wrote:

I can't speak for other clients, but in the 151 times I've hired on Upwork in the past 3 years

 

Very impressive how you went from 117 to 151 in less than 48 hours. That's quite an increase in hiring rate, from 117 over three years (an average of just over 3 per month) to 17 per day since you originally gave us your hiring stats in this thread.

Guess I was mistakenly referring to a different stat in my first reference.... I've posted 117 jobs and hired 151 contractors.  See screenshot attached. 

Either way, exact numbers are not really important... just wanted to make it clear I wasn't new to hiring on Upwork. 🙂

 


@Mark B wrote:


Either way, exact numbers are not really important... just wanted to make it clear I wasn't new to hiring on Upwork. 🙂

 


 Agree. They're just taking on a distorted importance because the way you keep throwing them around is clearly impacting the way that people interpret your posts and respond to you--and not in the way you intend.

Mark, you may also like to know (as I suspect most clients don't) that Top Rated freelancers are allowed to occasionally remove one private feedback, so it no longer affects their JSS. They can do this once every 3 months (or after another 10 completed jobs, if that takes longer).


@Richard W wrote:

Mark, you may also like to know (as I suspect most clients don't) that Top Rated freelancers are allowed to occasionally remove one private feedback, so it no longer affects their JSS. They can do this once every 3 months (or after another 10 completed jobs, if that takes longer).


 Not only that, there's 2 ways of doing so. One replaces the feedback with a message "This feedback has been removed." so a client can tell that feedback was removed. With the other method, the feedback is still visible in the freelancer's work history but excluded from JSS calculation, which can account for a freelancer being Top Rated with 100 JSS despite one or more low feedbacks, provided they occured 3 months/10 jobs apart.

__________________________________________________
"No good deed goes unpunished." -- Clare Boothe Luce
Latest Articles
Learning Paths