Reply
cancel
Showing results for 
Search instead for 
Did you mean: 
Reply

Flaws and bane of job success rate metric in freelancer profile

rahulserver
Ace Contributor
Rahul P Member Since: May 13, 2015
1 of 22

Initially when we started odesk, there was nothing as like "Job Success" score. It was suddenly introduced and at a time when we did not expect that we would be counted in for jobs we don't complete. While that may sound malafide from a freelancer's end, we ought to consider the quality of clients you get on odesk. 

 

Clients wanting to get a 5000$ job done for 500$ or even 5$ are in abundance on this platform and its not surprising that you encounter one of such annoying people who post for some "easy job" (which more often that not is synonymous to "Low price" or even "No price" jobs). This is in particular for odesk, such things are very rare on elance/freelancer. 

 

Now coming to my agenda, I had about 98% job success rate. Suddenly I dropped two projects and that completion rate fell to 93%. I had to drop them because the client's requirements went on increasing with time. Each time I completed the work, their requirements became another, while their budget did not budge. 

 

And today its yet another shocker!!!!!!! My job completion rate suddenly fell to 88%. The past week's activity is such that I took 3 projects. One was hourly, 2 were fixed price. Hourly contract is on and still running. One fixed price contract got closed with a 5*.  And this results in me losing my top rated badge (as my completion rate now is 88%)!!! 

My profile has over 30 5* on upwork(with about 50+ 5* on elance rahulserver.elance.com ) yet this completion rate simply puts me out of competition on odesk. I was wondering whats the need to have such metrics and god knows how they calculate. I ain't gonna raise any support ticket as they would always give me this in reply: 

 

"

To calculate your Job Success Score, we subtract instances of poor performance from good performance, leaving your net overall performance. And to ensure we properly reflect your overall track record of performance, we look at how this calculation changes over time. Your Job Success Score is updated every two weeks, and it’s important to note any movement in this score (either up or down) reflects both recent activity and activity over a longer period of time. We look at trends over a 24-month period, meaning you won’t see work from years past surface in your Job Success.

Additionally, we consider all your oDesk jobs, whether or not work was performed. Jobs ending without receiving payment or with a full refund may be excluded from your profile history, but are still an important factor in calculating your Job Success Score. The goal is to capture a breadth of information not already reflected in your star-rating or profile.

We don’t look for perfection, but we do expect an overall track record of positive performance on jobs. One, two or even three client concerns or cancellations aren’t likely to have a significant negative impact on your score unless this is all the history you have. If there is a problem, don’t ignore it! Fix it, show us you can do better, and your score will improve over time. However, any significant negative trends could trigger an account performance review.

"

 

The flaws I see are as follows:

 

1) The client's feedback is not taken into consideration while calculating job success score.(i.e. all clients are considered equal, even if they are just time wasters)

2) Its often buggy and the exact method of calculation is not provided to us. Thats certainly not good as there are many people who depend on odesk for their living and they OUGHT TO KNOW how is this metric calculated.

3) Sudden introduction of this metric would kick out more experienced freelancers out of business. We should have been notified of this before this stat is incorporated into our metric. We would have been more careful in selecting clients, particularly on odesk/upwork.

4) If client does not end our feedback and disappears and/or leaves a feedback late, the system seems to deduct the job success rate from our profiles. Now its logical to think that what can a freelancer do in such cases?

 

 

I would conclude with only writing that odesk has done a lot for the clients. It should do something more to favor the freelancer as well, particulalry experienced freelancers who deliver quality to people.

 

rahulserver
Ace Contributor
Rahul P Member Since: May 13, 2015
2 of 22

And above all, its most pathetic to know that as a client, if we browse freelancer profiles, only the job success score gets displayed(check attached image screenshot).

 

This means all our effort, infact hard worked efforts to obtain 5* reputation on this platform are now in vane, just because some(forming significant percentage of people hiring on this platform in particular) clients simply screwed up. 

 

This is totally unfair and very hard on us who have worked day and night to gain this reputation on this platform.

 

ltolentino
Active Member
Luningning T Member Since: Jan 13, 2009
3 of 22

I share the same sentiments and views. It is a pity that the hard work we put up for several years were put to waste because of this new system.

piyushseem
Community Leader
Piush P Member Since: Apr 29, 2015
4 of 22

@Rahul,

 

I almost agree with you, and also want to add one more important thing as ZPEC.

 

https://community.upwork.com/t5/Freelancers/Need-to-improve-JS/m-p/79286#U79286

 

I think 100% freelancers who have JS as less than from 100%, are suffring from this common pain. I think it is not a right decision anymore... Please upWork mod look into this very important matter ASAP basis.

 

 

rahulserver
Ace Contributor
Rahul P Member Since: May 13, 2015
5 of 22

@Piyush and @Luningning

 

Thanks you all for the support.

 

I saw your post @Piyush and agree to the concerns you have raised. Its quiet dissapointing to see how people are so adamant to stick to this system(refering to reply in your thread by a community guru). Now there is no value for our stars. So I would then suggest have it totally scrapped from profiles. Thats quiet misleading for we freelancers as we can't seem to figure it out that what we need to ask from the client. It used to be 5* for a good work.

 

If someone has done just one job, and has 100% job success, his profile would be weighed greater than someone done 100 jobs and has about 80% or 70% job success. Even on elance, the job recommended score shows up if you have done about 5-6 jobs. So is it a move to penalize the veterans of this platform? 

 

I can stop working totally on upwork and continue well on elance. But with elance announcing that it would now merge with upwork, we are left totally perspiring and with no other hope.

 

garnorm
Community Guru
Garnor M Member Since: Oct 29, 2014
6 of 22

Hi rahul,
I'd like to respond to the 'flaws' you've outlined in your post. See my replies below, preceded by **:


1) The client's feedback is not taken into consideration while calculating job success score.(i.e. all clients are considered equal, even if they are just time wasters)

 

**This is incorrect as final Feedback ratings are included in your Job Success score.

 

2) Its often buggy and the exact method of calculation is not provided to us. Thats certainly not good as there are many people who depend on odesk for their living and they OUGHT TO KNOW how is this metric calculated.

 

**This is incorrect as the Job Success score calculations are stable. True that don't provide the exact calculation to users nor do we expect to do so. Keeping the exact calculation private is a necessary measure to ensure the measurement doesn't get abused by users who have less-than-altruistic objectives in mind. We do provide insight into what factors are included in Job Success here in our Help documentation.

 

3) Sudden introduction of this metric would kick out more experienced freelancers out of business. We should have been notified of this before this stat is incorporated into our metric. We would have been more careful in selecting clients, particularly on odesk/upwork.

 

**We have been leveraging the factors included in Job Success for quite some time, particularly over the past year as you started to see more notice and actions from our team when freelancers showed dangerously low rankings in these factors. If you've been working on oDesk for some time, you should have always been selective in your client selection and it should be no different now.

 

4) If client does not end our feedback and disappears and/or leaves a feedback late, the system seems to deduct the job success rate from our profiles. Now its logical to think that what can a freelancer do in such cases?

 

**I've responded on this concern before and we'll be adding it to our Help documentation as well. Unresponsive clients who don't leave final ratings are problematic. However, I'll reiterate that for the vast majority of freelancers these don't factor into your Job Success score because the "no feedback" jobs are such a small portion of your overall track record. In fact we specifically ensure these jobs have no negative impact on your Job Success score. It appears many freelancers may be overstating the significance of this scenario.
For others, where the job is ongoing, a long history of work with a client indicates to us there's a good relationship and you're gaining the benefit of this long-term engagement and not getting negatively impacted by a lack of final ratings.

rahulserver
Ace Contributor
Rahul P Member Since: May 13, 2015
7 of 22

@Garnor

 

Thanks for such clarifications.

 

But I would want to clarify you on my point 1)

 

"1) The client's feedback is not taken into consideration while calculating job success score.(i.e. all clients are considered equal, even if they are just time wasters)

 

**This is incorrect as final Feedback ratings are included in your Job Success score."

 

I meant that if I drop a project of a client with no review in his profile or even a 2* review in his profile, my job success rate gets affected equally.

 

Besides, My primary concern is that why has our public feedback reputation been stopped from being show to the client and instead only our completion rate is shown?(Check screenshot in my previous message). If this was to happen we should have been intimated before so that we could have been careful and focused on % of job completion rather than feedback ratings.

kissgabor2030
Community Guru
Gabor K Member Since: Apr 1, 2015
8 of 22

quoted from Rahul A:
"And above all, its most pathetic to know that as a client, if we browse freelancer profiles, only the job success score gets displayed(check attached image screenshot)."


This IS pathetic, indeed - and a first for me here.

What about freelancers, who mostly had fixed-price jobs? (as the number of jobs are NOT displayed either, only the hours worked)


So this means, that client decisions of hiring are mostly up to the JS percentage of the contractor.
This too explains, why do newbies with 100% Job Success (and with 1-2 total completed jobs) pull away jobs from really experienced contractors (but with less then 100% JS score). "And the truth shall set you free." -Jesus Christ

kochubei_valeria
Community Manager
Valeria K Community Manager Member Since: Mar 6, 2014
9 of 22

Hi Rahul,

 

Star rating is part of Job Success score but Job Success score also includes other important factors that reflect freelancer's overall performance on Upwork in a way that the star rating doesn't. That's why Job Success score is a more complete metric and is being shown to clients. That being said, client are still free to check star and written feedback left by previous clients before they hire a freelancer.

~ Valeria
Upwork
vraish
Active Member
Torrie R Member Since: Jul 23, 2017
10 of 22

Hello,

I have contacted Upwork Customer Support but I am not happy with my Job Success Score. I am relatively new to Upwork and my first JSS was 80%. I have had extremely positive feedback and multiple repeat clients. I think part of the problem was that I have clients who keep funding the initial project rather than creating a new job contract. I ended some contracts that were finished that the client did not end and provided that feedback to the client. All of my jobs with feedback are very positive. I had one job that I ended without pay due to an unresponsive client and then she assigned me a new contract which I am finishing up right now but the first contract ended without pay. I did not realize how to leave feedback initially but now I always give the client feedback. I do not feel that my JSS is representative of my work on Upwork and is harming my ability to attract clients regardless of the proposals I submit. I was told I had 80% positive feedback, 20% no feedback, and 20% poor feedback but that does not add up. I would appreciate a revision of my JSS. 

Torrie 

TOP SOLUTION AUTHORS
TOP KUDOED MEMBERS