Reply
cancel
Showing results for 
Search instead for 
Did you mean: 
Reply

Successful!? JS back-test and prediction :)

setumonroe
Community Guru
Setu M Member Since: Jan 26, 2014
1 of 21

Over the past month I was able to successfully back-test 5 different profiles accurately, obtaining the correct JS score figures for up to 4 previous iterations. Of course, I have used what information was given by the platform as the components of the JS score, and deduced what I figured was relevant (all important) in making a job "Successful."

 

With the help of an un-named "accomplice" providing additional profile information, I was able to reproduce 3 past JS scores in said profile; but more importantly, correctly predict the current score updated overnight Smiley Happy. It is simple, and may not contain all the possibilities - but it worked. Undoubtedly this is not fool proof and may be incorrect, but worth a try.

 

I have no idea what the repercussions to this may be by Management, but knowledge should be free.

 

Once again I will say, knowing the formula for the JS score calculation, can in no way lead to the system being "gamed" - since the freelancer has no way of knowing the private feedback left, and can only "game" the score by actually doing better work. Which is ultimately what Upwork touts it wants.

 

Microsoft Excel - JS score.xlsx (07-12-2015 08.33.58).png

---- easy like Sunday morning ----
monstersong
Community Leader
Patrick P Member Since: Jun 17, 2015
2 of 21

You're a GD champion, Setu.

setumonroe
Community Guru
Setu M Member Since: Jan 26, 2014
3 of 21
Only drawback is that it will only work on relatively new accounts that came about right before or after JS introduction. Since those accounts can be checked against the changes in the score.

For older accounts Upwork would have to provide us with our data.
---- easy like Sunday morning ----
setumonroe
Community Guru
Setu M Member Since: Jan 26, 2014
4 of 21

The JS score should be updated again in roughly 12 hours. I have been modifying and testing a basic formula to at least see if the primary components are correct. However posting after the fact cannot be a true test. A real test lies in making an actual public prediction based on past assumptions.

 

Therefore this is the predicted score using the same profile. Hopefully it will be successful; if not, no loss incurred.

 

Microsoft Excel - JS score.xlsx (07-25-2015 07.36.24).png

 

There can be three variants to the score:

Contract 15/07/2015 = 0.5, then JSS = 93%

Contract 15/07/2015 = 1.0, then JSS = 96%

Contract 15/07/2015 = 0, then JSS = 89%

---- easy like Sunday morning ----
jsutherland
Community Guru
Jean S Member Since: Oct 22, 2007
5 of 21

Thanks for all the hard work to get this information for us.

setumonroe
Community Guru
Setu M Member Since: Jan 26, 2014
6 of 21
Much appreciated Jean.
---- easy like Sunday morning ----
jsutherland
Community Guru
Jean S Member Since: Oct 22, 2007
7 of 21

How did you do with your predictions?

setumonroe
Community Guru
Setu M Member Since: Jan 26, 2014
8 of 21

Sorry to disappoint you Jean, but it was unsuccessful. The profile showed 95%.

 

I won't make any excuses. Using "jobs" as an item the formula worked on many profiles for many updates of the JSS. I believe this was changed (perhaps related to the massive glitch everyone encountered with the JSS last week), from "jobs" to "outcomes."

 

(successful contract outcomes – negative contract outcomes) / total outcomes.

This was not what was explained to me originally. And no, I am not pulling formulas out of my head.

 

There are two ways you can look at it:

[{(Job1 + Job2 +Job3) / Total} + feedback + disputes etc + longterm] = JSS

or

[(Job1 + feedback + disputes etc + longterm) + (Job2+ feedback + disputes etc + longterm) + (Job3 + feedback + disputes etc + longterm) / Total] = JSS

 

Either way, I have found some thing to be apparent:

 

Your Job Success Score on Upwork - Upwork Blog - Google Chrome (07-26-2015 10.40.08).png

Long term relationships don't really count in trying to understand the formula since you can have 100% JSS without it. Which raises another question, it poses a means of inflating the score.

 

Your Job Success Score on Upwork - Upwork Blog - Google Chrome (07-26-2015 10.42.09).png

Public feedback seems to contribute next to nothing and now I am worried about the actual role of private feedback. Consider these profiles:

 

Hire Data Entry Specialists - Upwork - Google Chrome (07-26-2015 11.01.41).pngHire Data Entry Specialists - Upwork - Google Chrome (07-26-2015 11.02.46).png4.png5.png

 

Remember there can be no private feedback without public feedback.

I was told however that feedback is collected throughout long term contracts - not just at the end. I don't know what form that takes, but I guess that is why we can have so many profiles like these.

 

Your Job Success Score on Upwork - Upwork Blog - Google Chrome (07-26-2015 10.41.31).png

We are told that these only affect your JSS when in excess. I have found that to be true. So therefore what exactly constitutes the core of the JSS calculation, if all these factors can be ruled out?

 

(successful contract outcomes – negative contract outcomes) / total outcomes.

 

What these outcomes are, how many per contract, if they are handled separately; I could not say.

 

In concluding, I know less now than I did a month ago. Advice to myself is to forget about figuring out what is important, much less the formula itself.

 

For the record, I took this on months ago because it is something that is not going away and affects every aspect of your success here - unless you have lifelong clients. I was also forcing more information out of management by doing things this way, unfortunately only a little was revealed. I know more about this topic than anyone here because I have had dialogue with the powers that be on this subject - just to indicate that I am not spouting garbage.

 

With all that I am putting it to rest because it is pointless.

---- easy like Sunday morning ----
christian1220
Community Leader
Cristian F Member Since: Jun 21, 2015
9 of 21

Thanks Setu for all your great work! To me, you had a very important part in better understanding how JS is calculated and what's in there.

 

Personally I am not that interested in JS stats / JS formula but I read all you wrote 3 times Smiley Happy

 

I say that making JS easier to understand is the real win for this community.

monstersong
Community Leader
Patrick P Member Since: Jun 17, 2015
10 of 21

Setu, could your results be due to a change in the way the formula is weighted. It seems to me one of the reasons the exact formula and values might be kept secret is Upwork then has the ability to make tweaks and changes with each update.

TOP SOLUTION AUTHORS
TOP KUDOED MEMBERS