Reply
cancel
Showing results for 
Search instead for 
Did you mean: 
Reply

Successful!? JS back-test and prediction :)

setumonroe
Community Guru
Setu M Member Since: Jan 26, 2014
11 of 21

I appreciate the positive feedback.

 

@ Cristian
Your point is well received. I will no longer consider it a failure since we actually know alot more about the metric in general.

 

@ Patrick
I know long-term contracts weigh heavily on the formula in a positive way.

 

Because of the many uncertainties, I started with something basic hoping to improve on it. What had been working for some time (and with a 'simple' profile) was:

 

completed jobs / n

where jobs:
successfully completed = 1
neutral = 0.5
not successful = 0
n = total jobs

Based on your suggestion it could be possible that there is now a gradient between 5 & 7.

However that would be impossible to prove, since any combination of numbers could be ballparked to give the right score in a test.

 

I believe that it has become more complicated and perhaps a job is no longer one variable in the formula; but multiple individual components.

---- easy like Sunday morning ----
suznee
Community Guru
Suzanne N Member Since: Aug 15, 2012
12 of 21

Setu,

 

I am not great with stats. Math was not a strong point in college for me. Although stats was. Problem with the JS is we don't have the unknown factors to contribute so the numbers will only show a portion of what really affects the JS.

 

I have bee watching mine the past couple months. I have had 3 jobs close over the last 3 two week spans. My public feedback with 5 star and good comments on all 3 jobs. I don't know numbers on the first contract closed but knew it would drop my score. I am sure you remember my post on the job that changed the description and parameters once I got the job. It did drop my score. Closing of first one of the three dropped my score from 100% to 94%,

 

Taking one new client for some reason raised it to 95%, Second to last client I discussed in detail what he left me privately. He left a 7 on Recommending me and not sure on the rest but it dropped my recommended score from 80% to 63%, and my JS to 93%. Last client I know the feedback was good public and private and it had been a long term (although Krisztina said something in another thread about long term having no impact when it was closed.

 

My JS today did not change at all. Recommended stayed at 63% and my JS stayed at 93%. There was absolutely no change with the 3rd closed contract with good feedback public and private.


Their system makes me wonder if factors calculated in do change to fit their needs.

 

It makes absolutely no sense that a JS would not change when it was closed since the other 2 did change and almost immediately after the contracts closed. When the last one closed nothing changed.

setumonroe
Community Guru
Setu M Member Since: Jan 26, 2014
13 of 21
Suzanne,

I must admit that it is far more complex for freelancers with many long term clients and few jobs like yourself. Also the point about unknown variables is a fact.

What I know is that long term contracts are special because they are "counted more than once." That's as much as I can explain. This is why the Mods always say it counts as positive.

For instance, consider if you had 4 long term contracts in a two year span, like you do. I can easily consider those counting as 8 contracts in the formula.

People with low contract count suffer greatly due to the law of averages, and this has been evident throughout the platform. If your calculation was conventional, you had 4 great contracts, then the suspect one. That would leave you reeling at 90% (neutral feedback) and 80% (not successful feedback). However I believe your long term contracts cushioned that effect by "making up numbers."

But these are all the reasons / possibilities why I said it is a futile task, to try and figure out the formula.
---- easy like Sunday morning ----
suznee
Community Guru
Suzanne N Member Since: Aug 15, 2012
14 of 21

Setu,


Thanks for the reply. I still do not understand why my JS would have stayed the same after a contract (long term) closed postitively. My metrics did not change my score did not change, and I understand my long term clients most likely cushioned the affects of the other scores, but it still makes no sense that stats nor JS did not change even by 1 on the last closure.

lysis10
Community Guru
Jennifer M Member Since: May 17, 2015
15 of 21

It's kinda a waste of time to try to unravel algorithms where you don't know the factors, they likely change during updates, and all you have to go off of is 1 data point -- your own profile.

 

Is the score relative to others? That alone would make it a complete wot since you can't track how others are doing.

kugrin
Community Guru
Krisztina U Member Since: Aug 7, 2009
16 of 21

yup

lysis10
Community Guru
Jennifer M Member Since: May 17, 2015
17 of 21

@Krisztina U wrote:

yup


 

lol! This is getting reposted. Excellent. 

spiderwisp
Ace Contributor
Tim H Member Since: Jul 7, 2015
18 of 21

Perhaps Upwork is taking a stab at using fuzzy logic.

sadanmasroor
Community Leader
Sadan M Member Since: Sep 3, 2010
19 of 21

Kristina I think the more appropriate defination of Algorithm now would be:

 

 A feature coded by Upwork Engineers which does not work the way it is suppose to work and is portrayed by Mods as a true implementation of what they were trying to accomplish. 

boshoffirene
Community Guru
Irene B Member Since: Feb 17, 2015
20 of 21

I have read through the threads here and smiled.

 

I am a qualififed assessor and moderator. What this means is that I am qualified to do assessments on job performance (and this is done according to a set list of criteria which is available to the person being assessed), and I am also qualified to moderate...meaning I am qualified to, in the case of a dispute, asses the assessment as done by the assessor and, in conjunction with both, reach a conclusion, i.e. whether the original assessment is a true reflection of the person's work performance, and whether the person doing the assessment was in any way biased. 

 

The key, here, in ALL instances, is transparancy. The reason for this is that it affords the person who underwent the initial assessment the opportunity of 'fixing' their mistakes, if applicable. 

 

The JS is, in effect, an assessment. And it is not transparant.

 

.........................................that's all folks Smiley Happy My two cents worth.

TOP SOLUTION AUTHORS
TOP KUDOED MEMBERS