๐Ÿˆ Community
ยป Forums ยป Freelancers ยป "My Stats" page using 5 Week (35 day) Metric ...
Page options
setumonroe
Member

"My Stats" page using 5 Week (35 day) Metric !?

So I decided to do a bit of digging because of my stubborn scientific nature. But I should probably make some form of disclaimer before beginning .

 

I have enough evidence to suggest that the "Communication" and "Market Effectiveness" sections on the "My Stats" page are actually using 35 day metrics, instead of the 90 days shown on the site. It makes me wonder if this is an error, but more-so what about the elusive Job Success score? Is this also using a different time frame from what was stated?

 

My ultimate aim is to figure out a close approximation to the JS score formula, or at least the parts we have data on. This may be too big fr me however, but I will nibble at it every now and again.

---- easy like Sunday morning ----
17 REPLIES 17
setumonroe
Member

No takers on this one?
---- easy like Sunday morning ----

So as I promised, I am releasing more data as I figure it out on the whole JS score metric.

This is actual data collected from the site - No part of it was manipulated (addition, subtraction, count etc.)

 

This shows that Some parts of the Stats score are clearly not calculated on the 90 day span as indicated.

The Community seems to be uninterested in getting involved. However I will continue to probe.

 

The picture shows the actual "views" received on job applications. What is interesting is that it is clearly over a 5 week period (35 days). This is used to calculate those bars in the market effectiveness on the Stats page that says 90 days. I am curious to know what else is on a different time span.

 

Open image in 'new tab' to see details.

 

Microsoft Excel - Metric_test.xlsx (05-18-2015 09.10.25).png

---- easy like Sunday morning ----
kugrin
Member

The clients who would recommend you isn't a 90 day metric either, but it's calculated every 5 jobs. The JS isn't dollar weighted and supposedly a 12 but sometimes 24 months score, and none of it makes sense. Despite a several year strong track record of rolling out 5 broken features - 1 fix - and 7 new bugs, oDesk has an unwavering faith in its own capabilities, despite overwhelming evidence to the contrary, which is nothing short of disillusional at this point. 

Thanks for commenting Krisztina. People always mistake a valid complaint as mere criticism. There are serious problems with the system and we need them to be addressed, because it is affecting careers.

JS score is not dollar weighted, but public and private reviews are dollar weighted when being considered in the overall JS score. So in essence your JS score is actually dollar weighted. That puts most freelancers at a disadvantage.
---- easy like Sunday morning ----


@Setu M wrote:
Thanks for commenting Krisztina. People always mistake a valid complaint as mere criticism. There are serious problems with the system and we need them to be addressed, because it is affecting careers.

JS score is not dollar weighted, but public and private reviews are dollar weighted when being considered in the overall JS score. So in essence your JS score is actually dollar weighted. That puts most freelancers at a disadvantage.

Hi Setu, I completely agree with you that these issues must be addressed, I just grew tired of talking to a wall.

 

According to this post (click here), which was never addressed by the mods, but comes from a what I would deem trustworthy source, the JS and feedback seems no longer dollar weighted, otherwise a $25 job could not affect the JS by 6 points.

 

I do think that that the JS and all scores should be dollar weighted. I do not believe that this is putting anyone at a disadvantage, but is fair practice. Doing well on a $50k project should mean more than doing well on a $500 or $50 project. That said, it should also be normalized to suit both freelancer types -- those who work on short term jobs and those who work on long term ones. Right now, those who work many jobs seem to be at a disadvantage to those who only work few jobs, which makes no sense. 

I have seen at least 2 threads here where it was confirmed by a moderator that feedback is dollar weighted. I also spoke with CS yesterday and it was explicitly stated that feedback is dollar weighted.

 

I also do not support dollar weighted reviews ( you can read here) and think the opposite about which set of freelancers have the advantage. You must bear in mind that some jobs will never and can never value above a certain amount, however the level of expertise required is equivalent to any large contract job.

---- easy like Sunday morning ----

Still no official response on why the stated metric is not being used by the system. I thought this would have be an important point to warrant some investigation, clarification and change of metric or stated information on said screen.

I will continue to BUMP this thread.

---- easy like Sunday morning ----

Setu,

 

I appreciate all of the work you have put into your spreadsheet and findings. We understand your frustrations with the Job Success scores and lack of clarification. The team is working on making better information available that helps clarify Job Success scores and how it is calculated. We hope to have something available to you soon. 

 

Thanks, 

Lena

 

Untitled

Hi Setu,

I can confirm the Marketing Effectiveness and Communication stats are based on past 90 days while Job Success is based on past 24 months. I can also confirm, as Lena mentioned, that we're working on some additional Help documentation that can provide more details into what's being calculated in Job Success. Keep in mind however, we won't be able to share exactly how this is calculated because we need to avoid any manipulation of the score. I know that's not your intent and you're simply asking for more transperancy into this metric and we want to provide it. 

 

Finally, I won't be able to validate the metrics you provided here regarding your account. If you think there's a discrepancy in what's being calculated please contact our Customer Support team so they can check directly.

 

 

Garmor

 

I have a question Garnor Please see the attachment and please explain what will be the end of it?

 

Since its started it was 

 

You replied to 9 of 9 invitations to apply in the past 90 days:

 

Now its 3 of 3 as my portfolio is set to private and my clients direct hire me means without intevites so I would like to ask what will be the end of it when it goes to 0 of 0?

 

 

Will it still shows the complete green bars or just collpse to nothing?

 

Please shed some light.

 

Thanks

 

 

Hi Iftikhar,

 

The Communication metric is based on invitations you have received in the past 90 days. If there are none, you will see "You haven't received any invitations in the past 90 days."

~ Valeria
Upwork

Thanks Valeria but can you please confirm that my green bars will remain full green at "Alway"reply when it goes to 0 of 0 in 90 days?

 

Will it effect my JS if it went to 0 of 0 in 90 days?

 

Thanks

Valeria, "You haven't received any invitations in the past 90 days." That's nice to know. So if invitations received are not of interest to the freelancer, those invitations still have some value as long as replied to in a timely manner. I had been thinking of setting my availability to not available.

Ron aka LanWanMan

Thanks to both Lena and Garnor respectively for their replies. Lena took a more CS approach and sidelined the issue, while I hope Garnor is just repeating what was told to him by the engineers after asking. I understand that you both have a job to do and I am surely not trying to throw the community into anarchy, or I would have easily divulged the method used to obtain the back-end stats.

 

I welcome and look forward to the forthcoming detail on JS score and also understand that there are those who would endeavour to manipulate the system if allowed. However I can also guarantee the authenticity of the metric I provided, and that it was copied and pasted from Upwork without any other manipulation whatsoever. The pleasure would be mine to validate it for you or anyone else that were interested. It would not make sense taking it up with CS as I am almost sure it is a site-wide calculation (not a discrepancy).

 

I just wanted some public acknowledgement of either an error or misleading statements, on Upwork's part.

---- easy like Sunday morning ----
lanwanman
Member

Hi Setu! I admire your quest for transparency (my words; not yours) and your interest in research. I have performed various studies myself, regarding the methods used on this platform.

From spreadsheets to statical studies using statistical analysis software and even design of experiments (DOE). Unfortunately, my studies were always based on insufficient data, fragmented data, and a continual "moving target" (repetitive and at times conflicting changes).

 

In addition, I appreciate the replies given my Garnor, Valeria, Lena, and others. At least they are trying to provide meaningful clarification to members requests for information.

 

You "bump" it, I'll watch it....

Ron aka LanWanMan

I don't know if I have said this to you directly before Setu, but I am interested in this as well.

I saw you interest some time ago throughout your posts in the community Chad. As Ronald said, the data accessible is poor and incomplete. But even with incomplete data some things could be better understood. I think private feedback is the only real piece of missing data. The problem is the data is separated in chunks in each persons profiles, and I am sure Admin would boot us for any collusion.

I actually stopped collecting the little I was doing because this thread died twice as there was no response for days even after being "bumped."

I differ from most in that, along with complaining loudly, I think if strong Upwork data can be presented to show inadequacies in the system, it may bring about some change. My hope may be misplaced however.
---- easy like Sunday morning ----