uday_k
Member

Just had a chat with the support, can anyone explain the "System"?

Here's the transcript-

 

**edited for Community Guidelines**

 

Who's running Upwork? LOL

67 REPLIES 67
testsandeep
Member

Upwork is hiding everything under private feedback. Why are clients kind enough to give 5 star rating if they are not satisfied with the freelancer. It is Upwork's algorithm that messes things. Why do all client do that. Why do my clients also do that. 

 

I am sure that it is not the PRIVATE feedback that is lowering JSS. Something is wrong. I am sure. Upwork is hiding things and claiming that clients have given a negetive private feedback.

The reason you dropped in your JS score is because a contract or contracts dropped off the 2 year time frame which ended up giving more importance to a bad job star score you had in the past. If you continue getting good feedback it goes back up again quickly.

Thanks for the reply but that's a very generic reply that upwork supports keeps on giving. I'm assuming you didn't take the time to read the transcript.

 

Anyway can you please (if your're able to) take a look at my profile and explain this image with my profile in mind?

 

Screenshot 2015-09-20 16.12.49.png

If you just came from Elance you have to wait 2-4 weeks until your profile is fully assessed. 

Several months ago, I wrote that I was sure there was a 3 month line - but was told otherwise (only 6, 12 and 24 months).

Now it is proven in black and white.
---- easy like Sunday morning ----

Well I wouldn't be so sure if the system was designed by the system!

uday_k
Member

I was on odesk from the beginning but yes I had two contracts on Elance that I merged with this account, been more than 4 weeks 😊

The snapshots we take are of a rolling window, which means sometimes a contract can slip out of your best calculation, causing your score to change significantly without any recent activity.

 

Thanks for sharing this K M. At least I now know why my Elance stats took a tumble (which are also based on consistent bidding etc.) - I won't be merging those with Upwork until (if) I get them up again! 🙂

 

 


@Nichola L wrote:

....At least I now know why my Elance stats took a tumble (which are also based on consistent bidding etc.) - I won't be merging those with Upwork until (if) I get them up again! 🙂

 


Nichola,

 

There is a big disconnect between Elance levels and Upwork's Job Success score. Though I've argued that there is a productivity element to the JSS, I'm now thinking that it has more to do with consistency—as you say—of earnings, while Elance's stats, even after a partial correction, are much more driven by absolute revenue.

 

Hope this helps,

MM

K M Fazly R,

I don't respond to JSS questions anymore because of the many nuances, but I will give yours a go.

Based on your profile, you can have as much as 18 projects included in your JSS (2 year window) - it would be impossible to hazard a guess as to why it is 50% if this time frame was being used.

I can guarantee though that neither the 3 or 6 month lines are being used. If these were being used, the score is usually 100% or close especially in your case where only 1 or 2 projects would qualify.

Therefore your score is being based on your 1 year or 2 year lines. This also means that you have been consistently receiving poor private feedback for some time.

If you really want me to help, you could provide a history of your JSS movement over the last few months.
---- easy like Sunday morning ----


@Setu M wrote:

1) Based on your profile, you can have as much as 18 projects included in your JSS (2 year window) - it would be impossible to hazard a guess as to why it is 50% if this time frame was being used.

2) I can guarantee though that neither the 3 or 6 month lines are being used. If these were being used, the score is usually 100% or close especially in your case where only 1 or 2 projects would qualify.

Can we know that?? We dont see if there was a bunch of contracts that were closed without any payment hence would not show on the profile.....?

uday_k
Member

Thanks for taking the time to go through this, so here goes the story-
My score over the last 3 months went like this- 85/70+/63/56/50

I left Upwork around 2 years back after receiving my first negative review. I had a lot of inactive contracts then (clients not responding) some of them were ended by me to push the one negative review down but most of them ended automatically I believe during the first week last month. What's your take on this?


@K M Fazly R wrote:
Thanks for taking the time to go through this, so here goes the story-
My score over the last 3 months went like this- 85/70+/63/56/50

I left Upwork around 2 years back after receiving my first negative review. I had a lot of inactive contracts then (clients not responding) some of them were ended by me to push the one negative review down but most of them ended automatically I believe during the first week last month. What's your take on this?

 Well, if there are several contracts that negatively affected you RECENTLY (those with no feedback, especially when several) then those fall into all three windows (last 6 months, last 12 months and last 24 months so will count regardless of which rolling window is usedto calculate.

Well that could be a very logical explanation, thanks. 

 

But does that mean I'm done for the next 2 years? And that again brings me to the point, if the client inactivity results in a contract being ended due to inactivity, why is the contractor penalized for that? I had one client that opened 10 contracts and then disappeared!

I had to rewrite this several times - try to keep-up.

 

And this is working on simple logic, and what we do know about the system.

Starting with the contract in March 2015, if this contract was positive, it could stand on its own @ 100% since that was the first contract in a year. Therefore it must have been the 2 year line that was being used from the beginning, because the March contract was either negative or neutral.

 

If the recently completed contract in September was also positive, it could stand on its own @ 100% in the 6 month line, therefore it also must have been negative. Proof also comes from the fact that your score reduced to 50%.

 

But why such a sharp decline? Because of the lack of activity on the platform, therefore low number of contracts = law of averages. So each time you saw a decline, a positive contract from before you left the platform fell from the 2 year window, so you see a gradual reduction.

 

Adding to that, your last two completed jobs were not successful, or neutral.

---- easy like Sunday morning ----


@Petra R wrote:

Can we know that?? We dont see if there was a bunch of contracts that were closed without any payment hence would not show on the profile.....?


 I am assuming that the OP is telling the entire history and not wasting out time.

---- easy like Sunday morning ----


@Petra R wrote:

 Well, if there are several contracts that negatively affected you RECENTLY (those with no feedback, especially when several) then those fall into all three windows (last 6 months, last 12 months and last 24 months so will count regardless of which rolling window is usedto calculate.


People are overestimating the effect of no feedback on a profile. Trust me, there is almost no effect on your JSS.

 

Just survey many of the 100% profiles, especially since the mass closure recently, and you will see many profile with >10 consecutive no feedback contracts. I have seen profiles with almost half no feedback and JSS of 90%+.

 

I am not saying it does not affect a profile, but not what we think.

---- easy like Sunday morning ----


@Setu M wrote:

@Petra R wrote:

 Well, if there are several contracts that negatively affected you RECENTLY (those with no feedback, especially when several) then those fall into all three windows (last 6 months, last 12 months and last 24 months so will count regardless of which rolling window is usedto calculate.


People are overestimating the effect of no feedback on a profile. Trust me, there is almost no effect on your JSS.

 

....

Setu is absolutely correct (as usual). There is almost no effect on JSS as a result of contracts closed without feedback until occurrences reach an excessive quantity of contracts closed without feedback.

 

I had attempted without success to determine the break-point when contracts closed without feedback are determined by the system as "excessive" (identified as a trend). After reviewing 125 randomly selected profiles and noting some with essentially no feedback at all on closed contracts yet favorable JS scores, I concluded that concerning oneself about the lack of feedback is among the least metrics to cause concern. I do believe that private client feedback is among the top metrics causing JS scores to drop.

 

That is, client's "private" feedback that can only be analyzed by the "mad robots"? Really? Actually, I am beginning to think that the platform's secret, highly confidential "algorithms" are not actually algorithms at all; just a bunch of logic statements that are poorly integrated throughout the entire code base. Investors should have had the opportunity to review the books, which is a typical practice before committing funding. Myabe they should have insisted that their chosen experts review those so-called algorithms.

Ron aka LanWanMan


@Ronald T wrote:

@Setu M wrote:

People are overestimating the effect of no feedback on a profile. Trust me, there is almost no effect on your JSS....


....I had attempted without success to determine the break-point when contracts closed without feedback are determined by the system as "excessive" (identified as a trend). After reviewing 125 randomly selected profiles and noting some with essentially no feedback at all on closed contracts yet favorable JS scores, I concluded that concerning oneself about the lack of feedback is among the least metrics to cause concern. I do believe that private client feedback is among the top metrics causing JS scores to drop.

 

....I am beginning to think that the platform's secret, highly confidential "algorithms" are not actually algorithms at all; just a bunch of logic statements that are poorly integrated throughout the entire code base. Investors should have had the opportunity to review the books, which is a typical practice before committing funding. Myabe they should have insisted that their chosen experts review those so-called algorithms.


Ron,

 

Like you, I tend to trust Setu's analysis.

 

One difficulty with determining the effect of no-feedback jobs is that the criterion for "excessive" has been identified by a mod as a moving target: higher than average. I'm sure you can see the methodological speciousness of using such a target. Given its likely dubious value, we can be grateful it seems to carry little weight. I have no idea why Upwork doesn't scrap it entirely as high-cost and low-yield.

 

As for the investors, I argue frequently that we have every reason to believe they are actively directing, and quite possibly micromanaging, Upwork's strategy—with CEOs retained, let go, or promoted based on their willingness to cater to the investors' latest fads and fancies. Under this scenario, I suspect the investors are as much invested in the algorithms, their experts, and inadequate support for the code base as they are invested financially.

 

Best,

Michael


Douglas Michael M wrote in part:
As for the investors, I argue frequently that we have every reason to believe they are actively directing, and quite possibly micromanaging, Upwork'sstrategy—withCEOs retained, let go, or promoted based on their willingness to cater to the investors' latest fads and fancies. Under this scenario, I suspect the investors are as much invested in the algorithms, their experts, and inadequate support for the code base as they are invested financially.

 

Best,

Michael


Interesting theory, possibly factual, albeit definitely not good for Upwork's future or its members. Still, some of the negative things we see today were evident before the Elance / oDesk meger -- before oDesk was branded as Upwork -- before the new investors came into play.

 

It does appear someone, somewhere is indeed "micromanaging" those that need the organizational freedom to function efficiently and economically in order to establish and maintain workable policies, procedures, work instructions, and all aspects of the Upwork platform. This seems especially true with regard to engineers and programmers tasked with implementing those micro-managers' ideas no matter how impractical their wants and desires might be.

Ron aka LanWanMan


@Ronald T wrote:

Douglas Michael M wrote in part:
As for the investors, I argue frequently that we have every reason to believe they are actively directing, and quite possibly micromanaging, Upwork'sstrategy—withCEOs retained, let go, or promoted based on their willingness to cater to the investors' latest fads and fancies. Under this scenario, I suspect the investors are as much invested in the algorithms, their experts, and inadequate support for the code base as they are invested financially.

 

Best,

Michael


....Still, some of the negative things we see today were evident before the Elance / oDesk meger -- before oDesk was branded as Upwork -- before the new investors came into play.

 

It does appear someone, somewhere is indeed "micromanaging" those that need the organizational freedom to function efficiently and economically in order to establish and maintain workable policies, procedures, work instructions, and all aspects of the Upwork platform. This seems especially true with regard to engineers and programmers tasked with implementing those micro-managers' ideas no matter how impractical their wants and desires might be.


I'm not sure there are new investors, Ron, though I haven't checked lately. My guess is the same old investors from both companies—who I believe overlapped even before the acquisition—cut a deal at the club (or equivalent), and the corporate structures and PR spin followed from that.

 

But hey, what do I know? Never been in one of those clubs.

 

Best,

MM

I think I made that clear, I did have 10 contracts from the same client that ended without any financial activity.

stencil_media
Member

If the private feedback is anything like Elance, then the score they leave is from a scale of one to ten, with the question being 'how likely would you be to recommend the provider?' or something along those lines. Not sure how everything else is calculated, but if the private feedback is anything to go by, then it doesn't have to be either a 0 or a 100.

"Welcome, humans. I'm ready for you!"
- Box, Logan's Run (1976)


@K M Fazly R wrote:

I think I made that clear, I did have 10 contracts from the same client that ended without any financial activity.


Somehow I missed that somewhere.

Well enough said. That would be 10 negative outcomes.

---- easy like Sunday morning ----


@Scott E wrote:

Not sure how everything else is calculated, but if the private feedback is anything to go by, then it doesn't have to be either a 0 or a 100.


It is based on the NPS system.

0-4 = x

5-6 = y

7-10 = z

 

So most times it actually ends up being 0 or 100%.

 

Edited to add

I used to follow the JSS calculations keenly, and even predicted several several profiles for many iterations.

I have additional information via other means that cannot be disclosed here.

I was not told the actual formula, but I could calculate it from the info given.

I gave that up because I want to continue contributing here freely.

---- easy like Sunday morning ----

@K M Fazly R,

 

Work on your 3 month line going forward.

If you ensure you only get positive outcomes from now on, in no time you will be back to 90%+.

Just be strategic in the jobs you choose, and the past contracts will not influence your score.

 

Basically, you can 'abuse' the system to ensure it always chooses your most recent line, by making sure it is the highest one. By doing that, your 3 month line then becomes your 6 month line, which becomes your 12 month line etc. Without your past poor jobs ever affecting the JSS.

 

But of course, that means getting all positive outcomes, which is exactly what Upwork wants.

---- easy like Sunday morning ----

re: "I think I made that clear, I did have 10 contracts from the same client that ended without any financial activity."

 

How does that even happen?

 

If you have one contract with a client and it ends up that the client doesn't pay you, then the normal thing would be to no longer enter into a contract with that client.

He wanted to do SEO for 10 sites, targeting 10 localities, and opened up 10 jobs, one for each of them. Does that make sense?

re: "He wanted to do SEO for 10 sites, targeting 10 localities, and opened up 10 jobs, one for each of them. Does that make sense?"

 

K.M.:

Your explanation DOES make sense.

 

Unfortunately, this proved to be a tactical mistake.

 

One needs to understand that things that make sense to humans are often not understandable by computer algorithms.

 

Your explanation to us seems innocent enough, but to Upwork's automated algorithms, this doesn't look good.

Giving you the facts here! I'm mobile right now so not ready to peek into your account and see if you're from the technical side of things, from a company like Upwork, we do expect algorithms that can take more than direct variables into account. 

 

And bs while you're true about that being a tactical mistake, we did not have these JSS bs back then 😉

Yep, definately a "tactical" error; violates the typcial business 10% rule....

Ron aka LanWanMan
lanwanman
Member

I just took a quick look at the freelancer profile in questions. The job history is replete with jobs ended with less than 5-stars. As clients gave less than perfect public ratings, it's safe to say their private feedback was not the best either.

 

It's no secret that anything less than perfect, whether public or private, has a significant effect on the freelancer's overall JS score. One does not need any formal analysis to see the "trend" already established by that profile.

 

ADDED: Now, what effect if any does running an Agency on Upwork have on the Agency owner's freelancer profile? I closed mine long ago, and have no idea what the answer is. Also, what effect if any do freelancers working for that Agency have on the Agency owner's freelancer profile (especially one carrying a 65% JS score)?

Ron aka LanWanMan

re: "I just took a quick look at the freelancer profile in question..."

 

Robert, thanks for doing that. I took a peek, too.

 

Yeah... The original poster really needs to stop pestering Customer Support about his Job Success Score. There is no mystery here.

 

Let me say this GENERALLY and not about any specific contractor:

 

It is possible for a contractor on Upwork to incorrectly estimate their capabilities and get into projects that are over their head.

I can see what you mean too lol...but the question was the JSS system, not the profile or the trend set by it 😉

re: "I can see what you mean too lol...but the question was the JSS system, not the profile or the trend set by it "

 

JSS system will lower a contractor's scores for contracts that resulted in zero payment.

 

The way forward is to avoid ever having contracts with zero payment, and to work on achieving positive client results which result in clients leaving positive feedback (without actually asking clients to do so).

Lost me there.... Please note that in part, the "profile or the trend set by it" is what determines the JS score. No offense, but there are other job categories that might lead to better success.

Ron aka LanWanMan

You missed the initial part of the discussion, we were trying to figure out exactly how Upwork was calculating the JSS. So it's pretty normal that you got lost. No offense but if you can't point someone in the right direction, don't point at all 😊

Hi K M Fazly R! I doubt anyone outside of Upwork will determine "exactly how Upwork was calculating the JSS"; that is, determining all of the metrics, how metrics are weighted, and exactly what calculations take place. It might take over a year to recover from the damage already done. Pointing toward the right direction, maybe you can start over on another freelance site. In the freelance-world, any "failure" is not an option.

 

Sorry if it appeared that I "hijacked" your thread. I was just communicating with other very successful freelancers here on Upwork.

 

Best of Luck! No, seriously, best of luck!

 

Anyone wanna do the Interent shuffle?

Ron aka LanWanMan

Lol no one outside google knows how their algorithm works but still google is being gamed! And thank you man, finally something good came out yes? Sarcastic, but good ....keep up the 'trend' lol