🐈
» Forums » Freelancers » Upwork - Job Success Thread **title edited**
Page options
jademilburn
Community Member

Upwork - Job Success Thread **title edited**

I want Community Forum Moderators to add to this thread, and really listen to what is going on.

 

I, like many others am sure have the same story.

 

I used to love Odesk, and got many job interviews each week. I had all 5 star feedback, and glowing written references from my clients. Then Odesk made alot of changes and introduced Job Success rate. I was shocked that mine was 67%. That was until I looked at other discussions and heard of other people who had five star feedback, lots of good references and yet had a very low job success rate.

 

This system is beyond unfair as

!)Upwork introduced new changes and a new feedback system. They should NOT include information that people believed was being entered anonymously, prior to the changes. The Job success rate includes information that clients did not think would be shown publicly. They left star feedback that would be shown publicly. You CANNOt use retroactive data for a new system.

2)Since the Job success rate has been started, I am now not getting invited to interview for ANY jobs, so how could I get my Job success rate, even if I tried?

 

You have made a very bad error with this.

 

Can every freelancer on this forum who wants to get rid of the job success rate feature, reply with 'Get rid', and lets see if they listen to us.

1,457 REPLIES 1,457


@shamontiel V wrote:

I was surprised when my score suspiciously dropped after working with one particular client who continued to pay me. I closed that assignment on my own because I refused to just let the client keep giving Upwork feedback without letting me in on the feedback.


It is important to keep in mind that Upwork's JSS calculations have factors in play that will lower your score even if your clients love you.  I went from 100% to 80% based on nothing my clients did.  Instead, it is Upwork who has decided that idle jobs (i.e., clients who still want to work with you in the future) are a negative.

 

There is zero evidence to believe that hidden client feedback is significantly negative when public feedback is positive.  It is just the justification Upwork rolls out to sidestep the issues with their secret scoring algorithm.  Your clients are the problem only if you know your clients are problematic.  Everything JSS related is Upwork's drama.

 

shamontiel
Community Member


@Darrin O wrote:

Instead, it is Upwork who has decided that idle jobs (i.e., clients who still want to work with you in the future) are a negative.

 


 Will you please elaborate on this? I don't believe I knew idle jobs were an issue for scoring. What if a client leaves an assignment open but only has occasional work?


@shamontiel V wrote:
Will you please elaborate on this? I don't believe I knew idle jobs were an issue for scoring. What if a client leaves an assignment open but only has occasional work?

I wish I could elaborate, but part of the problem with the secret calculations is that it's all a lot of guesswork and inference.  There are many threads that discuss the matter, so if you go through them you might come to a different conclusion than me.  Upwork wants contracts closed so they can get the feedback that feeds their JSS statistics wanking.  So after some time period of idleness, contracts count against you.  Fixed priced ones are auto-closed after 3 months, but hourly can go on indefinitely.  When exactly the penalty kicks in is unclear (and may differ for each type of contract).

 

If you are given work often enough, though, it might not be a problem.  Again, nobody really knows because Upwork is being tight-lipped on the issue.  They'll just tell you it's better to close the old contract as soon as possible and start a new one when more things need to be done.  But if you close too many, that will also count against you if the client can't be bothered to give feedback.  Upwork never says what quantity represents the "too many" pattern.

 

If that seems unhelpful or confusing . . . welcome to the club.  Smiley Frustrated

 

Woman Sad Darrin O., thank you for trying. I'm impressed that you tried to break it down for me and saddened that it's exactly as ridiculous as I thought it was. I have asked similar questions via customer service inquiries and gotten the same generic, political response. Upwork, if you are reading these messages, you are REALLY going to lose great (or at least hard-working) writers on this site. I'm not sure if you care or not because average/decent/OK ones will always pick up the pieces, but it is bizarre how indifferent Upwork is about this scoring mechanism.

 

If not for one particular client, I'd have left long ago because I genuinely do not like how the filtering process is with clients. I can't defend the scoring. I can't say what I did wrong. I can't even say, "Oh man, that one assignment was the one I really goofed on." I don't mind the constructive feedback. I welcome it. I've had clients give me a perfect score and I still ask, "What could I do to make it better?" I do, however, mind the hiding part. To be honest, if my one regular client ever closes the contract, there's about a 95 percent chance I'm outta here.

 

I cannot think of any other (successful) online business that is so one-sided. Not one.

Yeah, I honestly don't understand the hidden calculations. If a client gives me 5 - stars, why does upwork have to create hidden mechanisms (that have nothing to do with the opinion of who we actually deal with -- the CLIENTS) and create possible avenues of lowering the scores of good freelancers who will have no idea why their score has lowered. 

 

 

Upwork, this system is bad for BOTH freelancers and clients. If anything, make the calculations transparent so freelancers can improve upon the information. THIS at least gives a chance for freelancers to improve, hence, improving the client experience as well. 

 

I mean, I just had 2 clients tell me I'm great and want me to do all their writing work, but upwork is telling me i'm horrible? I mean, why sink a freelancer with 4 finished jobs on your site ( three with 5 star feedback, one with no feedback) because hidden calculations has give me a 50% JSS.  

 

Doens't make sense to me, but to be honest -- it won't sink me, but this set back is highly annoying.  

 


@shamontiel V wrote:

I can't say what I did wrong. I can't even say, "Oh man, that one assignment was the one I really goofed on." I don't mind the constructive feedback. I welcome it. I've had clients give me a perfect score and I still ask, "What could I do to make it better?"


I'm a problem solver at heart, too.  The problem here is that the stack ranking imposed by the JSS calculation just isn't valid.  The "solution" is only to get rid of it, which Upwork isn't going to do.  So the outward-facing problem for us to solve is simply moving to a site that uses a more scientific metric for evaluating people.


To be honest, if my one regular client ever closes the contract, there's about a 95 percent chance I'm outta here.

I just came here for 2015 as an experiment to see what the world of online work was like.  As I've noted in other threads, it has not been profitable (JSS is only partly to blame) and I'll be "leaving" myself at the end of the year.  Upwork's welcome to keep working at fixing their problems, but I came here to find interesting work, not feed their arbitrary statistical algorithms.  Your policy of working so long as there's work to be done is fundamentally the only right approach.

 

alekseevpavel78
Community Member


@Sandeep C wrote:

Atleast make JSS system fair. Atleast make users understand how JSS system works. It is difficult to work with a system which is not "transparent" "logical" and "reasonable"

 

PS: I know many will not agree because they have got the scores they want.


Absolutely agree!
Just make the jss system clear for understanding, and questions about the system will be gone.

 

the picture is the simple example how it could be.

 

jss_.jpg

One of the major problems which caused this dreaded JSS system, was the constant harassment clients faced (and still face) about 'x' feedback, and you need to change it to 'y'.

Therefore most clients just gave 5* feedback to avoid such annoyances, after getting similar experiences. This is the reason private feedback will always remain private - because of freelancers.

Did you know ~95% of freelancers had > 4.5* feedback. What do you think was the cause? By the way, this phenomenon is not limited or exclusive to Upwork. Studies have shown that all establishments that utilize * feedback have this same problem.

On eBay, I am always forced to choose between a vendor with 99.9% and 99.8%, both with > 10,000 reviews. How is that logical?

I am not saying JSS should not be changed, but we need to remember how we got here, why changes were made, and suggest practical solutions.
---- easy like Sunday morning ----

Setu,

 

it may be as well that if we found out that some of our clients give us bad private feedback we would not work for them and upwork would see their 10% jeopardized Smiley Embarassed

 

Anyway that of hiding the rating algorithm is an interesting approach. They value clients privacy but despise ours at the same time to the extent of forcing us to display our earnings to everybody out there (either that or we switch to private and become like a ghost...)

 

It is hidden because they do not want anybody to see what they are doing: artificially segmenting the freelancer performance sphere so that clients get the feel they can choose top/ elite freelancers. If they get 20 proposals and 10 of them are 5 stars they will not feel any confident. However if there are only 3, it is more likely they will enter into a contract (and with upwork).

 

M i s r e p r e s e n t a t i o n

 

Same thing goes with the blue bar and the same with applying a rating algorithm retroactively. They target is clear and they do not care about being fair or not.

 

Regards,

 

Javi 


@Javier M wrote:

Setu,

 

it may be as well that if we found out that some of our clients give us bad private feedback we would not work for them and upwork would see their 10% jeopardized Smiley Embarassed

 

Anyway that of hiding the rating algorithm is an interesting approach. They value clients privacy but despise ours at the same time to the extent of forcing us to display our earnings to everybody out there (either that or we switch to private and become like a ghost...)

 

It is hidden because they do not want anybody to see what they are doing: artificially segmenting the freelancer performance sphere so that clients get the feel they can choose top/ elite freelancers. If they get 20 proposals and 10 of them are 5 stars they will not feel any confident. However if there are only 3, it is more likely they will enter into a contract (and with upwork).

 

M i s r e p r e s e n t a t i o n

 

Same thing goes with the blue bar and the same with applying a rating algorithm retroactively. They target is clear and they do not care about being fair or not.

 

Regards,

 

Javi 


No you are wrong.

 

It's being done because "some" freelancers from certain countries ABUSED the old system.

 

This is the result of that abuse. 

I'd be pretty sure that SOME freelancers from EVERY country abused the system. The issue of abuse is there, no doubt, but there's no need to play the race card - that doesn't get anyone anywhere.

I completely agree with you about the blue bar.

 

It is completely off and I have no idea what their reasoning is being this. But then, very little on here works the way it is supposed to.

 

But the JSS is a direct result of some freelancers abusing the system. Not fair I agree.


@Setu M wrote:
One of the major problems which caused this dreaded JSS system, was the constant harassment clients faced (and still face) about 'x' feedback, and you need to change it to 'y'.

Therefore most clients just gave 5* feedback to avoid such annoyances, after getting similar experiences. This is the reason private feedback will always remain private - because of freelancers.

Did you know ~95% of freelancers had > 4.5* feedback. What do you think was the cause? By the way, this phenomenon is not limited or exclusive to Upwork. Studies have shown that all establishments that utilize * feedback have this same problem.

On eBay, I am always forced to choose between a vendor with 99.9% and 99.8%, both with > 10,000 reviews. How is that logical?

I am not saying JSS should not be changed, but we need to remember how we got here, why changes were made, and suggest practical solutions.

 

1. Will you elaborate on why there would be a need to change "x" feedback to "y"? What would be a reason that Upwork would ask clients to CHANGE feedback for a freelancer?

 

2.  I can't see the logic in ignoring honest feedback from clients because of freelancers? If they did a bad job, then they did a bad job. It's really that simple. Making bad feedback private will only make the next client equally confused about why the freelancer has all five star ratings but has a lower job score, I think. 

 

3. Are you sure about that percentage of 95%? That sounds like a percentage you made up as opposed to being accurate. I can guarantee you, as a previous Amazon/Half/Ebay seller and buyer that I have meticulously gone through reviews from other customers before purchasing something and pay even more care to thinks that I've bought. I pay as much attention to bad reviews as positive reviews and am especially grateful when buyers leave feedback. 

 

4. I could see why there are so many high scores for PAYING customers. If they PAID for the work, then I'm hoping they got the work that they were looking for. If they didn't get the work they paid for (or didn't pay for it), then it should be a surprise to neither the client or freelancer if the score is low. Honestly, I'd just expect bad feedback if a client is unhappy. 


@shamontiel V wrote:

3. Are you sure about that percentage of 95%? That sounds like a percentage you made up as opposed to being accurate. I can guarantee you, as a previous Amazon/Half/Ebay seller and buyer that I have meticulously gone through reviews from other customers before purchasing something and pay even more care to thinks that I've bought. I pay as much attention to bad reviews as positive reviews and am especially grateful when buyers leave feedback. 

 


 The Problem With Online Ratings

Olga was kind enough to post one of her short-lived documents.

---- easy like Sunday morning ----


@Olga Q wrote:

@shamontiel V wrote:

3. Are you sure about that percentage of 95%? That sounds like a percentage you made up as opposed to being accurate. I can guarantee you, as a previous Amazon/Half/Ebay seller and buyer that I have meticulously gone through reviews from other customers before purchasing something and pay even more care to thinks that I've bought. I pay as much attention to bad reviews as positive reviews and am especially grateful when buyers leave feedback. 

 


 The Problem With Online Ratings


Thank you. I read it. It just backed up my original opinion though, and I'm not sure if that was your intent. I don't just look at the ratings. I read WHY a reviewer gave a less-than-stellar score. This is why comments matter as much to me as a buyer as they do as a seller. I care about WHY someone wasn't pleased, not just that they gave something 1 star or 5 stars. That's yet another reason I think the job score leveling is not only sneaky (for someone who won't bother to read the comments) but not always a good predictor. I can't count the number of times I've seen bad reviews for a company and loooooved the place or thought a place/book/movie/etc. was terrible although everyone else loved it. I remember being hired to read and complete a book review for a publishing company, and the client told me, "But everybody else gave the book 5 stars. I'm concerned that you didn't like the book so much." I told the client that I wasn't going to lie and make the book seem like it was better than it was just to fit in with the popular opinion. So, yep, I stick to my original opinion about honest reviews. 

1. Will you elaborate on why there would be a need to change "x" feedback to "y"? What would be a reason that Upwork would ask clients to CHANGE feedback for a freelancer?

Upwork would not ask client to change feedback - it is against their policy. Freelancers as clients to change poor feedback.

 

2.  I can't see the logic in ignoring honest feedback from clients because of freelancers? If they did a bad job, then they did a bad job. It's really that simple. Making bad feedback private will only make the next client equally confused about why the freelancer has all five star ratings but has a lower job score, I think. 

Rather than making all feedback public, they decided to implement a new system (JSS). You are correct about the confusion that mis-match would cause - perhaps that is why average 5* feedback is no longer shown on your profile.

 

3. Are you sure about that percentage of 95%? That sounds like a percentage you made up as opposed to being accurate. I can guarantee you, as a previous Amazon/Half/Ebay seller and buyer that I have meticulously gone through reviews from other customers before purchasing something and pay even more care to thinks that I've bought. I pay as much attention to bad reviews as positive reviews and am especially grateful when buyers leave feedback. 

I could post the document, but the document is banned. Also I am sure our community manager mentioned this, but I can't be bothered to search for this reason. You are able to go through negative reviews because they are not hidden - they can be hidden here if you refund in full. I don't make up arbitrary numbers. Also you can do a bit of data mining yourself to get your own facts on this - then share it with us.

 

4. I could see why there are so many high scores for PAYING customers. If they PAID for the work, then I'm hoping they got the work that they were looking for. If they didn't get the work they paid for (or didn't pay for it), then it should be a surprise to neither the client or freelancer if the score is low. Honestly, I'd just expect bad feedback if a client is unhappy. 

I don't understand this point at all.

---- easy like Sunday morning ----


@Setu M wrote:

1. Will you elaborate on why there would be a need to change "x" feedback to "y"? What would be a reason that Upwork would ask clients to CHANGE feedback for a freelancer?

Upwork would not ask client to change feedback - it is against their policy. Freelancers as clients to change poor feedback.

 

Noted. I have only been asked to change feedback once by a client due to a misunderstanding. That actually made sense. 

 

2.  I can't see the logic in ignoring honest feedback from clients because of freelancers? If they did a bad job, then they did a bad job. It's really that simple. Making bad feedback private will only make the next client equally confused about why the freelancer has all five star ratings but has a lower job score, I think. 

Rather than making all feedback public, they decided to implement a new system (JSS). You are correct about the confusion that mis-match would cause - perhaps that is why average 5* feedback is no longer shown on your profile.

 

Not true. I can see five-star ratings on my public profile. 

 

3. Are you sure about that percentage of 95%? That sounds like a percentage you made up as opposed to being accurate. I can guarantee you, as a previous Amazon/Half/Ebay seller and buyer that I have meticulously gone through reviews from other customers before purchasing something and pay even more care to thinks that I've bought. I pay as much attention to bad reviews as positive reviews and am especially grateful when buyers leave feedback. 

I could post the document, but the document is banned. Also I am sure our community manager mentioned this, but I can't be bothered to search for this reason. You are able to go through negative reviews because they are not hidden - they can be hidden here if you refund in full. I don't make up arbitrary numbers. Also you can do a bit of data mining yourself to get your own facts on this - then share it with us.

 

So you came up with a statistic and then told the other readers to do their own research because you don't want to provide a credible reason for doing so? Um, OK. I can go through just about any video rating site, movie rating site, music rating site, book rating site, consumer seller site and company rating site and see just as many negative reviews as I see positive reviews. The reason I asked for your percentage analysis is because it sounds flawed, but I wanted to give you the chance to provide a credible link to prove it. I'm not going to go look for something I'm already 99.9% sure is false. 

 

4. I could see why there are so many high scores for PAYING customers. If they PAID for the work, then I'm hoping they got the work that they were looking for. If they didn't get the work they paid for (or didn't pay for it), then it should be a surprise to neither the client or freelancer if the score is low. Honestly, I'd just expect bad feedback if a client is unhappy. 

I don't understand this point at all.

 

In simpler terms, why would someone pay for something if they didn't like the work. Not sure how else to break this down. A paying client with private feedback clearly had something to hide, but if they were more interested in secretive follow-up responses than getting the assignment they wanted and STILL paid, the client loses, too.


 


@Setu M wrote:

 

 

3. Are you sure about that percentage of 95%? That sounds like a percentage you made up as opposed to being accurate. I can guarantee you, as a previous Amazon/Half/Ebay seller and buyer that I have meticulously gone through reviews from other customers before purchasing something and pay even more care to thinks that I've bought. I pay as much attention to bad reviews as positive reviews and am especially grateful when buyers leave feedback. 

I could post the document, but the document is banned. Also I am sure our community manager mentioned this, but I can't be bothered to search for this reason. You are able to go through negative reviews because they are not hidden - they can be hidden here if you refund in full. I don't make up arbitrary numbers. Also you can do a bit of data mining yourself to get your own facts on this - then share it with us.

 

 


 Setu didn't make the percentage up. If you wish to confirm your suspicion you may want to spend some time searching the web... The document is out there, and it shows your suspicion is wrong. 

 

EDIT -  "I'm not going to go look for something I'm already 99.9% sure is false." - Ok. Never mind.

 

 


 Setu didn't make the percentage up. If you wish to confirm your suspicion you may want to spend some time searching the web... The document is out there, and it shows your suspicion is wrong.


 Unless somebody is going to link me to this super secretive private document that I must look for all over the web, I'm not going to do it. Especially not when I can click on 15 sites right now and see sites flooded with both positive and negative reviews. I can even point to sites I work for right at this very moment that publicly showcase negative and positive feedback for both freelancers and clients. If the community manager would ban it and you can't link it, then tell me this: Why is the document so much more credible than the flood of websites full of public positive/negative feedback? 

@ Shamontiel,

I was told just this morning to avoid antagonistic discussions on the forum - and I will head the advice.

I have been warned too many times in one month to let you cause me to get banned. You can hold true to your opinions, that does not make them facts.

---- easy like Sunday morning ----


@Setu M wrote:
@ Shamontiel,

I was told just this morning to avoid antagonistic discussions on the forum - and I will head the advice.

I have been warned too many times in one month to let you cause me to get manned. You can hold true to your opinions, that does not make them facts.

To each his own. I don't understand why this document would get you banned. I would have to see it to understand why you and another user  are so secretive about it though so I'll leave it alone. I still stand by my opinion that I see sites flooded with negative and positive feedback from honest/grumpy/constructive reviewers. I just don't see any harm in Upwork doing the same. As far as the moderator, noted. I have already gotten a reply about my comments being edited because I mentioned a competitor to make a much larger point about improving Upwork, but the issue of me mentioning a competitor seemed to be a bigger deal. I'm still shaking my head about the moderator only feeling the need to say my name although another person was involved (and being extremely petty) in her comments, but apparently competitors are the bigger deal. 

 

Anyway, I'll leave it be. 


@Setu M wrote:

@ Shamontiel,

I was told just this morning to avoid antagonistic discussions on the forum - and I will head the advice.

I have been warned too many times in one month to let you cause me to get banned. You can hold true to your opinions, that does not make them facts.


I think her remarks have been extremely insulting and I think you're being baited.

 

I for one here am very happy that Setu helps us with understanding the numbers and the system here.

 

He's done more to help us than anyone else and doesn't deserve your remarks. 


@Jean S wrote:

@Setu M wrote:

@ Shamontiel,

I was told just this morning to avoid antagonistic discussions on the forum - and I will head the advice.

I have been warned too many times in one month to let you cause me to get banned. You can hold true to your opinions, that does not make them facts.


I think her remarks have been extremely insulting and I think you're being baited.

 

I for one here am very happy that Setu helps us with understand the numbers and the system here. He's done more to help us than anyone else and doesn't deserve your remarks. 


Nope. Not trying to be insulting. Just trying to read the other side of the argument to see if there's an entry that says the exact opposite of what I can see over a plethora of sites. Doesn't deserve my remarks? For asking to see the document they're referring to? It's insulting to ASK to see the source? If that's the case, every journalist in the history of mankind is "insulting." I'm not about to humor a conversation with you on something this silly. And I already said in a comment above that I'd leave the document query alone since he thinks he'd banned for linking it, so you can skip past the instigating now. Moving on!


@shamontiel V wrote:

@Jean S wrote:

@Setu M wrote:

@ Shamontiel,

I was told just this morning to avoid antagonistic discussions on the forum - and I will head the advice.

I have been warned too many times in one month to let you cause me to get banned. You can hold true to your opinions, that does not make them facts.


I think her remarks have been extremely insulting and I think you're being baited.

 

I for one here am very happy that Setu helps us with understand the numbers and the system here. He's done more to help us than anyone else and doesn't deserve your remarks. 


Nope. Not trying to be insulting. Just trying to read the other side of the argument to see if there's an entry that says the exact opposite of what I can see over a plethora of sites. Doesn't deserve my remarks? For asking to see the document they're referring to? It's insulting to ASK to see the source? If that's the case, every journalist in the history of mankind is "insulting." I'm not about to humor a conversation with you on something this silly. And I already said in a comment above that I'd leave the document query alone since he thinks he'd banned for linking it, so you can skip past the instigating now. Moving on!


Maybe you don't understand. Setu is a freelancer and NOT an employee of Upwork. He is not allowed to post his numbers here or he risks getting banned permanently. I now understand the gut feeling I had on the weekend when you applied to a job.

 



Setu wrote: Maybe you don't understand. Setu is a freelancer and NOT an employee of Upwork. He is not allowed to post his numbers here or he risks getting banned permanently. I now understand the gut feeling I had on the weekend when you applied to a job.

 



For the third (and final) time, I've already said "OK" after he said he'd be banned for linking the document. 
Journalism 101: “If your mother says she loves you, check it out”
He said he'd be banned. I said I now see why he won't link it. And you can move on. As far as applying to a job, I've been hired for three more after that (both on this site and another so...) that's kinda off topic and going into your "gut feelings" that have nothing to do with me nor are of interest to me. Did you want to stay on the topic of the comment board or bicker? If the latter, I'll pass and just ignore your replies going forward. 

That has not been my experience with public ratings. I see it on eBay as there are many power sellers and if they have thousands of feedback, it would likely be all good because this is what they do for a living, but I have seen lower scores on occasional ebayers.  But elsewhere...like the Apple App Store, or Amazon, or other similar sites, I see a LOT of bad ratings and poor feedback. I can't even count the number of apps I didn't download or things I didn't buy because of bad ratings. I will be launching my first book soon and have had zillions of conversations with people in groups regarding how to handle bad reviews and everyone seems to get them - even the greatest books out there with thousands of great reviews have hundreds of poor feedback as well. 

 

Regardless, it looks like the moderator was in here and pretty much told us to just deal with the JSS - meaning all this feedback from so many unhappy freelancers doesn't really matter - we should just read the blog because it doesn't matter if it doesn't work for us or we, collectively, don't like it. So...I am not sure if there is a purpose to it all anymore. 

The document is actually irrelevant.

You can search this forum and see many many threads about the abuse of the 5* feedback by a lot of freelancers.

 

The JSS score system was implemented to serve many functions, of course it has to serve the investors also. That is a given. What I comment on, is the purpose it served in weeding out those freelancers and the practice of blackmailing, hounding, and holding clients ransom for 5* feedback.

 

A freelancer can refund 100% and the review is hidden from everyone - I cant tell you why this was the case and why it cant be changed. Nevertheless 5* feedback did not cover the freelancers who had multiple disputes, hidden reviews, refunds, etc. These freelancers were ruining the system.

 

A client saw 5.0 * then got crappy or no work done and left the platform. Upwork had to address that. Clients work was being held ransom, in exchange for 5* - how could that continue.

 

I thinks we are discussing 2 different matters. Because my points have been clear as to why the 5* system no longer worked here. Straw men.

---- easy like Sunday morning ----


@Setu M wrote:

The document is actually irrelevant.

You can search this forum and see many many threads about the abuse of the 5* feedback by a lot of freelancers.

 

The JSS score system was implemented to serve many functions, of course it has to serve the investors also. That is a given. What I comment on, is the purpose it serced in weeding out those freelancers and the practice of blackmailing, hounding, and holding clients ransom for 5* feedback.

 

A freelancer can refund 100% and the review is hidden from everyone - I cant tell you why this was the case and why it cant be changed. Nevertheless 5* feedback did not cover the freelancers who had multiple disputes, hidden reviews, refunds, etc. These freelancers were ruining the system.

 

A client saw 5.0 * then got crappy or no work done and left the platform. Upwork had to address that. Clients work was being held ransom, in exchange for 5* - how could that continue.

 

I thinks we are discussing 2 different matters. Because my points have been clear as to why the 5* system no longer worked here. Straw men.


Noted. I'm not sure I'd even want the project if a freelancer is holding it for ransom, but I guess if they got down to the wire and REALLY needed, this could happen. Okay, makes sense. It's disappointing but it makes sense. It still doesn't make me like the scoring system though.  

No freelancer, well 99.9% 🙂 do not like the JSS system, that doesn't mean it is irrelevant.

We can expect some
***Edited for community guidelines*** soon.
---- easy like Sunday morning ----


@Setu M wrote:
No freelancer, well 99.9% 🙂 do not like the JSS system, that doesn't mean it is irrelevant.

We can expect some
***Edited for community guidelines*** soon.

*nodding head* I now understand what you mean. Sorry if you took my earlier queries to heart. It was a genuine question trying to understand the other side. I am open to hearing both sides of ANYTHING whether I agree with it or not. And yes, I've been one of the "edited for community guidelines" folks, too. I haven't come to the comment board often (actually in about a year, I think), but I get what you meant now. 

No worries.
See my motto I am trying to live by below.

And let me clarify the 5* comment further, since you have a journalistic mind (are a journalist). Star rated public review systems (all) tend to be skewed to the far right - for systems which allow mutual rating of vendor and buyer.

The buyer is not rated on Amazon, so there is no fear of reprisal, malice or spite. Or is there fear of future malice (like you cannot buy from 'x' store). So the buyer feels at ease in giving a more honest review.

That is not what happens on a site like this. Both vendor and buyer get to public rate each other blindly. So there is a tendency to rate high regardless of the experience, to avoid reprisal, future problems etc.

Ideas like - I wont give a client 2 stars because I may want to work with them again. Or what will other clients think? I may scare future potential buyers because I gave a negative review etc etc.

The interactions are complex here. But we do not talk out of thin air (at least me). There is data to back this up. And others have also sourced similar independent data.
---- easy like Sunday morning ----


@Setu M wrote:
No worries.
See my motto I am trying to live by below.

And let me clarify the 5* comment further, since you have a journalistic mind (are a journalist). Star rated public review systems (all) tend to be skewed to the far right - for systems which allow mutual rating of vendor and buyer.

The buyer is not rated on Amazon, so there is no fear of reprisal, malice or spite. Or is there fear of future malice (like you cannot buy from 'x' store). So the buyer feels at ease in giving a more honest review.

That is not what happens on a site like this. Both vendor and buyer get to public rate each other blindly. So there is a tendency to rate high regardless of the experience, to avoid reprisal, future problems etc.

Ideas like - I wont give a client 2 stars because I may want to work with them again. Or what will other clients think? I may scare future potential buyers because I gave a negative review etc etc.

The interactions are complex here. But we do not talk out of thin air (at least me). There is data to back this up. And others have also sourced similar independent data.

Motto: Yes, I know. Smiley Very Happy The first time I saw it, I assumed it was Bill Withers and then looked it up. Lionel Richie. Learn something new everyday. 

 

Buyers: I was an Amazon seller who rated buyers, too. It gave sellers the chance to explain why a buyer was giving them a bad review. I sold things on Amazon for many years. Half and Ebay, too, but Amazon the most. Or, maybe I'm mixing them up and its Ebay that lets sellers review buyers. One of them DEFINITELY does though. I haven't sold things online in a long time. I now just donate on FreeCycle, but my point is the reviewing is still there.

 

Independent data: I will look up your percentages exactly to narrow down this report.

 

Review backlash: I just can't agree to it. I can respect your opinion, but I do not agree with it. If I don't want to work with a client again I just don't care about repeat work with him/her. I posted on another forum how I wish we had the option to block clients altogether that we no longer want to work with. For [insert competitor here, I will not do so again] I have done that 7-8 times. It's not a regular thing I do. I've done a couple of hundred assignments before even getting to the point of having to block a user. But if we don't mesh well, then I would hope other clients would understand that. If I'm a hiring manager, I wouldn't expect every resume I come across to be loaded with bosses that were in love with their employees. I'd be prepared that some may like the applicant more than others. But it's really the feedback part that I think is important. I have made a point of going to places or buying things, even with negative feedback, because I just didn't agree with the people giving the place/product bad feedback. And then I actually liked it! 

@ Setu

 

i totally agree about the backlash. And especially about not wanting to rate someone badly because it shows up on your own profile.  That can easily be solved though just by not posting feedback on both profiles. A clients feedback on me would display on mine but not display on their own profile and vice versa. The double blind feature plus not worrying what other people will think if you give bad feedback will ease much of these issues and people will likely be more honest. And to differentiate from the best freelancers, to the not so great ones, just don't give an option to have bad feedback removed - roll it off their star rating after 3 or 6 months. Freelancers can reply to it and that should be enough. I don't think I would want to work with a client again, who I gave 2 stars, so that is a non-issue. 🙂 


@Setu M wrote:

A freelancer can refund 100% and the review is hidden from everyone - I cant tell you why this was the case and why it cant be changed.


I think a big underlying problem to how Upwork operates is their desire to keep clients anonymous.  It's hard enough to correlate the feedback of clients who are active on the site.  Trying to validate the feedback of clients who were refunded and subsequently left would be next to impossible.  The ability to abuse the system even with hiding is staggering, thanks to the JSS.

 


@Setu M wrote:
One of the major problems which caused this dreadedJSS system, was the constant harassment clients faced (and still face) about 'x' feedback, and you need to change it to 'y'.

That is not a reason to switch to the JSS.  There are many other mechanisms that can be used to solve the problem of problem freelancers.  Indeed, the JSS does nothing to stop the requests for perfect feedback.


Did you know~95% of freelancers had > 4.5* feedback. What do you think was the cause? By the way, this phenomenon is not limited or exclusive toUpwork. Studies have shown that all establishments that utilize * feedback have this same problem.

Ratings bias is indeed difficult to correct.  Stack ranking via the JSS doesn't solve that problem, though.  Clients would be better served if the star ratings had simply been normalized.  Or even some more complex calculation, but not a secret calculation.


On eBay, I am always forced to choose between a vendor with 99.9% and 99.8%, both with > 10,000 reviews. How is that logical?

Logic has nothing to do with it.  How is it a problem if you have two great providers?  Or 2,000?  That's at the heart of why the JSS is so badly screwed up.  Just get past your decision paralysis and pick one at random.  It is after the decision that the statistics can then be adjusted to be more valid, instead of doing what Upwork is doing by trying to make them predictive.


I am not sayingJSS should not be changed, but we need to remember how we got here, why changes were made, and suggest practical solutions.

**edited for Community Guidelines**

 

@ Darrin,

We will be back and forth all day, so I will leave that alone.

Would you be satisfied if you saw the JSS calculation for yourself? Inspected it, and saw it was adequate?
---- easy like Sunday morning ----


@Setu M wrote:
Would you be satisfied if you saw the JSS calculation for yourself? Inspected it, and saw it was adequate?

That'd be too big a presumption.  The very fact that it exhibits wild swings based on little to no change in the data indicates it is inherently flawed.  They're using it as a hashing algorithm (i.e., stack ranking) rather than a performance measure.

 


@Darrin O wrote:

That'd be too big a presumption.  The very fact that it exhibits wild swings based on little to no change in the data indicates it is inherently flawed.  They're using it as a hashing algorithm (i.e., stack ranking) rather than a performance measure.


 What formula? What data? You have seen neither the formula, nor the data, but yet you are sure it is flawed. How is that scientific, or correct?

 

 

---- easy like Sunday morning ----
Latest Articles
Top Upvoted Members