tsawicki
Member

Is the upwork freelancer private feedback question still 'how likely are you to recommend...'?

Is the upwork freelancer private feedback question still:
'how likely are you to recommend this freelancer to a friend or colleague?'


I'm pretty agitated to find that this was ever the actual question used for the private feedback rating.  There's a huge chance the client would take this... I don't know... at it's meaning - and not at all be giving a rating of the freelancer's work.
EDIT: I'm incorrect to say there is a 'huge' chance.   I do think it could be a better question to avoid the occasional confusion however.

What if the client doesn't know any other colleague's or friends that would have need of the freelancer's work?  

Is this seriously the question that determines the jss?

31 REPLIES 31
timefighter11
Member


Tim S wrote:

Is this seriously the question that determines the jss?


Any question you suggest?  

If  you have a bad experience with a client, would you recommend him to others?

Like hay everyone, I have wasted my time and energy with this client and I highly recommend you to work with this client if you want to experience the same. Would you do that?


Muhammad Zeeshan H wrote:

Tim S wrote:

Is this seriously the question that determines the jss?


Any question you suggest?  

If  you have a bad experience with a client, would you recommend him to others?

Like hay everyone, I have wasted my time and energy with this client and I highly recommend you to work with this client if you want to experience the same. Would you do that?


If you read the post in its entirety - I'm sure you'll realize you've missed the point.

tlbp
Member

The question is modeled after the Net Promoter Score method of assessment. This is a very common and well-known assessment method. You may not like it, but that doesn't mean companies aren't using it. 


Tonya P wrote:

The question is modeled after the Net Promoter Score method of assessment. This is a very common and well-known assessment method. You may not like it, but that doesn't mean companies aren't using it. 


That's kind of a non-answer, isn't it?   I don't like it - that's pretty obvious.  Companies are using it.  Yeah - including this one.   Also, the sky is blue, and the grass is green.

I'm saying this is not a good assessment method to use here.  It harms the freelancer.  It's a bad question to calculate 'job success'.

---

Net Promoter Score®, or NPS®, measures customer experience and predicts business growth. This proven metric transformed the business world and now provides the core measurement for customer experience management programs the world round.

The NPS Calculation

Calculate your NPS using the answer to a key question, using a 0-10 scale: How likely is it that you would recommend [brand] to a friend or colleague?
---

Points to consider here.
1.) A freelancer is not a brand.  Upwork is not trying to sell one freelancer to everyone.  
If the question said 'how likely are you to recommend 'upwork' to a friend or colleague based on your experience with the freelancer - then it would make sense.

2.) Upwork doesn't need to grow.  It's fricking enormous - and if it chugged along at its current pace, it would still be enormously successful.

This is a bad metric to score freelancers on a freelancer hiring platform.  It hurts the freelancer.


Tim S wrote:

Points to consider here.
1.) A freelancer is not a brand.  Upwork is not trying to sell one freelancer to everyone.  
If the question said 'how likely are you to recommend 'upwork' to a friend or colleague based on your experience with the freelancer - then it would make sense.

2.) Upwork doesn't need to grow.  It's fricking enormous - and if it chugged along at its current pace, it would still be enormously successful.

This is a bad metric to score freelancers on a freelancer hiring platform.  It hurts the freelancer.

Upwork is a brand. And we are their stock in trade. So we are the brand, because we are the product.

Upwork been successful in capturing the lion's share of its market. It has not been successful in over ten years of trying to turn a profit. It does need to grow, or move, into a profitable market.


Upwork is a brand. And we are their stock in trade. So we are the brand, because we are the product.


-- Yes, they are.   But we're not measuring performance for upwork employees providing a service to many customers.  We're measuring freelancer client agreements with - for many - infrequent data points.   I average about 6 large jobs a year.  If one goes sour, and one goes so-so - even if the other four are perfect, I'm considered a liability by this system - regardless of the amount of revenue I generated for upwork.


Upwork been successful in capturing the lion's share of its market. It has not been successful in over ten years of trying to turn a profit. It does need to grow, or move, into a profitable market.

If they're not making a profit, they're doing something very wrong.  The platform should be generating a profit if it's taking 1/10 - 1/5 the profit from most contracts.

It's nonsensical that they could 
a.) have the lions share of the market and
b.) not be making a profit.  
If that's true - then their business model is seriously flawed - and this system isn't going to help - it's going to remove potential sources of revenue.  


wrote:

If they're not making a profit, they're doing something very wrong.  T

For the majority of its life, Amazon did not make a profit. 

 

Reframe your non-point again, maybe?


Your JSS is low because you consistently end up with clients who aren't happy.

 

 


and this system isn't going to help - it's going to remove potential sources of revenue.  

How so? It's not as if clients are saying "Oh, this freelncer has a poor JSS, so I'll won't hire anyone"

Clients will say "this freelancer has a low JSS, but I'll give him a try because his proposal and interview were great" or "I'll hire another freelancer with better metrics"

 

Freelancers don't drive revenue. Freelancers are a penny a dozen.

Clients drive revenue.



 

Reframe your non-point again, maybe?

 


Petra, 
You never fail to dissapoint.  I was awaiting **Edited for Community Guidelines**

I didn't ask about my JSS, not once.  That's not the discussion here, and it's not part of any of the points I was making.   I've seen you in that kind of argument - on the pro-JSS side in the past, and I'm not going to bother debating you on your assessment of my score.  Thanks though. 

Amazon is obviously incredibly successful.  Was Bezos independantly wealthy before Amazon?  Someone made money - even if the company did not at first do so.  Someone is making money here at upwork - and if not - yeah I'd argue they're doing it wrong.

You're correct about clients driving revenue.  I think the freelancers shouldn't be treated like disposable units - even if they are a penny a dozen.  And that's just my opinion.

The feedback question is flawed.  The jss system is flawed.  That's my point.

Also - please be a little kinder.  People look to you for answers.


Tim S wrote:



Petra, 
You never fail to dissapoint.  I was awaiting the snark and like clockwork, here you are, mean-spirited as ever.
Also - please be a little kinder.  People look to you for answers.


Funny, only one person in this thread threw around personal insults, and it wasn't me.

I give answers. I just don't always sugarcoat them.

 


Tim S wrote:



 I've seen you in that kind of argument - on the pro-JSS side in the past

I am not "Pro-JSS" as such. It is the system we have and it's not going away any time soon. So I take time to understand how it works and frame my answers according to what it is, not how I would want it to be.

Every such system will have flaws, but understanding it helps avoid common pitfalls.

 

I didn't personally attack you, so maybe you could try not to attack me persoally simply because you don't like my point of view, which comes from experience, or the way I express it.

 



"I am not "Pro-JSS" as such. It is the system we have and it's not going away any time soon. So I take time to understand how it works and frame my answers according to what it is, not how I would want it to be."

That's probably the realistic standpoint, I think.  Ultimately - I think you might have the right approach here - in that it's just not going to change - so the only practical thing to do is to work within the system. 

I think the fallacy lies in my thinking anything posted here would ever change anything about the jss.

That being said - this is the community forum - if there's every going to be a consensus against how things are going - it's here.  But I see the wisdom in what you're saying.


Tim S wrote:

I think the fallacy lies in my thinking anything posted here would ever change anything about the jss.

That being said - this is the community forum - if there's every going to be a consensus against how things are going - it's here. 


I don't really think there is ever going to be a consensus about the JSS. As far as I am concerned it's a necessary evil. I take care to understand it and my personal main take-away is to choose clients and projects very, very carefully. A high and safe JSS is as much determined by the contracts I turn down as by those I accept. I don't work with clients who raise red flags and I don't take contracts I don't know for a fact  I can complete perfectly and within the deadline, short of being hit by a bus.

 

The thing is: The JSS largely does what Upwork needs and intends for it to do. Not perfectly, and there will be collateral damage, but it does it as well as it needs to do. It does it much better than the ridiculous star system did which bunched everyone at 4.9 to 5 stars.

 

But there have been several situations where Upwork did roll back things about which there was a well expressed and reasoned consensus. 

 


Petra R wrote:

A high and safe JSS is as much determined by the contracts I turn down as by those I accept.


I think this is a particularly important piece of wisdom. I'd even like to see this become a prominent bullet point in the jss help documents.

But there have been several situations where Upwork did roll back things about which there was a well expressed and reasoned consensus. 

 


 

Thanks for pointing that out - maybe there's some merit to bringing my point up again in the future - but expressed in a calmer way.



Funny, only one person in this thread threw around personal insults, and it wasn't me.

I give answers. I just don't always sugarcoat them.

 



I think some find that abrasive.  I find that abrasive.  I can respect the opinion that I'm being overly sensitive here.

"I didn't personally attack you, so maybe you could try not to attack me persoally simply because you don't like my point of view, which comes from experience, or the way I express it."

When you bring up my client's dissatisfaction with my work as the crux of the matter - that does feel like a personal attack.  When I'm trying to speak objectively about a point, it does make it personal when info is taken from my personal profile to illustrate a point.


Tim S wrote:


Funny, only one person in this thread threw around personal insults, and it wasn't me.

I give answers. I just don't always sugarcoat them.



I think some find that abrasive. 


I didn't swear at you, didn't call you names and didn't get obnoxious with you. You can't swear at people, call them names and then go all snowflakey when they call you out on it. Note how I STILL didn't swear back at you and didn't call you names.

 

Petra, I didn't swear at you.  

I think whatever the guildelines here are to edit out comments might have been used a tad liberally.  The edited out part was the word**Edited for community guidelines**



Still not a productive word perhaps - techincally still an insult.

 

But bleeping it out makes it seem like I took this to a whole different level.  Agreed?


Tim S wrote:

Tonya P wrote:

The question is modeled after the Net Promoter Score method of assessment. This is a very common and well-known assessment method. You may not like it, but that doesn't mean companies aren't using it. 


That's kind of a non-answer, isn't it?   I don't like it - that's pretty obvious.  Companies are using it.  Yeah - including this one.   Also, the sky is blue, and the grass is green.

I'm saying this is not a good assessment method to use here.  It harms the freelancer.  It's a bad question to calculate 'job success'.

---

Net Promoter Score®, or NPS®, measures customer experience and predicts business growth. This proven metric transformed the business world and now provides the core measurement for customer experience management programs the world round.

The NPS Calculation

Calculate your NPS using the answer to a key question, using a 0-10 scale: How likely is it that you would recommend [brand] to a friend or colleague?
---

Points to consider here.
1.) A freelancer is not a brand.  Upwork is not trying to sell one freelancer to everyone.  
If the question said 'how likely are you to recommend 'upwork' to a friend or colleague based on your experience with the freelancer - then it would make sense.

2.) Upwork doesn't need to grow.  It's fricking enormous - and if it chugged along at its current pace, it would still be enormously successful.

This is a bad metric to score freelancers on a freelancer hiring platform.  It hurts the freelancer.


1. Of course each and every freelancer is a brand, a brand which he has to build, maintain, grow, like any other business. I believe only the people that understand this concept can be successful.

2. They need to grow big time. They need tons of more clients, and it would be really nice for them and everybody else if they made a profit at some point. They might reach a plane of stability and not trying out new and weird ideas every day. 

 

I believe that clients have enough abstract thinking capability so that when answering the question, they don't rack their brains for a friend that it actually applies to, as in: Would my cousin Wayne who lives on a ranch and grows beets recommend this freelancer who just did a fabulous Korean to Japanese translation for me? And then call Wayne to discuss this hypothetical question with him? I don't think so. You should stop worrying about it. 


1. Of course each and every freelancer is a brand, 

 

 

I believe that clients have enough abstract thinking capability so that when answering the question, they don't rack their brains for a friend that it actually applies to, as in: Would my cousin Wayne who lives on a ranch and grows beets recommend this freelancer who just did a fabulous Korean to Japanese translation for me? And then call Wayne to discuss this hypothetical question with him? I don't think so. You should stop worrying about it. 


Yes - marketing oneself is important - but the point i'm trying to make here is
a.) that this rating system is meant for companies with a product, service, etc, and it's meant to measure growth potential for that level of business - dealing with a decent volume of customers.

If I do five large jobs a year, complete and get paid for all of them, and get 10, 10, 10, 7 and 5 - I'm considered an 8, which is not considered a benefit, accoding to this system.  It's not designed to gauge infrequent data points.

b.) And I agree most clients would be able to make the mental leap and rate accordingly.  But why -why- are we asking a question that requires interpretation?  If one of my five clients a year misinterprets, and says 'well this was really a one off - I'm probably not going to have an opportunity to recommend this person' it risks tanking my score - and for no good reason.

 
If you disagree with a.) above - find me some concrete evidence that I'm incorrect, and that this system works well - demonstrably - with fewer data points - and when applied to freelancer/client arrangements.

If you think that in b.) that getting a score below 90 isn't that bad - let me tell you firsthand - the amount of work offers for top rated vs. not top rated - it's substantial, and it can ruin a person who depends on the work. 

A glance at your feedback shows that several of your clients have given you low marks for skills, quality, deadlines, communication etc., so unfortunately, it doesn't look like your low JSS is the result of anyone misinterpreting the recommendation question.


Christine A wrote:
A glance at your feedback shows that several of your clients have given you low marks for skills, quality, deadlines, communication etc., so unfortunately, it doesn't look like your low JSS is the result of anyone misinterpreting the recommendation question.

I'm not at all complaining about my current upwork status here.  I'm asking if the private feedback question has changed - and saying that I think that feedback question - and the metric being used - is a bad system.

Really my marks - good or bad, have nothing to do with the validity of my points - do they?


Tim S wrote:

I'm not at all complaining about my current upwork status here.  I'm asking if the private feedback question has changed - and saying that I think that feedback question - and the metric being used - is a bad system.

Really my marks - good or bad, have nothing to do with the validity of my points - do they?


Let's just say that I've never seen anyone with a 100% JSS score come into the forum and say that the rating system is flawed. 

Christine A wrote:

Let's just say that I've never seen anyone with a 100% JSS score come into the forum and say that the rating system is flawed. 


I've had an 100% score and argued that it was flawed while I had it.

But yeah - I haven't seen that either - and it makes sense.  

I snapped at you and that wasn't fair or right on my end.  It's a pet peeve of mine when people check out my profile instead of addressing the question - but that was a crappy response on my end.


Tim S wrote:

Really my marks - good or bad, have nothing to do with the validity of my points - do they?

They do, really.

The system is the same for everyone, yet many people keep their JSS high to very high year in year out.

Totally happy clients don't leave poor private feedback no matter how the question is framed.

 

Christine A wrote:

Really my marks - good or bad, have nothing to do with the validity of my points - do they?

Let's just say that I've never seen anyone with a 100% JSS score come into the forum and say that the rating system is flawed. 


😄



They do, really.


No petra - they don't.  The worst worker in the world should still get the chance to make a point.  My bringing up a question about jss doesn't automatically involve my profile and work history.   Don't be a **Edited for Community Guidelines**.

"The system is the same for everyone, yet many people keep their JSS high to very high year in year out."
A bad system can still work some of the time for some people.  The fact that many people keep their JSS high isn't an argument for the system working correctly.

"Totally happy clients don't leave poor private feedback no matter how the question is framed."
I'd agree it would not be a common occurrence.  But you don't know for certain that the framing of the question has never had an effect.  Why not take out the possibility of misinterpretation?  

But you're right - the question itself probably doesn't matter so much.  That isn't a hill I'm aiming to die on - I really just wish the system could be a little more nuanced - or just balanced a little less harshly against the freelancer.  


Tim S wrote:


.   Don't be a **Edited for Community Guidelines**..  


And here we go again with the insults.


Q.E.D.


Petra R wrote:

Tim S wrote:


.   Don't be a **Edited for Community Guidelines**..  


And here we go again with the insults.

Just going to reign it back here - I don't know if you get to see what I actually typed there - but the 'edited for community guidelines' makes it seem like I said something really awful. 

It was the mildest possible word - I thought it was acceptable in common speech.


Petra R wrote:

And here we go again with the insults.


Petra, in looking back over the thread - I think I jumped at you too fast.  I've found your responses in the past to be abrasive at times - but I can't overcompensate in a reply because of some half-remembered frustration - that's not fair.  

I'll try to keep that in mind going forward.

Hi all,

 

I would like to remind everyone to be mindful of the Community Guidelines and avoid making personal attacks. 

 

Thank you.

~ Aleksandar
Upwork

I think you're clutching at straws. The context of the question is quite obvious and very few would miss it. It's not Upwork's fault that your clients are not satisfied - there's no point in trying to blame the system. 

Jamie F wrote:

I think you're clutching at straws. The context of the question is quite obvious and very few would miss it. It's not Upwork's fault that your clients are not satisfied - there's no point in trying to blame the system. 


Again - I'm not talking about my current jss.  I'm not talking about my clients.  I really wish this could get talked about objectively - without folks saying 'well look at your history - it must be you that's the problem'. 

And that's my point.  Very few would miss it.  But when your data points are below 10, it only takes one data point to deal a major blow.  I think that makes sense.  Why not just directly ask how satisfied the client is with the work done?

I agree with the OP. The question should reflect the name of the metric that's based on it (or vice versa). If the metric is called a "success" score then the question should be about success. 

 

And incidentally, my understanding (though I don't claim great knowledge of the subject) is that the Net Promoter Score system was not designed to be used in this way. I believe it's designed for assessing support for a company or its products, for the company's own use. It's not intended as a rating system. 


Richard W wrote:

Thanks Richard - that is exactly what I'm trying to say.  You summarized it better than I did.