Certainly it is a red flag if I see that a client has a history of consistently giving low ratings to freelancers. This could be a very difficult client to please.
But if a client has a mixed history... Maybe it just mean he takes the ratings very seriously and when he has encountered poor performance or unprofessional behavior among freelancers, he has provided feedback accordingly.
If there was a client I was interested in working for who had such a track record, I WOULD consider working for the client, but I would do so on a limited basis to start out with. I would tell the client, yes, I can do this first part of the project as a contract, and we'll see how things go from there.
For example, a fixed-price contract or hourly contract that involves no more than two or three hours of work. Then the contract is closed. If the client seems to be treating me fairly during that contract, and leaves feedback that I feel is fair, accurate, beneficial... Then I could agree to additional contracts that cover more work.
The advantage of a limited contract is that it is more feasible and less painful to refund it entirely and thus eliminate public feedback if the feedback is horrible. That won't eliminate the impact of private feedback. But it is something.
