There are several solutions. What the best one is will depend on what the issue is; you've only reported symptoms, not underlying condition.
One is to block out a time when you are "at work." That means your offspring, your sisters and your cousins whom you reckon up by dozens, and your aunts don't bother you from ten a.m. to two p.m. Be firm about it.
Another is to offer to find qualified people to do this other work. That takes time, also, for which you should be paid. My former partner made a living doing that on Upwork (eLance at the time). She would find videographers and Martian-Lemming translators and seven-legged dancers and whatever else was needed. If the client needed her to do it, she would manage their work. I used to take on complex projects requiring subject matter experts in multiple domains, finding people and managing the project in addition to doing whatever my own slice was. This isn't easy. It's much more difficult than just freelancing in a single domain.
The best might be to find people who do project management and partner with them. You've got the client and oneslice of his pie; the PM takes responsibility for the rest. Consider posting a job as a client asking for Project Managers to spend half an hour communicating with you about how to do this and what a partnership would look like. Hire the most likely four or five for half an hour each. This isn't free, of course, but Upwork's paranoia about freelancers talking to anyone, ever, outside of Upwork's micromanagement, means we need to invent workarounds.
Hi Maureen. I can sympathize, as I have some of the same tendencies myself, though probably not to the same degree. Over time I've got a bit better at saying "no", but I still find it difficult, and feel bad afterwards. I tend to think, "surely I could have fit that in; it's only a few hours work". But everything always takes longer than I think it's going to. I'm currently doing a job which has been quite good financially, even though I haven't charged for all my hours because I think I've been slow. But it's really not work I like to be doing, and I find it quite stressful. I guess if I was more of a "chad", I would charge for all the time or end the job. But I don't want to let the client down, as the work is urgent and it would take time to get another person up to speed.
I find it helps to be honest about your limitations. If you tell your client that they're giving you too much work, or work you're not suited to, then, if they still insist, you will probably find it easier to accept that it's their responsibility, not yours. And that should make it easier to say "no".
As for being quicker, I don't really have any answers. But I find that stress and rushing mean that I make more mistakes and end up taking longer. So taking on less work-- and being less stressed--may help.