A client who has paid you is a good thing.
If you close the contract, the client may or may not leave feedback--they don't have to.
If the client closes the contract, they must leave feedback to complete the process.
If you contact the client, avoid asking them to close the contract so that you can get feedback--IMO, that is basically telling the client that you want them to do some work for you.
Have you noticed whether the other freelancers with work in progress for this client appear to have earned more than one milestone's worth of work from the client?
Ongoing projects are not a bad thing.