re: "Client not response after job done."
Clients are not obligated to respond to freelancers.
re: "My client didn't respond after the job is delivered."
That does not matter. Clients are not obligtated to respond to freelancers.
re: "I submit a milestone and he didn't respond to it."
Clients do not need to respond when a freelancer submits a milestone.
re: "What should I do?"
You don't need to do anything.
You already clicked the "Submit Work and Request Payment" button. If the client does nothing, then all money in escrow will be released to you AUTOMATICALLY after 14 days.
re: "Should I end the contract for getting the money we agree with?"
Don't end the contract until the money has been relased into your account.