Hi Wateringcan,
It’s always so hard to know what to do for the best for our children with regards to how much time and contact they spend with their dad once we have left. Unfortunately, unless the abuse was aimed directly at the child, the courts seem to think that an abuser should still be entitled to spend time with them, and often, they will continue to abuse them in some way, the most common way by putting their mother down, preventing them from having phone or video contact, or controlling how long they are allowed on the phone for. When my son went to my ex he wasn’t allowed to ring me at all, even if he asked to. Yet when my son was with me, if he wanted to talk to his dad on the phone I would let him ring him.
Only you know what he is capable of and what he is likely to do. If you really don’t think your child would be safe with him alone and unmonitored then speak to a solicitor about supervised contact. It is very rare cases that get no contact at all.
Does his temper and anger get taken out on your child? Does he smack them when he is angry, or if they are annoying him? If he did this in your presence then he will continue to do it when he is with them alone and no one will be around to protect your child or know what has happened, so it really depends on what he is like to your knowledge and how much of a risk he poses to them when you are not present.