Hello SilkyHalide, that all does sound hard! Its not ok for him to threaten you, and to scare you, sounds like living on the edge and so bad for you when you deserve better.
Its good to have been reconnecting with others, so much strength and balance can come from having strong connections with supportive people that matter to you. I would be worried about the effect on the children if he is using them to get to you, I wonder if you could talk to SS about the effect he is having on them? Mind you, I then wonder what they would do, possibly tell you you should leave him but give you no tools to do that, but whatever you do, they can’t be continually suffering this. What do your friends say about what he is doing to them?
take care and keep strong xx