They want the convenience of a family who are always there, serving their needs and making them look good.
Well, you and your kids are real people. You aren’t there as his puppets. You only want to spend time with someone who cares for the real you and takes notice of your needs and those of your children.
Leave him to his selfishness. Better things are on the horizon for you. Not everyone is like your abuser. There are better men out there.