Hello Icandothis,
Feeling guilty for leaving/not going back is unfortunately really common when we have experienced abuse. Abusers make themselves the centre of our universe, to the point that we often begin to prioritise their needs and wishes over our own. It can be really difficult to break away from this dynamic and not feel guilty for leaving.
FOG (Fear, Obligation and Guilt) is something many users speak about here on the forum and is something that may be relevant here also.
It can be difficult to rebalance that feeling of responsibility – try to remember the reasons why you made the decision to leave, it was because of his behaviour, which he chose to do/continue, so he is the one at fault here, not you.
Take care and keep posting,
Lisa