Was i right in this situation? I'm starting to regret it?

You should do what you think is right for you and your baby but don't act out of emotion. You will regret it later. If he keeps on messing things up then you should let him go for good and stop all the drama. If he starts to show improvement then stick with him and try to work it out.