Time to give up?

I am a single mother of 5 and 3 year old boys. Have had a boyfriend for about a year and a half now. me and my boyfriend got along great to start with discussed everything, he was really understanding with things like not being too involved with the kids and that they will always come first etc. we were living about 2 1/2 hours from each other when we met and he had some trust issues which i thought were just related to his ex cheating on him (understandable). i have now moved myself and my children closer as it worked out to be a better lifestyle over all for us and a plus that i would be closer to him. however it now seems like everything is a problem. we dont live together. he has a problem with how many people ive been with before him, what im doing, who im talking to and what i tell them, that i have responsibilities, you name it it seems like hes insecure/has a problem about it. he can get quite angry and verbally abusive and he now wont have anything to do with my kids (whereas before he might see them in passing or small amounts of time. they know who he is basically) i dont think this affects them tho they dont ask after him or anything. i just feel like i know the person he can be (or was pretending to be) so i guess i keep holding on to hope he will be that person again. ive tried talking to him but nothing i say is good enough or means anything almost like i shouldnt have an opinion or feelings on anything because im the one with baggage that he has to accept. i dont feel like im ready to give up on him but its starting to really affect me emotionally. i feel like im never going to be good enough. any ideas on ways to move forward or is it time to give up?