Should I let go?

4 years ago I became pregnant with my daughter, and life looked great. My bf had a good welding job, and just bought our first house. Then, right before I had her my bf lost his job, 7 months later, still no job and I'm pregnant again. we are surviving with my maternity leave and money he had saved. I decided to go bacek to cut my maternity short and go back to work. but only 2 months after I have my son early. 3 months after having him and I go back to work again, while my bf has a few odd jobs here n there adding up to maybe a few months of work. Eventual we cannot keep struggling to make ends meet n decide to move to the province he is from to start over, so we pack up our 1 1/2 yr old and our 7 month old and move. but a year goes by there and he works for only maybe 4 months of that year. I left him for 6 months and moved back home with the kids and did pretty well for us, but missed him and missed being a family so I came back. now its been another 6 months, I have a 4 yr old and a 3 yr old, a husband that still isn't working and I'm having a hard time letting go. he loves our kids and is a good father, our relationship is rocky, but I love him so much. But when is enough enough?. He is not willing to try as hard as he can to help me give our family a good home. I feel like the answer is to leave once and for all and give my kids a good home, but I am struggling with saying goodbye for good

Then don't say good bye. You can keep I'm touch and agree to him having the kids on the weekends