Husband or Lover or myself?

So I'm 34 married for 9 years. No kids. Love my husband, but we are no longer sexually intimate or affectionate. I deal with all the housework, bills, appointments etc. He will do the dishes or the bins occasionally. He wants children yet won't have sex with me unless it's near my fertile time otherwise I get told he's too tired or can't be bothered. Been this way for around 4 years. Tried spicing things up with underwear, holidays etc no change when I talk about it he says I'm being silly or goes silent for days. He has suggested IVF which I feel is a cop out. I have never been bothered about kids but wanted to give him one as I love him. We are more like best friends. I feel bad but a year ago I slept with someone who made me feel wanted, and appreciated. He has 2 kids so I left it at a one time thing so things wouldn't get more complicated. I couldn't stop thinking about him but never told anyone let alone him. Then he messaged me around 2 months later saying he couldn't get me out of his head. We have been talking and having amazing sex since. Lover and I are close in everyway. Discussed leaving partners, even what marriage we would like to eachother. For us both, it seemed we were getting from each other what we weren't from our partners. He was worried that she would be awful to him about having access to his kids - which I fully understand so we decided to take our time and not upset our partners without needing to. Except, he has now found out his long term gf and mother to his children has also been having an affair and he says 'its a lot to process, I need some time to get my head together' which I understand. Again, it's his kids and she is awful to him. I'm now feeling a bit lonely from all angles and yes I know it's selfish but I'm constantly thinking I'm going to get dropped by my lover. Either way I know I have to leave my husband, but as we are 4 months into finally owning our home (after me supporting us both financially for 5 years) and I know his mental health could not handle the breakup I feel so guilty and I do genuinely care and love my husband but I feel as though I get nothing back from him and he clams up whenever I approach it. I don't know where to go I'm so confused. Do I stay in this marriage to ensure my husband's bipolar is well managed and he is ok at my own financial, emotional and mental expense? Or do I get out, enure the consequences and pray he is ok without me?

Absolutely do not bring a child into this! Do not get pregnant! Having an affair is not a healthy coping mechanism nor a solution, so put an absolute stop to the affair. Meet with the doctor who manages your husband's mental health issues, and tell the doctor that your husband will need extra care and support by the doctor and additional mental health professionals because you are going to divorce him. Then, divorce him. And don't allow the purchase of the house be a deterrent to your divorcing your husband. I left a horrible marriage 5 months after we purchased our home ... it can be done.

p.s. Another option would be to work with a mental health agency to see if they can help more with your husband to get him well enough so that he leans on you less and becomes more of an equal partner instead of depending on your for everything, and to get you to a to a point where you don't feel so responsible for your husband's well-being and can be less co-dependent. Still, I would not bring a child into this situation!