New husband but I hate myself for being depressed

My husband and I are in our sixties, we met on a mature dating site and have been married nearly two years. His first wife cheated on him, they had two sons but divorced after 5 years. He had a happy marriage after that which lasted 12 years, his wife sadly died, that was over twenty years ago. I had a similar life story. He told me he'd had a relationship lasting about 5 years but it had ended 5 years before we met. He is such a lovely, caring man and I couldn't believe my good fortune in meeting him. A couple of months after meeting he sold his house and moved in with me, this is when we realised he had erectile dysfunction, I wasn't worried then and thought it would be just a matter of time. We decided at our age we couldn't waste time so booked our wedding, I honestly thought that things would be better after we married as he was always saying he couldn't understand why someone like me would want a man like him, I adore him. One night I asked him if he had been ok sexually in his last relationship, he said it was, it had ended badly and he had nothing but hate for this woman, he summed her up in one word - rejection. This didn't do much for me! Since before we married he has lost all feeling, the doctor gave him tablets but nothing works. I am trying to come to terms with this but it does hurt and upset me, what hurts me more is the constant thoughts of him with his last partner, the fact that she rejected him but I have never had a chance. My hobby is ancestry, I said I would trace his family tree for him and his sons, on doing this I found out that his last partner was in fact his cousin and also the ex-wife of the man his first wife left him for. This makes me feel quite uneasy, I don't know why, he confirmed it but doesn't mention it. I am so depressed and it's worrying him, I haven't told him why because it seems stupid, the woman died just after we met so it's not like she's a threat. I dislike myself for having these thoughts. Should I get therapy?