Unrequited love

My boyfriend of many off and on years had an affair with a younger,petite woman who had a common law husband.They were neighbors.He chose her company over mine,when I found out.Several hours later,he called me back,pretending that nothing had happened!He moved back to my home within the next three days and in three weeks,she was evicted from the apartment.My boyfriend refused to tell me anything.It has been almost two years.I think he still misses her,and was calling her until last year.He treated me very badly,for the first year.When I would tell him to go back to the building,he says I am picking a fight.I am not.I am just very tired of feeling second.Our sex life sucks.Once,He was at the foot of my bed,with an erection that he tried to hide from me.I feel more and more distant from him.I doubt seriously that he loves me and am trying to prepare myself for the end of our relationship.He always wanted to remarry me,but not since his affair with this woman.Help!!!!!!