Should I settle with or finish everything that's left between us

i dated a guy 1 amd half year ago. We dated for 8 months and had broke up twice. Initially both of us very deeply and very passionately in love but later ever since i learned we couldn't get married and told him about my ex-boyfriend things he started having distance and eventually we broke for real. He is dating someone else now. I was hurt so for 4 months i didnt speak to him. But since he worked hard to be my friend i became his friend. We got emotinally attached again and physical happened. And i had no romanric at all for him any more. 1 day i was at place and i thi k it was wrong because his sister was there too. But after an hour she left. That opportunity made him loose complete control of himself and forcefully had sex with me or made love to me. I am not sure what was that...i really didnt want to do it....in that moment he told lots of crazy stuffs like how could have crush on some other guy when he still loves me?? Ever since that day im restless... I have no peace. After that day he applogised for what happend n i forgived him and met casually to just exchange movies twice and since that till now we never spoke. Seems like to me all he wanted was to take his feelings out and finish it off. During those he calld me many times and met me many times but he doesnt do all that any more. I'm thinking of never ever talking to him any more or should i be casual and maintain a "hello hi friendship ????"