Ex won't accept break up

So I've been with this guy for about 7 months, throughout that time we've been on and off, not speaking for about a week due to petty arguments which was resolved afterwards. But during those break ups which were initiated by me, I made it clear I didn't want to be with him, specifically telling him I was too busy and had more important things to focus on. He even said he was too busy as well, but them got rude and angry He just kept saying to forget about it and let's start over. The last time we broke up he just said " lol ok " and that was the last conversation until now, I always feel bad when he contacts me so I told him I'd think about it but he always tells me not to think about. t I just give up and say ok because I feel like he's hurting. He then got a bit upset when I didn't say yes right away. I missed him a lot during the breakup (about a month) but I DID NOT make contact with him. I Now he's waiting on me to answer yes or no, I feel like there's no point in saying no because he will just text again in a week or two saying forget about last time. Btw I have told him that I don't want to be with him in person but he kept trying to say don't think so far ahead.