Former student / lecturer “relationship” issue

So I’m looking for some impartial advice about a sticky situation I find myself in. I will keep it as short as I possibly can...! 3 years ago while I was at University I became involved with one of my lecturers. I liked him, he liked me, we were in our 30s/40s so nothing creepy. We met up a few times and had to keep it under cover but I did actually enjoy his company and the few times we met up we did sleep together. After a month or so he ended things because rumours started to circulate about him seeing students and he said he had to stop seeing me. I was gutted to be honest but couldn’t do much. He told me they were lies/rumours but I still don’t know if there’s any truth to it. My gut tells me there probably is truth to it because he got involved with me so could just as easily been with others but his position is that it was all just rumours. So he didn’t speak to me until 6 months later when the term ended at summer. He told me he was leaving the university and the back story, then tried to sleep with me again but I declined and he moved away. I didn’t hear from him again until the following summer. During that year I heard a lot of stories from different people at university that he had been involved with various students, some of which apparently happened after he told me he had to end things with me because of the rumours. Again, I don’t know how much of it was true or just rumours. So when he got back in touch I told him what I heard and he denied it saying it was rumours and people making stuff up. If I’m being honest, I believe it did happen and I think he lied to me which makes the whole thing sleazy because I thought he was genuine. But that was 3 years ago. So when he got back in touch last summer and we cleared the air about the rumours etc at uni he told me he had moved to another country for work. He invited me to visit him saying he liked spending time with me etc but made it seem like it would be very casual because of the distance. I decided I wasn’t going to be a long distance fuck buddy, and he didn’t think anything more would work out long distance, so we fell out of touch. Last Christmas he got back in touch to ask me to go over again, but I said no for the same reasons - he was also awful at texting/phoning (like telling me he was busy and would be in touch then going 2/3 weeks before speaking to me). So I said no, again, because why would I make that effort for someone who can’t do the bare minimum and text/phone regularly? Besides, the casual thing wasn’t for me. Ultimately I was looking to start properly dating someone. So we didn’t speak for like 8 months. In August, he got back in touch. This time he is saying it’s not a fling or casual, he wants me to go visit and spend time with him etc. I said I would think about it, but I don’t know what to do. All I can think is, was he involved with other students? Has he invited them to stay with him? Will he invite others in future? I only got involved with him because I genuinely liked him (I’m quite a shy/reserved person looking to get to genuinely know someone) so the thought of him doing that makes me feel sick, but I’ll never actually know. I also met someone recently who was one of his students, she told me that at one point during uni he invited her to stay at a hotel with him. Again, he would say it’s a rumour, but I think why would she make it up after all this time? That’s all in the past but I can’t get past it because if it’s true it tells me he’s dishonest and not genuine. If I had concrete proof he did it, beyond hearing rumours, I’d walk away 100%. But I don’t. So I don’t know what to do. My instinct tells me to run a million miles, but there’s another part of me that says just take the risk, he only lives an hour or so away, have some fun while you’re single and just don’t expect anything serious from him. I’ve been single a loooong time, and I could do with having a bit of fun, but our past situations have just been such a mess that I wonder if I should just walk away. But I also could use a break away to have some fun. I don’t know if I’m just crazy and trying to convince myself it’ll be ok for the sake of not being so lonely- what would you guys suggest?

He can't be labeled as anything with no proof. He did the right thing breaking it off when the rumors began circulating. He saved his career and he saved you from getting grief for it if the uni found out it was you. Yes the rumors were true because he was seeing you duh. Who know's if what your friend said was true, does it even matter? You're not with him anymore. She may have had a crush on him herself but got rejected. She may have been the one who started the rumor out of vengeance. Don't villianise him without any solid proof. If you can't get any solid proof then maybe there is nothing to prove like he said. He seems to genuinely like you because he always contacts you. You must have meant something to him in that respect.