Is my boyfriend secretly gay??

Hi, Im 20 and I've been in a relationship with my boyfriend for a year and half. We are like best friends as well as partners and get on extremely well. At the beguining of our relationship my boyfriend told me something what he's never told anyone else. That he's once been with a man - all the way. At first i thought it was a joke and laughed it off because we knew each other when this first came as we were friends and he strongly denied this. But then i knew, I worry if he is gay and im terrified hes going to leave me for another man. I just dont want our relasionship to be based on a lie, he swears it was a mistake and it was before we got together. He said it was a drunken mistake, and regretted it as soon as he woke up, he also told me he tried to watch gay porn to see if he was gay and it did nothing. Hes a good looking guy and does have self esteem issues, even more so before we got together. He slept with alot of ugly girls, and in his words "build up his confidence again" as an ex girlfriend really hurt him. I keep thinking maybe im over reacting,Whenever i mention it i do get the gut feeling like it was a one time thing and a mistake. its just i often wonder if im a cover up, and i dont know what id do without him now. I've checked his history on his computer a few times through our time together and i dont find nothing. I cant talk to anyone about this as we share friends, and he doesnt want anyone to know. Everyone often tells us how happy we look/cute together and how perfect we are for each other, so am i just being daft? Apart from this, he doesnt show any signs of being gay and our sex life has never suffered. I think im being stupid, but i do catch myself worrying about it...