Side guy problems / am I his side girl?

There is this guy from my high school who I now go to college with. Him and I "hook up" occasionally and it's pretty fun and casual. We talk/snapchat all the time. I'm friends with his suite mates and particularly good friends with his roommate and the roommate's girlfriend. We go home on weekends and hang out with mutual friends on occasion. They know abouy me and him. It's great. But now I think I'm catching feelings for him pretty hard core. But here's the thing, I have a steady boyfriend of three years at home preventing me from exploring a relationship with side guy. And my boyfriend is an okay guy. I wouldn't mind continuing even the casual relationship with side guy at this point but I would have to break up with my boyfriend. I've never broken up with anyone before and he doesn't deserve this, I don't have the courage to go through with it. Also, does side guy have feelings for me? There is this other girl he hangs out with pretty regularly but I can't tell if he has feelings for her or if they're just friends since they both started college last year (I started this year) and are in the same major. Could it be he wants more from me but my boyfriend is in the way? Am I his side chick? Am I just paranoid? Is it worth pursuing? Is it worth ruining my relationship with both him (as friends) and my boyfriend? I've never been in this kind of situation before and I've never had to break up with someone before. But I cannot stop thinking about this other guy. Just looking for some outsider insight to the whole situation.

I feel I should add Things he does that makes me think he's caught feelings: 1) Asks me to stay the night after hooking up. If I say maybe he says "how about yes". Cuddles me all night. 2) The next morning (when they would normally kick you out) even though he has 8am classes he let's me sleep in cause my classes are later. 3) Comes back after 8am class and we nap together till I go to class. 4) We talk all the time despite only occasionally seeing each other. Things he does that makes me think he doesn't have feeings: 1) Literally just anything to do with that other girl. 2) Doesn't hang out with me often. 3) I think there is a correlation between our relationship and a few beers in his system (but that doesn't explain the next morning treatment I get or the talking all the time)