I hate myself for this but I can't ignore what my heart and head is telling me

I am 22 and I recently ended a 6 year relationship with my ex husband. Who i was sure I was going to spend the rest of my life with. We had a child together. My son is 3 and a half and I have him full time. His father ended losing himself and turning into a complete drug addict and there was nothing healthy about how our relationship was the last couple years. It took awhile to get over my sons father but Ive accepted it is what it is. My problem now is that I am absolutely in love with my best friend, who is married. Before you judge me please keep in mind Ive been on the other side of this scenario when my exhusband cheated on me the entire 6 years. So I hate myself for even thinking about coming in between a marriage. I started working at my job 6 months ago and this is where I met my best friend, james. His wife works with us as well. We have gotten very close and I work with both of them everyday and now I go to their church and house on sundays with my son. My son loves him. James isnt happy in his marriage at the end of the day and i k ow that doesnt make it ok but me and him get each other better than anyone else. We just clicked right off the bat. He is definitley interested in me and would sleep with me if I would agree to it but I refuse. I hug him all the time and we hold each other while we just talk about anything on our minds. We cant get enough of each other and I can damn near for sure say I am in love with this man. We talk about packing up and moving without notice but he could never leave his daughter and he wouldnt take her away from her mother. Part of me knows both I g is going to come of this but part of me hopes more than anything one day he'll love me enough to pack up and move away with me.. I know theres a chance I'll just end up heart broken and alone at the end of this but what if theres something I can do? Maybe make him fall completely in love with me or something.. What im asking for is advice on what to do? I believe this man might be my soul mate and Ive never met a man that ive adored and respected so much in my life. I cant leave him alone, i love him. But the fact that at the end of the day im alone in my bed is whats hurts. He tells me one day.. But what if one day never comes? What if it hurts after too long of waiting?