What to do about me and my best guy friend ?

Me and this guy have been friends for about 2 years and a bit and I was dating his best friend for 2 years of it, since me and my ex have been over me and my best friend have gotten closer, I see him more and talk to him more. Then the other day I spent a whole night with him, we were hanging out till about 3 and it felt so different compared to other times, then we left all our friends and on the way home when we were near my house he stopped the car and asked if I wanted to just hang out for a bit and I said yea nws. The whole night I was getting vibes from him, it really felt like he wanted to kiss me and I had a big urge to kiss him then while we were in the car we started talking and we both feel the attraction and then I brought up that I had an urge to kiss him and we stared at each other for a while then I lent over and kissed him and we got really into it but would occasionally stop and say no this is wrong but it continued for about 2 hours. We hung out the next night too, it wasnt as bad as the night before because we were at the movies so we would only kiss sometimes but he kept trying things. He just got out of a relationship and so did I. It felt like he wasnt doing it because he likes me but just because he wasnt thinking. I dont know what to do now because I dont wanna lose him as a friend or anything and we cant be in any sort of a relationship because all of our friends would turn against us and its not normal for us. I dont know if this is normal, does this usually happen ? how wrong is it ? and how bad is it that I dont wanna stop, but we’ve both said we have to.