Am I dumb to go back to my cheating boyfriend?

This past summer, i dated this guy for about two months. most of our relationship was physical, but it wasn’t a bad thing. i have a problem with commitment so, we were never really “officially boyfriend/girlfriend” and i wanted to make sure the relationship was something i really wanted. anyway, he went on vacation this past summer to australia and slept with three girls. he came back and i heard it through the grapevine what had happened. when i brought it up to him he told me that “he knew it was a big mistake and just wanted it to go away and that he didn’t want to hurt me” but i got scared and i stopped trying to get into a relationship with him. we stopped talking for a while, because i avoided him and i got into a different relationship for three months. but we started talking again, on my call.. because i really missed him a lot. i broke up with my boyfriend because it wasn’t fair to him that i was interested in another guy. and now, my summer guy is somewhat back in my life. we don’t talk as much as we used to, but its my fault. i’m just scared to get close to him then find out that he doesn’t want me anymore and that he just wants me for physical pleasures. he doesn’t really make any attempt to be with me unless he’s horny, but i really do want to be with him. i know its naiive and stupid, but i need advice. what should i do to move on? or should i stay and MAKE things work? afterall, i don’t know what he wants. he makes it seem like he wants to be with me, but he doesn’t want to me shot down. please, i need advice.