GUYS AND GIRLS: kissing guyfriend (considered boyfriend now) but not comfortable to stay together?

okk, heres what happened. my guyfriend, or guy i had a crush on for 2 yrs now, told me he has had a big crush on me too ever since we met. he kissed me, and i kissed him back. it felt so good, but for some reason, i didnt really feel comfortable being with him, as in, going out with him. i really like him, but maybe it has something to do with reputations in school? what will others think? thats what im afraid of. would you care what others think, or just care about how you feel about being with him? i dont know what to do. should i tell him that it was too soon, and i wasnt ready???? hes really sweet and kind to me, but i dont know if it was too soon. plz helpppp! need advice kthnxxx :)