This is a touchy situation!
I think that since he responded to your "love you" with a sincere expression of his feelings, it's going to be hard to "take back" what you said without him feeling somewhat foolish.
If this is a man you do care about, but just aren't ready for the L word with....
Would it be possible to say something like "I spoke somewhat impulsively the other day. We're still early in the relationship, and it's important to me that we take things slowly. I care about you, and my feelings for you grow each day. But I don't want to put a lot of pressure on the relationship at this point. Let's wait a bit longer before we take the step of talking about or defining our feelings, okay?"
Hope that gives you some ideas of how to smooth out the awkwardness --I think it communicates in a way that won't make him feel foolish for having responded.
I wish you the best!