Be honest

With him
There's no point in letting him think you're going to leave your partner for him if you know you're not going to. You'll only keep up his hopes and make the end harder to deal with. Don't lead him a dance. 

With yourself
What is it you're looking for?
A stop-gap? Your confidence back? A serious relationship? A kick up the rear for your partner, to make him pay you more attention?

Once you've worked out what you want and why, you can stick by your decision and get what you want out of it. And, of course, you need to be honest about the consequences if you're in a serious relationship: what questions should you be asking yourself and your partner? Should you split up? If you're having an affair, do you really love him?