Surprise, surprise, your work colleague looks a lot less like Brad Pitt once he takes his clothes off! If you don’t like what you see, don’t let him know. This is a time to contain your nervous laughter.
Don’t do to others what you wouldn’t like done to you ! Remember that penis size has absolutely no baring on pleasure, it’s an old myth and time it was updated. If he’s not as much of an Adonis under his clothes as you’d hop, give him a chance and remember Charlotte from Sex and the City, who fell madly in love with bald, hairy, tubby Harry, the absolute antithesis of anyone she’d ever gone for in the past. You never know, he might be the one to push your buttons, but if you really can’t get over it, then don’t ever reveal why you decided not to take things any further. The poor man probably already has a complex about it anyway.