You should just tell your husband that you don't want to discuss anything personal about ex-boyfriends especially if the previous one had a bigger penis or a better lover.  They don't want to hear that.  They want to hear that your previous lovers all have tiny penises and very lousy in bed that you could not stand them.  Instead of lying to him just say, you prefer not to talk about them.  At the same time, you should not also inquire about his ex-wives or girlfriends.  The past is the past, and should be left at that.