It sounds like your taking only the clomid now, correct? No more Test E?
Have you had estradiol levels tested, both with & without clomid? Estradiol is one of the most potent negative feedbacks to FSH, which is the gonadatropin that calls for sperm production (granted the clomid is supposed to prevent that negative feedback). It's also very important for libido in the male. Too much or too little can negatively effect libido.
I'm assuming the same doc gave you the clomid as the cialis, which would then bring me to assume you are secondary hypogonadal. If you are self-medicating the clomid, do you know if you may be primary hypo (testicular failure)? If you're primary, the clomid won't help.