Clomid can cause vision problems. I've read that long-term use is not good. If he's worried about your fertility, ask about hcg monotherapy (if you're secondary), or even testosterone+hcg.
However, before jumping feet first into anything, you need proper testing done. Point out that it would be helpful to know if your hypogonadism is primary or secondary. LH and FSH results should help ferret this out.