The estrogenic effects of soy protein are exaggerated. I wouldn't rely on it for the majority of my protein intake, but do I get 20-30g of it a day. It's good to have as broad a spectrum of protein as possible. Egg is your best bet if you can't take any milk protein, along with chicken and lean beef (like 95% lean hamburger cooked on the Forman grill). Fish is really good and really low in fat, but it can get expensive. I hate canned tuna but a lot of guys swear by it (I do love tuna steaks, cooked rare, but that's too pricey to eat all the time).
Casein is a milk protein, but some people are only allergic to whey and not casein (and some people are only allergic to casein and not whey). It would be worth your while to see if you can tolerate casein even if you are allergic to whey.
/karp