I disagree. There is plenty cause for concern. No, you won't have acute poisoning most likely, but that's far different from elevated levels in the body, which is metabolically very bad for so many things.
The only way to know is a 24 hour urine collection after taking a chelating agent like DMSA. This will give somewhat an idea of the current metals in the body. I've had this test done and I was quite elevated.
With respect to sea food, I know there are guidelines on what to avoid. For example, larger fish like tuna, halibut, shark, swordfish, etc are higher in metals, while smaller fish less so, and things like sockeye salmon being one of the lowest around. Shellfish and other seafood from certain areas can be contaminated with heavy metals and other toxins. Also I forget what type of tuna is lower in mercury than other types, but supposedly there is a difference. Tilapia and trout are also supposed to be rather low in heavy metals.
Also I believe drinking green tea or taking chlorella tabs with the seafood meal will help reduce absorption of heavy metals.
Adequate good mineral intake such as ZMK also helps keep bad metals out of the body.