How To Interpret Conflicting Studies
By Glenn D. Braunstein, M.D. HuffPost Healthy Living
The endless trickle of reporting on health-related studies that claim to offer advice about how ordinary folks should live — what we should eat, how we should exercise, what's good for us, and what's bad for us — can become so confusing and downright contradictory that I can hardly blame people for throwing up their hands and swearing at the vagaries of science.
The vocabulary of scientific research loses something in the translation as heard by the general public. Two recent studies illustrate the bewilderment we might feel, as well as the need to understand terms like association, correlation and causation.
In one, scientists leafed through common cookbooks and made a list of 50 often-used ingredients. They then searched the scientific literature on those food ingredients and found that for the vast majority of common foods, an association has been made, either claiming an increased risk of cancer, or a decreased risk for cancer.
In another, much of what we thought we knew about exercise may not be altogether true. The study agreed that moderate exercise is optimal and what we should all strive for but it also suggested a correlation between extreme exercise and bad health outcomes in older athletes who have been at it for years. What's a health-conscious person to believe?
Clues not proof
Most issues involving diet or lifestyle are put to the test in the field known as epidemiology. These studies, sometimes called observational, may involve many thousands of people. Among the most famous and revered are the Nurses' Health Study, the Physicians' Health Study and the Framingham Heart Study. Over many decades, researchers connected with these studies have monitored information on diet, exercise habits, prescription drug use and other factors. They also monitor disease and death rates.
The information results in hypotheses about associations between, say, wine drinking and heart health or obesity and diabetes. Researchers study data and come up with: associations, linking a food to an increased cancer risk; or a correlation, showing that most people who do X end up with Y degree of increased risk for disease. But associations and correlations are not definitive proof of causation. X and Y may be paired most of the time, but we still don't know conclusively that X causes Y. For instance, people who eat a healthy diet often also exercise, don't smoke, and drink alcohol moderately or not at all. And if they are blessed with a good genetic background, they may live close to or more than a century.
An analysis of what such people have in common with each other versus those who die early may show that the centenarians ate a healthy diet. However, we cannot say that the diet was the cause of their longevity. It may be due to a combination of all the factors under their control or their genes or the mental frame of mind that causes them to live what we define as healthy lives. One way epidemiologists try to account for this is through multivariate analysis, in which each associated factor is examined after statistically removing the effect of all of the other associated factors. Still, the results remain only associations that can be graded as strong or weak, but they do not prove a cause-and-effect relationship.
These studies often form the basis for further study in an experimental clinical trial, in which groups of people, as similar as possible in age and other demographics, are put on regimens. Ideally, the trial is a randomized, double-blind, placebo-controlled study. A large group of volunteers is randomly divided into two groups, so that the characteristics, like age, gender and health status, of the people in the two groups are similar. One group receives a treatment, the other group an identically appearing inert placebo. Double-blinding means that neither the subjects nor the investigators know who is getting the active ingredient and who is getting the placebo, so as to not even subtly bias the results. The "blind" is broken after the study is completed and then analyzed. The greater the number of people tested, the more likely the result is firm.
A classic example of an observational study that pointed in one direction and was put to the gold standard test of a clinical trial was the use of beta carotene supplements. Scientific logic and early studies found an association between high beta carotene dietary intake and lower risk of lung cancer. However, in a randomized, double-blind clinical trial involving more than 26,000 male smokers, those who took beta carotene actually had an increase in lung cancer and an 8 percent higher mortality rate than those who did not.
In a recent analysis of studies, the title of an article published in the American Journal of Clinical Nutrition, says it all: "Is everything we eat associated with cancer?" The answer was almost everything: 80 percent of ingredients found in common recipes they studied were associated with cancer either by increasing or decreasing the risk of disease. But the researchers also found that the associations reported in the medical studies they analyzed were weak, almost insignificant. A full three quarters of the studies touting a decreased risk of cancer, and about the same percentage of studies warning of an increased risk, showed very thin statistical relationships — barely significant.
They listed 50 of the most common ingredients found in cookbooks and then searched the scientific literature for information on those items. Forty of the foods they examined showed either a positive or negative association with cancer in several studies. And those foods are on most of our plates regularly: salt, pepper, flour, egg, bread, veal, pork, butter, tomato, lemon, beef, duck, onion, celery, carrot, parsley, mace, sherry, olive, mushroom, tripe, milk, cheese, coffee, bacon, sugar, lobster, potato, lamb, mustard, nuts, wine, peas, corn, cinnamon, cayenne, orange, tea, rum and raisin.
They found 10 foods with no association, good or bad, to cancer: bay leaf, cloves, thyme, vanilla, hickory, molasses, almonds, baking soda, ginger and terrapin.
So what does that mean, exactly? It means that of 50 randomly selected ingredients from common cookbooks, 40 had been studied specifically looking for a link to cancer; 39 percent of the foods were associated with an increased risk of cancer; 33 percent were associated with a decreased risk.
In the imperfect communication that translates scientific findings into public pronouncements, a thin thread of an association found between, say, tomatoes and a reduced risk of cancer, can come off as a headline that touts a sure-fire way to prevent cancer. The associations between veal, pork and beef and an increased risk of cancer have no doubt created a lot of vegetarians.
But cancer can be cruelly arbitrary. Lifelong smokers can elude it, though we know from decades of sound research that smoking causes cancer. And people who eat a vegetarian or vegan diet can get a diagnosis of cancer, though we know there are associations between dark green vegetables, lentils, beans and other healthful foods and a decreased risk of cancer.
An association isn't a guarantee. A correlation, such as noting that many redheads have green eyes, isn't a certainty. And even when a cause of cancer is proven, such as smoking, not everyone who smokes gets cancer.
When a food is associated with an increased or a reduced risk of cancer, it means that researchers have asked hundreds or even thousands of people what they normally eat. They then compare, say, heavy meat eaters with light meat eaters; or they look at vegetarians and those who consume beef. In either case, they may conclude that more meat is associated with greater numbers of cancer cases. But there can be an unlimited number of variables between those two groups. Maybe the heavy meat eaters also were more sedentary than the comparison group. Maybe the occasional meat eaters had less exposure to cancer-causing environmental chemicals.
Nutrition research is hampered by the faulty memories of research subjects enrolled in epidemiologic studies, which ask people to think back and report their own eating habits. And it's nearly impossible for nutrition scientists to engage in long-term, tightly controlled clinical trials. They would have to isolate comparison groups of research subjects, controlling every morsel eaten and ensuring no one snuck in a candy bar or fed their Brussels sprout to the dog.
That doesn't mean that epidemiologic or observational studies lack value. Indeed, after decades of such studies of smoking behavior, the correlation between smoking and bad outcomes like cancer, heart disease, stroke, emphysema and other ills became so apparent as to consider it irrefutable that smoking causes many diseases.
We're not there with nutrition studies, yet. When it comes to diet and cancer, we have some good correlations — that is scientific guesses as to how well certain foods can predict certain health outcomes. We have no guarantees. For example, cruciferous vegetables (broccoli, cabbage), beans, lentils and green, leafy vegetables are associated with lower cancer rates. We're a long way from knowing all the causes of cancer — dietary, environmental, genetic- or how far diet alone can go to help prevent it.
Can we overdose on running?
A recent editorial about extreme running in the British journal Heart is controversial, and still has exercise researchers arguing its merits. It found that as runners age, extreme running — that is, more than an hour each day and faster than 8 mph — might begin to work against good health as we age. The editorial writers conceded that exercise such as running is healthy. They cited a study of more than 400,000 runners who were followed for eight years. That study showed that 40 to 50 minutes of vigorous exercise every day reduced the risk of death by 40 percent. But after about 45 minutes of vigorous exercise, the runners ran up against the law of diminishing returns. The extra minutes of running gave them no added benefit. Researchers argue that running too hard, too fast, for too long can tax the heart.
In one analysis of 26 studies of endurance athletes, researchers found that about half the athletes had increased levels of a marker of cardiac damage called cardiac troponin T. That led to a theory that, since running causes the heart to pump five to seven times as many liters of blood than normal, that constant stress on the heart after many years of extreme running weakens the heart. The scientists who wrote the running editorial found that the consequences of extreme running actually put those runners in the same heart health category as couch potatoes.
How is a satisfied middle-aged runner to interpret a study like that?
Regular, moderate to vigorous exercise is one of the best things you can do for your overall health. Science indicates that there is an association between extreme running — more than 8 mph for more than 20 to 25 miles per week — and diminishing returns in heart health. Maybe you or I or the man or the woman we watch run past our windows each day can escape that association. But we should be aware of the risk.
So go ahead, take your morning jog, knowing that fast, hard and long isn't necessarily better for your health. And load up on fruits, vegetables, whole grains and nuts, knowing that there are no guarantees for a long and healthy life. Understanding that there are no certainties, you might as well associate yourself with foods and lifestyle choices that just might improve your odds.