This is clearly a case of great minds thinking alike. I started writing this post earlier in the week, and just saw that Jack Norris posted an article on the same topic yesterday. If you're at all interested in this topic, I hope you’ll read both; his is a more in depth discussion and this one is a faster overview.
It’s an issue that comes up frequently in the comments section of this blog—how we analyze and use nutrition research. In order to draw conclusions about diet and health, we need to look at all of the research on a certain topic, evaluate the strengths and weaknesses of the studies and then see what most of the research is saying. And even then, conclusions are sometimes not much more than an educated guess.
Two important considerations in all of this are type of study and the issue of bias.
Different types of studies carry different weight
Aside from the ethical considerations, findings from animal research often don’t translate very well to what happens in humans and should always be considered weak evidence. But even among human studies, some types are better than others.
Ecological studies, which compare disease rates among populations (rather than among individuals) are generally used only to generate hypotheses about diet and health, not to make conclusions. This type of research doesn’t control for the many factors that might affect outcome, and makes assumptions about how pooled data from a population translates to effects in individuals (a weakness known as the “ecological fallacy.”) The study that compared rates of hip fractures to protein intake among different populations is one example of ecological research. While the results suggest that high protein intakes are linked to risk for bone fractures, that’s only one of many possible explanations. In fact, in the case of this study, the findings can be explained away by genetics, geography, lifestyle, and other nutrition factors.
 
It doesn’t mean that ecological studies have no value. But their value lies in raising questions that should be explored by other (better) research, not in proving relationships. Nutritionists never (or at least they shouldn’t ever) make dietary recommendations based on ecological studies.
 
More reliable information comes from epidemiologic research that compares findings from individuals (as opposed to groups of people) and includes control groups. Two examples are the Adventist Health Study and the EPIC-Oxford Study which have both produced a great deal of information about vegetarian—and sometimes, vegan—diets and health. These epidemiological studies show only associations, though, not cause and effect.
The best types of studies are randomized controlled trials (RCTs) in which researchers feed a particular diet or food or food component to people and compare the effects to a control group. If they are well-designed, these studies do show cause and effect. Ideally, the study is double-blinded which means that the subjects don’t know if they are receiving the “treatment” or the placebo (this isn’t always possible, of course) and the analysis of the data is blinded as well. While RCTs are the gold standard in nutrition research, they’re expensive so they tend to be shorter and smaller than ideal.  An example of an RCT that is familiar to most vegans is the research by Dean Ornish in the early 1990s.
Findings need to be published in peer reviewed journals to be meaningful. This means that other professionals scrutinize the research design and analysis. It’s not a perfect system, but it adds an element of oversight that increases the chances that what gets published is meaningful.  It’s interesting and thought-provoking when physicians and other health professionals write books or internet articles about their own experiences with their patients, but it’s not research. Without an actual research design, a control group, statistical analysis and peer review, the findings have limited value.
Bias is hard to avoid
Academics scramble for funding wherever they can find it and the food industry has lots of money. This doesn’t affect the data, though; researchers can’t change their findings to accommodate the industry that supported their research. It might affect what the public hears about different topics, though, since the funding industry can press release the findings—or not, if they don’t like them. And even the way researchers talk about their study could reflect bias.
Those of us who interpret research findings for non-nutritionists have to work hard to avoid bias as well—because we all bring some to our work. And, because there is so much research, it’s pretty easy to pick and choose the studies that support one’s particular bias and viewpoint. That’s why a list of citations at the end of an article is not proof that the content is reliable and balanced. Professionals who have built a career on a particular type of diet may gravitate toward research that supports that diet and discount the studies that don’t. Those of us who promote a vegan diet for ethical reasons obviously need to be extremely careful about how that bias affects interpretation of studies.  
Evaluating the source of nutrition information
So how do you know whether someone is truly knowledgeable about a nutrition topic or if they are biased in their interpretation of the studies? It’s not all that easy to tell but a few guidelines can help. Generally, those who share nutrition information should be willing to express a little bit of uncertainty regarding recommendations. If somebody tells you that they know the one and only healthful way to eat, that’s something that should raise a red flag or two.
Nutrition experts also have to be willing to go with the flow a little bit and change their perspective and admit to being wrong or to not knowing an answer. My take on vegan nutrition is very different today than it was 20 years ago because new research has made me discard some beliefs. In this field, if you never admit to being wrong, you’re clearly not keeping up with the research!
With vegan nutrition in particular, experts shouldn’t minimize potential concerns. There are some challenges in vegan nutrition and, rather than ignoring them, we have to admit that they exist and find the best ways to deal with them. It may not always present veganism in the best light, but staying true to the science definitely provides the best information to help vegans stay healthy.