Who calls the tune?

You may have read about the study in PloS Medicine by Lesser and colleagues (including our friend Merrill Goozner at the Center for Science in the Public Interest) who find that studies funded by the beverage industry are more likely to be favorable to the beverages in question than studies with neutral funding.

Some readers will just say, "Well duhhhhh," but others may find this unsettling, or somehow disparaging to the claims of science to unique epistemological authority. Don't scientists claim that properly conducted investigations are merely revealing a reality which is "out there," independent of the ideology or preconceptions of the investigator? How then could the funding source influence scientific conclusions?

Well, scientists do sorta kinda make that claim but it's not that simple. I'm not going to address the question here on a profound level -- I've already taken a flyer at that here and here, among other places. But on a common sense level, there are several reasons why funding sources can affect conclusions, that don't take us down to the question of the fundamental privilege of scientific inquiry.

The most egregious abuses have been limited in recent years by the cleansing action of sunlight and somewhat stricter policies, but in the past it was common for industry funders of research to retain absolute editorial control over the findings. That meant that if the company didn't like the results of a study, it could prevent the investigator from publishing them; or it could edit the report to its liking. Selective release of findings still goes on, but there is movement to require that drug companies make all clinical trials available for public scrutiny, while universities are not allowing faculty to sign contracts which surrender their autonomy. This problem is far from solved, but it has been ameliorated.

Even so, there is a lot of power in simply picking and choosing what questions to ask in the first place. By the time you go into a large scale, expensive study, you generally already have a pretty strong hypothesis. A company will simply choose not to go where it doesn't like the lay of the land. It's also easy in many cases to set up a study to come the desired conclusion. For example, drug companies compare their new drugs to placebo, rather than existing drugs that are already known to be effective. That way, they don't have to show that their products are better than the alternative. They also often test their products against sub-optimal doses of competing products, or otherwise rig the trial.

Then there is the problem of subtle bias. In any complex study, there are countless decision points. Is the data for this case valid, or should we throw it out? What are the cut-off points for classification? What constitutes a clinically significant effect size? How should this questionnaire item be phrased? It's very easy to consistently make these decisions in a way that favors a desired conclusion, without being consciously aware of it. Of course, you could also cheat on purpose. But psychological research has shown that people's decisions can be powerfully influenced by very modest incentives and feelings of obligation to a sponsor, even when they emphatically believe that they are completely objective and fair.

Of equal importance is the way in which an existing body of knowledge -- which is usually beset with contradictory findings or complexities of interpretation -- gets summarized in review articles. Reviews in major journals can define the ways in which issues are conventionally understood, but it turns out that these reviews have often been written by drug company flacks, rather than the prestitious authors whose names are attached, who merely sold their reputations for money and an extra dollop of fame. I think these prostitutes should have had their tenure revoked and been fired, but in fact there have generally been no repercussions and their careers have sailed on unaffected. Medical journals now have policies against ghost-written articles, and they require disclosure of conflicts of interest, but that obviously doesn't stop this scam. There is no way to tell if the putative author really did the work, and just disclosing conflicts doesn't do anything to stop them from having their full effect.

The bottom line? Science is what it claims to be, an attitude toward the world and a tool kit for revealing reality. But, like every human enterprise, it is corruptible, and frequently corrupted. The solutions are structural. Ultimately, it really does matter who pays the piper.