Image courtesy of [University of Michigan’s School of Natural Resources and Environment via Flikr]
Health & Science
Trusting Scientific Research: Who Funds Our Opinions?
You just sunk every penny you have into opening a restaurant. After spending years perfecting every detail from menus to music, you forgot one major element: an advertising budget. How will you make sure people experience your culinary genius? Fueled by passion, you do something you know you probably shouldn’t. You write a glowing review of your restaurant on Yelp.
Is the review a lie just because it came from the owner? Not necessarily. Just because a biased party tells you something is good doesn’t necessarily mean it’s not; their recommendation just holds less value than one from a disinterested party.
But, bias fueled by passion and economic interest happens in medical research all the time. Like you and your restaurant review, medical study funders often promote the information best for their cause. They finesse study designs and findings to make results seem more favorable for their product or service. A manufacturer of a new blood pressure drug might play up one specific benefit of their product in a way that makes you believe the drug beats others overall. It’s not fraud or misconduct, it’s just spinning results in a positive way–something any human with a vested interest in an outcome tends to do. Furthermore, data from a study means nothing by itself to most people, so invested parties can play with it as much as they want to tell a good story.
In discussing the interpretation of statistics in Made to Stick, authors Chip and Dan Heath put it this way:
Ethically challenged people with lots of analytical smarts can, with enough contortions, make almost any case from a given set of statistics.
Keep reading for more on funding bias and what you can do about it.
How is scientific research funded?
When you read a juicy new bit of research about something that might kill you or change the world as we know it, the study’s funding probably escapes your interest. But any study you read, whether it’s about drugs, medical treatments, nutrition, or even sleep, costs money. Where does the money come from? It can come from government grants, nonprofits, independent companies, and even you. When you buy dish soap, some of that profit might go to funding a Procter & Gamble study on a new cleaning chemical. Some portion of your taxes might trickle down to a government grant and end up in a laboratory. And of course, if you donate to a charity, that money might fund studies supporting that cause.
Money Talks
Since money fuels science, science itself can be shackled by economic interests, and interests of any kind can lead to bias. Even without fabricating results, funders have tools they can use to sway study results.
In a paper titled “Tobacco industry manipulation of research,” Dr. Lisa A. Bero calls out some reasons why research findings might not be as concrete as they seem:
- Any study has a context that can be skewed by framing the study, defining the problem, and sculpting that language of study questions and results.
- Data doesn’t present itself. It’s up to the funders and researchers to deliver it to the world, and this delivery can be nuanced to serve a given purpose.
That covers underlying reasons why bias happens, but how does it happen? The World Health Organization analyzed thousands of books, articles, and other materials to see how bias can occur in drug studies and promotions. Its paper, “Drug Promotion – What We Know, What We Have Yet to Learn,” outlined several ways researchers can skew results:
- Publishing results in multiple journals and with multiple authors. Different researchers can write papers on the same exact study. The multiplication of evidence revealing the same findings makes the results look more credible and can lead to a general overestimation of the studied drug’s treatment power.
- Leaving out unfavorable conclusions. Industry-funded studies left out negative results more often than their nonprofit counterparts.
- Using retrospective design, which looks backward to prove a determined outcome. With a known outcome, it’s easier to manipulate study designs to show X might cause Y.
- Putting focus on some features and leaving out others. Industry-funded studies tend to focus on acute benefits of drugs and stray away from ranking the drug’s benefits overall.
- Publishing only favorable results. If a study doesn’t achieve the desired outcomes, the industry funder can simply choose not to publish it, like a lie of omission.
Now let’s look at some real-world examples. In practice, a common method of skewing public scientific opinion involves funding counter blows to combat damaging research.
Bias in Practice
So we know bias happens, we know how it happens in theory, and in the real world the skewing attempts get even scrappier. In these examples, the industries used their funding prowess to spin the science of others.
Sugar
This NPR article relays the story of Dr. Christin Kearns, a dentist who was shocked when a handout of government advice about diabetes didn’t mention sugar. Detecting the scent of industry involvement, she began digging for evidence of similar foul play in the dental community.
After months of research and scouring through internal beet and cane sugar documents dating back to the 1950s, Kearns found that the industry does in fact push policy, especially when it concerns potentially damaging research. When the sugar industry caught wind of dental professionals’ intentions to tackle sugar consumption, they launched a counterattack to help people combat tooth decay while eating as much sugar as they wanted. They looked into enzymes that busted up plaque and other ways to fight tooth decay.
Tobacco
No surprise here: the tobacco industry’s efforts to combat damaging research have been fodder for many public health case studies. Tobacco companies specialized in contending with the findings of detrimental studies. Their philosophy read something like this: The longer you argue, the longer it takes to make decisions. And the longer it takes to make decisions, the more time we have to continue business as usual before we’re hampered with new policies. The tobacco industry used this philosophy to fight regulations for decades.
The 1950s and 60s saw tobacco companies fighting claims that smoking was bad for you. After that, they gracefully transitioned to denying the harms of secondhand smoke.
Here’s a commercial for Camel cigarettes showing doctors smoking and enjoying cigarettes and even recommending the Camel brand. Their decision to use a doctor as the main character sends a strong message to the public: If smoking was so bad, would a doctor do it?
In the 1990s, tobacco companies moved to using PR campaigns focusing on “junk science” to criticize reports on the risks of tobacco smoke, even from the government.
In 1998, big tobacco and the United States reached an agreement about how tobacco could be marketed and advertised. This Master Settlement Agreement surfaced documents outlining tobacco’s science-fighting strategy that confirmed what many had already suspected. When it came to steering science dialogue, tobacco’s policy was to:
- Pay for, publish, and promote research supporting their goals; and ,
- Suppress and criticize research going against their goals.
Tobacco’s efforts mark some of the first concerted and funded campaigns against science in history. Their efforts demonstrate the power of manipulating public opinion. Luckily, physician opinion is much harder to shape.
Does funding bias influence practice?
Good news: doctors know how to evaluate medical studies.
In a randomized study of physicians’ interpretations of funding disclosures published in the New England Journal of Medicine, researchers found a majority of physicians were fully capable of evaluating research based on academic rigor and were not fooled by common manipulations. In fact, the knowledge that a study was funded by industry caused their evaluation of the study’s rigor and likelihood of prescribing the studied drug to decrease.
What You Can Do
We can’t ignore oodles of research just because it might be biased. Luckily, there are safeguards in place. Title VIII of the Food and Drug Administration (FDA) Amendments Act of 2007 (FDAAA) made researchers start registering and submitting results to clinicaltrials.gov where you can check them out at any time. Individual journals also have publishing regulations protecting us from misleading science. For example, the New England Journal of Medicine publishes information on funding, protocols, and the funding organization’s involvement in the study with all of their articles.
But if you’re feeling less than trusting, you can develop your own methods of evaluating the research you read. When you see something new, check for other studies on the same subject to see if they coincide and take an extra careful look at the study’s design.
This graphic from Compound Interest ranks study methods on a descending scale. Keep this in mind when you’re evaluating research.
Even if it’s just something you see in a magazine, you can look up the original study to investigate the design for yourself and form your own opinion. We have access to more science than ever before. With that comes great power, but also great responsibility. Science can be biased but it’s still up to you whether or not to buy into the bias.
Resources
Primary
World Health Organization: Drug Promotion: What We Know, What We Have Yet to Learn
Additional
Washington Post: As Drug Industry’s Influence Over Research Grows, So Does the Potential For Bias
NPR: Documents Detail Sugar Industry Efforts to Direct Medical Research
Heath, Chip and Dan: Made to Stick
University of California Museum of Paleontology: Who Pays For Science?
Public Health Chronicles: Tobacco Industry Manipulation of Research
Scientific American: Can the Source of Funding For Medical Research Affect the Results?
Comments