I believe I addressed this briefly with my series on Gioia, but I should say more about it. I'm an editor for two journals, and I have done hundreds of peer reviews myself in my career, and of course I've had my own papers peer reviewed. Journal academic editors and peer reviewers generally are not compensated for their time. (The publishers do pay managing editors, but they have no role in deciding what gets accepted.) We do it because we need it on our CVs, basically -- it's an expectation that faculty will do a lot of stuff that they aren't explicitly paid for. Serving on committees for our own institution, academic societies and conferences, writing letters of recommendation, all kinds of unpaid labor.
Anyway, as an editor and peer reviewer I can catch insufficient literature reviews, inadequate explanation of methodology and methodological weaknesses, ethical lapses, errors of logic, statistical mistakes and unsupported inferences -- whatever is discernible in the paper itself. But I can't know for sure that the authors really did what they said they did, how careful they were with their interventions and measurements, or whether they in fact fudged their data (there are a lot of ways of doing this while telling yourself you're being honest), retrofitted their hypotheses, or just plain made shit up. Also, journals differ greatly in their standards for what gets published.
There are in fact so-called predatory journals, that will publish just about anything for the publication fees. That's right, scientific journals don't pay the authors, and sometimes the authors have to pay them. There are good open access journals and bad ones. The good ones have standards and require legitimate peer review, and the benefit is that you don't need to pay hundreds of dollars for a subscription or have privileges at a library that does. Many subscription only journals are owned by capitalists who charge exorbitant prescription rates and have paid advertisements. Open access was a well-intentioned idea and I have links to a couple of open access publishers in my side bar. However, as I say there are some very bad ones as well.
So what does all this mean for the quality of the scientific endeavor and scientific literature? Well, it means we do face some challenges but it's mostly okay. First of all, if a conclusion really is wrong and the experiment really won't deliver the results, others will usually figure that out pretty quickly, if it's about something important that people will notice and want to follow up on. Outright fraud does often get detected Of course we can't know how often it doesn't get detected, but it seems to be committed by serial offenders and once they get nailed people start to look harder at all their output. Here's the Retraction Watch leaderboard so you can see what I mean. Also, people's colleagues and grad students are generally honest and it isn't easy to get something past people in your own lab.
However, as I have suggested above there are a lot of ways people can fool themselves, and there are a lot of weak results that manage to get published. So a competent scholar has to know what journals are reputable, and how to read papers to evaluate the strength and credibility of their results. There's often a lot of spin in the abstract and conclusions as well so you need to be able to see through that.
The bottom line is that science proceeds messily and can get things wrong for a while -- sometimes a long while. It was very hard for physicists to give up on the ether, or geologists to accept the reality of continental drift (now called plate tectonics), and there have been shorter term fads that didn't pan out. But the long-range trajectory of our knowledge and understanding is always upward. We did get rid of the ether and we do know that the continents move. We know that gastric ulcers is actually an infectious disease (not the result of suppressed anger), and that the earth is about 4.5 billion years old and Human Immunodeficiency Virus is the case of AIDS. We know a whole lot, and we know a whole lot of what we don't know, and we know a whole lot of what we have been wrong about, and why we were wrong.
The problem is not the scientific community, or establishment, or institution, or whatever you want to call it. The problem is people who think they're so smart they know more than people who really do have hard-won expertise from years of study and poverty and pain. (Try being a grad student, it's really not much fun.) Also, con artists and liars, who do often believe their own bullshit. And the problem is other con artists and liars who appoint such people to be Secretary of Health and Human Services, and the cowardly tools who vote to confirm them. But the truth will win out in the end. Stalin was the worst enemy of the Soviet people in many ways, but Lysenkoism is pretty well up there, because it created food shortages and famine. But it's gone now.
The same will happen to RFK Jr. Granted, we don't want to have to live through this, and some of us won't. But it will end.
No comments:
Post a Comment