A problem with science and steps toward a solution.

[KaptKaos]

Likely this will make it's way to PARF, but I figured I'd start here.

Scientific fraud is rife: it's time to stand up for good science | Pete Etchells and Suzi Gage | Science | guardian.co.uk

Scientific fraud is rife: it's time to stand up for good science
The way we fund and publish science encourages fraud. A forum about academic misconduct aims to find practical solutions

Science is broken. Psychology was rocked recently by stories of academics making up data, sometimes overshadowing whole careers. And it isn't the only discipline with problems - the current record for fraudulent papers is held by anaesthesiologist Yo****aka Fujii, with 172 faked articles.

These scandals highlight deeper cultural problems in academia. Pressure to turn out lots of high-quality publications not only promotes extreme behaviours, it normalises the little things, like the selective publication of positive novel findings – which leads to "non-significant" but possibly true findings sitting unpublished on shelves, and a lack of much needed replication studies.

Why does this matter? Science is about furthering our collective knowledge, and it happens in increments. Successive generations of scientists build upon theoretical foundations set by their predecessors. If those foundations are made of sand, though, then time and money will be wasted in the pursuit of ideas that simply aren't right.

A recent paper in the journal Proceedings of the National Academy of Sciences shows that since 1973, nearly a thousand biomedical papers have been retracted because someone cheated the system. That's a massive 67% of all biomedical retractions. And the situation is getting worse - last year, Nature reported that the rise in retraction rates has overtaken the rise in the number of papers being published.

This is happening because the entire way that we go about funding, researching and publishing science is flawed. As Chris Chambers and Petroc Sumner point out, the reasons are numerous and interconnecting:

• Pressure to publish in "high impact" journals, at all research career levels;
• Universities treat successful grant applications as outputs, upon which continued careers depend;
• Statistical analyses are hard, and sometimes researchers get it wrong;
• Journals favour positive results over null findings, even though null findings from a well conducted study are just as informative;
• The way journal articles are assessed is inconsistent and secretive, and allows statistical errors to creep through.

Problems occur at all levels in the system, and we need to stop stubbornly arguing that "it's not that bad" or that talking about it somehow damages science. The damage has already been done – now we need to start fixing it.

Chambers and Sumner argue that replication is critical to keeping science honest, and they are right. Replication is a great way to verify the results of a given study, and its widespread adoption would, in time, act as a deterrent for dodgy practices. The nature of statistics means that sometimes positive findings arise by chance, and if replications aren't published, we can't be sure that a finding wasn't simply a statistical anomaly.

But replication isn't enough: we need to enact practical changes at all levels in the system. The scientific process must be as open to scrutiny as possible – that means enforcing study pre-registration to deter inappropriate post-hoc statistical testing, archiving and sharing data online for others to scrutinise, and incentivising these practices (such as guaranteeing publications, regardless of findings).

The peer-review process needs to be overhauled. Currently, it happens behind closed doors, with anonymous reviews only seen by journal editors and manuscript authors. This means we have no real idea how effective peer review is – though we know it can easily be gamed. Extreme examples of fake reviewers, fake journal articles, and even fake journals have been uncovered.

More often, shoddy science and dodgy statistics are accepted for publication by reviewers with inadequate levels of expertise. Peer review must become more transparent. Journals like Frontiers already use an interactive reviewing format, with reviewers and authors discussing a paper in a real-time, forum-like setting.

A simple next step would be to make this system open and viewable by everyone, while maintaining the anonymity of the reviewers themselves. This would allow young researchers to be critical of a senior academic's paper without fear of career suicide.

On 12 November, we are hosting a session on academic misconduct at SpotOn London, Nature's conference about all things science online.

The aim of the session is to find practical solutions to these problems that science faces. It will involve scientific researchers, journalists and journal editors. We've made some suggestions here, but we want more from you. What would you like to see discussed? Do you have any ideas, opinions or solutions?

We'll take the best points and air them at the session, so speak up now! Let's stop burying our heads in the sand and stand up for good science.

Pete Etchells is a biological psychologist and Suzi Gage is a translational epidemiology PhD student. Both are at the University of Bristol

[Jim Richards]

OK, fraud in psychology and biomedical fields. What about other fields? Is this article exaggerating the problem? My initial impression is that it is. What about other fields? I think peer review is generally working, but there will always be cases where it fails. Making the process more open is a good first step.

[IROC]

Originally Posted by Jim Richards

OK, fraud in psychology and biomedical fields. What about other fields? Is this article exaggerating the problem? My initial impression is that it is. What about other fields? I think peer review is generally working, but there will always be cases where it fails. Making the process more open is a good first step.

Although I don't have any data to back it up, I don't think other fields of science are seeing high levels of fraud. I will admit there is a lot of pressure to publish and fighting for funding can push people to do things that they shouldn't do, but I would hesitate to paint all of "science" with a broad brush.

I have both authored papers and also performed peer reviews so I am familiar with this process (and the folks I work with do this a lot) and at least at ORNL, there is a very strick code of conduct and fraud of any sort is simply not tolerated. There was a physicist here in my building fired last year for fraud. Zero tolerance.

[nostatic]

Like any field, science is imperfect as it is practiced by humans. Scientists, broadly construed, are far less tolerant of fraud than other areas (*cough* sales *cough*).

[RWebb]

I agree that fraud is low in the sciences. Pressure to publish and get grants (to re-panel the Dean's office in jojoba wood) usually produces a spate of "small" journals papers based on the least publishable unit, rather than fraud.

I suspect problems are greatest in areas funded by large drug co.s, tho f-bt may disagree.

[M.D. Holloway]

I can see where faking data is a big problem, easy to do in pure research but much tougher in engineering where the work leads to actual applications and then wide spread duplication.

I guess the one area that would be tough to fake is math. Experimental data is easy to fluff. You can actually use math to do it but fluffing a paper on a math problem is a whole other thing. I can't see it being done...or many folks actually understanding the content.

[dennis in se pa]

Fraud in the academic world? Say it isn't so.

You can't do that in computer programming- if the code isn't right it won't work.

Nor the networking field - if the system is not setup correctly there will be no communication.

Research - now there you go. So much money spent with no expectation of an actual truth. But the guy who put up the money gets his name listed somewhere for all to appreciate.

[RWebb]

backdoors, and harmful executes can easily be hidden in code - maybe you don't consider that "fraud"??

[RWebb]

Originally Posted by LubeMaster77

I can see where faking data is a big problem, easy to do in pure research but much tougher in engineering where the work leads to actual applications and then wide spread duplication.

I guess the one area that would be tough to fake is math. Experimental data is easy to fluff. You can actually use math to do it but fluffing a paper on a math problem is a whole other thing. I can't see it being done...or many folks actually understanding the content.

you don't seem to realize that any important result will immediately attract others who will duplicate the experiment - if the results do not match... !!

I suspect a more serious problem than fraud is that certain areas are simply not investigated at all, but should be.

[KaptKaos]

I'd say that fraud in science falls within the same general distribution as fraud in any other profession. There is no reason to assume that scientists are more honest than bankers, brick layers or bean-counters.

[RWebb]

actually, there is

but science is a self-correcting enterprise anyway

[techweenie]

Media attention and participation in rush-to-publish articles (not vetted) lead to a decline in respect for science (anyone remember room-temperature fusion Cold fusion - Wikipedia, the free encyclopedia ).

Sensationalizing preliminary findings is a staple of supermarket tabloids ("Experts say..."). Sadly, actual journalism costs money, takes time and is not a profitable enterprise.

[DanielDudley]

Fraud is much more likely to be connected to money than science per se.

[KaptKaos]

Originally Posted by RWebb

actually, there is

but science is a self-correcting enterprise anyway

Leave peer review out for a moment.

Are you actually saying that research scientists are more moral than the rest of society? If so, what do you base that on?

[DanielDudley]

Originally Posted by KaptKaos

Leave peer review out for a moment.

Are you actually saying that research scientists are more moral than the rest of society? If so, what do you base that on?

Science is the same as it ever was. It tends to be self correcting for the same reason we now think that the world is round. (it is) It sometimes is slow to correct for the same reasons that Galileo was persecuted for saying so, only now there are many more factions of vested interest.

And you are just noticing this now for what reason ? I'm guessing because someone wants you to.

[Jim Richards]

Originally Posted by KaptKaos

Leave peer review out for a moment.

Are you actually saying that research scientists are more moral than the rest of society? If so, what do you base that on?

To what end? To satisfy a feeling you have in your gut? The process isn't perfect, just as people aren't perfect. The peer review and research conducted by those that don't by into the research results tend to cancel out the incorrect or fraudulent findings. Sometimes it takes time.

Originally Posted by KaptKaos

...bankers, brick layers or bean-counters

Bankers and bean-counters have accountability. Brick layers have their work inspected. The only {arguably} compelling reason for fraudulent reviewing of anyone's work is money. And we have seen this from time to time. In biomedical research, this sometimes factors in. It's a big money industry. But trying to generally apply your gut feeling to all of science is simply wrong.

[Tervuren]

Originally Posted by DanielDudley

Science is the same as it ever was. It tends to be self correcting for the same reason we now think that the world is round. (it is) It sometimes is slow to correct for the same reasons that Galileo was persecuted for saying so, only now there are many more factions of vested interest.

And you are just noticing this now for what reason ? I'm guessing because someone wants you to.

I think the world being round is a bad analogy. The % of people who thought it was flat is fairly small(people still think it today btw).

Galileo is a much better example, he came out with findings that went against the political structure of the day, and got sandblasted for it. I think this still happens today, and funding finds its way into the coffers of those who agree with the political machine.

[sammyg2]

I have a serious objection to people who haven't BTDT, and who's primary source if info is others who haven't BTDT, telling the rest of us who HAVE BTDT that they know more about it than we do.
I have an even bigger objection to them getting paid with my tax money to pretend to know what they don't.

[RDM911]

People believe that if you start a sentence with "Studies show...." that it must be fact. I dismiss these studies as opinion. How many of these studies have undergone the rigors of the scientific process and been proven to be fact?

[vash]

i dont think we use the word "rife" enough. thanks for the reminder..