This issue contains two interesting papers, one on reliability estimation and the other a systematic literature review. A Novel Reliability Analysis Approach for Component-based Software Based on the Complex Network Theory, by Kewen Li, Mingxiao Yu, Lu Liu, Jiannan Zhai, and Wenying Liu, presents a new method that uses network theory to estimate reliability of component-based software. (Recommended by Carol Smidts.) A Systematic Literature Review of How Mutation Testing Supports Quality Assurance Processes, by Qianqian Zhu, Annibale Panichella, and Andy Zaidman, presents a systematic literature review covering 196 papers on how mutation anslysis is used during quality assurance. (Recommended by Benoit Baudry.)
Scientific journals have helped scientists advance research for centuries. The peer-review system [1] lets journals filter for papers that advance our scientific knowledge and also helps authors improve their papers. This system has contributed to unbelievable scientific progress and incredible progress in human civilization.
More recently, an alternative type of journal has started to proliferate [2]. They have publishers, editors-in-chief, reviewing editors, and reviewers. They publish papers that claim scientific advances by authors with “Dr.” in front of their names. The papers have background sections, tables, figures, discussions, and even statistical analyses. But these papers have little or no scientific value!
They look like research. They read like research. But they are not doing what a true scientific paper does: They are not advancing human knowledge.
I call these facade journals. In architecture, a facade is the face of a building, which often uses more expensive and more impressive materials than the rest of the building. More generally, a facade is an outward appearance that disguises the reality of what’s behind the facade. It looks valuable, it acts valuable, but behind the facade is something that has little content and little value.
It is not easy to tell the difference between research journals and facade journals. I will postpone a discussion of distinguishing facade journals from scientific journals until another essay; here, I want to discuss the negative effects of facade journals.
I initially thought they were harmless. The benefits are clear—they give an opportunity for professors with inadequate scientific training to publish. So what’s the harm? Unfortunately, corruption of any sort not only damages the credibility of all journals, but corruption also insidiously creeps into non-corrupt areas. I’ve seen several examples.
I was recently invited to submit papers to a journal that is “international, refereed, peer-reviewed, indexed and print.” The email promised a “positive response” within one week of submission. Out of curiosity, I jumbled together bits of writing I had laying around with no results and no substance, made up a fake name, and submitted. Sure enough, the paper was accepted with minor revisions within a week! The complete text of the reviews were “excellent work,” “interesting paper,” and “accept.” The only revision was to add 8 references—to papers previously published in the same facade journal. I withdrew of course, but it was an eye opening experience.
Last year I received an email offering to pay me $50 USD for every citation I placed into a published paper. They would send a list of references if I agreed, and the money via paypal after my paper was published. The money was also promised to be “tax free.”
Last week a “college teacher in China” offered me $1200 USD per paper if I would promise “quick positive reviews” within a month and if I would “help with revisions.” Yes, a direct and clear bribe to circumvent our peer review process!
Facade journals can also fool young scientists. A few years ago a PhD student strongly disagreed with my assessment of his term paper. I tried to help him understand what was missing, but to no avail. I even shared the paper with a colleague, who liked it less than I did. The student would not relent, so I challenged him. If he could publish the paper in a scientific journal or conference, I would change his grade. Surprisingly, he came back in three months with an acceptance email! From a journal I didn’t recognize. From a journal that turned out to be a facade. I tried to explain this to him, but he couldn’t accept that either, so I finally asked him to show me the reviews. That’s the last I heard from him, but I’m confident I know what the reviews said: “Excellent work.” “Interesting paper.” “Accept.”
True, facade journals give opportunities to professors to pad their publication list, but there is a cost. The peer review process is crucial to effective science. Without it we can’t separate substance from facade. Without it, we wouldn’t know what to read and what to ignore. Without it, scientific progress would slow to a stop.
[1] Jeff Offutt. What is the value of peer review? (Editorial), Wiley’s journal of Software Testing, Verification, and Reliability, 28(5), July 2018.
[2] Publish and don’t be damned: Some science journals that claim to peer review papers do not do so, The Economist, June 2018.
Jeff Offutt
George Mason University
offutt@gmu.edu
19 July 2018