Editorial:
How to Get Your Paper Rejected from STVR

This issue contains three deep and compelling papers on modeling software and generating tests. The first, An improved Pareto distribution for modelling the fault data of open source software, by Luan and Huang, presents a new model, based on the traditional Pareto distribution, that accurately describes the distribution of faults in open-source software. (Recommended by Min Xie.) The second, Extending model checkers for hybrid system verification: The case study of SPIN, by Gallardo and Panizo, studies an unusual type of system, hybrid systems. The authors have developed an extension to model checking that allows engineers to accurately model the behavior of this complex type of software. (Recommended by Paul Ammann.) The third, Search-based testing using constraint-based mutation, by Malburg and Fraser, addresses the key problem of test value generation. They propose and evaluate a hybrid form of test value generation that combines search-based techniques with constraint-based techniques. (Recommended by Mark Harman.)

This editorial is based on a talk I gave at the ICST PhD symposium in April 2014. I had fun giving the talk and hope you enjoy reading this summary.

First, I want to make it clear that I am highly qualified to give advice on getting papers rejected. I have had well over 100 papers rejected and may well be the most rejected software testing researcher of all time.

As examples, let me share some quotes from reviewers:

• “As usual, Offutt got it wrong.” —TSE 1993. (The paper was accepted on the second revision with two accepts and one reject vote, and currently has over 150 citations.)
• “A study like this should have been published in about 1980.” —TAV 1989. (Unfortunately, no such study was published in 1980.)
• “The presentation needs considerable improvement.” —TAV 1989. (This was the complete review; no details were given.)
• “We are sorry to say your paper has been REJECTED.” —Letter from editor, and yes, that word was capitalized and in bold face.
• “Better than average American academic paper, below the standard of papers written by European (non-English) academics.” —FTCS 1990. (This comment, by the way, was about the writing as opposed to the research results.)

In this editorial, I assume that your goal is to get your paper rejected. My first concrete piece of advice is to be courteous to the reviewers. Reviewing is hard work so you should try to make it easy for the reviewers to reject your paper. Here's how.

The most effective strategy is the only one on this list I have not used—plagiarize! This not only gets the current paper rejected, but future papers. Merriam-Webster defines plagiarism as [1]: “To use the words or ideas of another person as if they were your own words or ideas.” To make this easier, I have collected a few specific types of plagiarism:

• Complete copying of an entire paper
• Copying key results
• Copying unpublished work
• Copying auxiliary text such as related work or background
• Copying figures
• Improper quoting

It is important to note that copying from your own paper is not plagiarism, so do not bother. This may, however, be a copyright violation. (Although I was once accused of plagiarizing from a paper that plagiarized one of my previous papers ...)

Some of my favorite strategies for getting papers rejected in list form:

• Choose problems other scientists have already solved. This works better if you either (a) do not cite the other paper, or (b) cite it but misrepresent it in the related work section. Strategy (b) has the advantage of making it more likely the scientist who wrote the previous paper will review your paper.
• Don't evaluate the solution. Evaluations, usually empirical, are necessary for publication in today's journal papers. Omitting an evaluation makes it easy for the reviewers to reject your paper.
• Don't connect the logical dots. Publishable papers present the problem, a solution to that problem, and then an evaluation of the solution. If the solution does not match the problem, or the evaluation doesn't determine whether the solution actually solves the problem, reviewers can happily vote reject.
• Write badly and don't edit. Not only does this obscure your points, it frustrates the reviewers so they want to reject your paper, no matter how good the research is.
• Don't include a relevant work section. This makes reviewing easy, because if you didn't reference the reviewer’s paper, yours must be wrong!
• Don't motivate your work. Reviewers love to write “Why in the heck are you doing this?” If you leave out the motivation, reviewers can happily and painlessly claim the research is pointless.
• Don't admit limitations to the research. This means any limitation the reviewer finds is a significant flaw in the research. This works best if the limitations are obvious, otherwise the reviewers might miss them.
• Send to the wrong venue. This saves the reviewers lots of time—they only have to read the title and abstract. With luck, the editor will notice and desk-reject the paper, saving time for everybody.
• Don't revise accepted conference papers. This strategy is for future planning and is a little bit subtle. The current paper is already accepted (no matter how hard you try, you can't get all your papers rejected), but the next time the reviewers read one of your papers, they will remember and reject it. So the payoff is in the future, but the potential is to have lots of papers rejected.
• Get mad about criticism. For example, you could write in a revision: “On this comment, reviewer #1 was being a moron, and we refuse to change the paper for morons.” This not only gets your paper rejected, it’s fun!
• Assume reviewers are smart. I have reviewed hundreds of research papers and can assure you, the first thing I do is put on my “stupid hat.” If an implication or result is not spelled out in simple, two letter words, reviewers will not understand.
• Criticize the reviewers in responses. Here is a good comment to use: “Based on this comment, it’s clear to us that reviewer #2 is not qualified to review this paper.” Reviewers love this, as do the editors who invited the reviewers.
• View a “revise and resubmit” decision as a rejection. For example, if you get this statement from a journal: “Taking into account the comments from the three expert reviewers, the journal cannot accept your paper in its current form, but you may undertake a major revision and submit again.” By not revising, you get the opportunity to self-reject!
• Use “et al.” in the reference list. Whose name did you omit in the author list? Hopefully the reviewer's name, a strategy that is certain to annoy the reviewer enough to reject your paper.

I trust these suggestions will help you achieve your goal of maximizing the number of papers that get rejected. They have been working well for me, and I expect them to continue to do so for many years.

As editor-in-chief, however, I view my job as finding good papers to publish in STVR. So in my next editorial, I will suggest some strategies for getting your paper accepted.

[1] Merriam-Webster Online Dictionary, http://www.merriam-webster.com/, last access July 2014.

Jeff Offutt
offutt@gmu.edu
23 July 2014