Updated May-2016.
Wiley uses Manuscript Central (MC), which implements a fully automated workflow for paper submissions, reviews, and decisions. This page describes the process that members of the Editorial Board use to handle papers, that is, elicit reviews and make recommendations to the EiC. MC uses the term “Review Editor,” which this document abbreviates as RE. The process is as follows:
STVR only allows one major revision. If a paper needs a second major revision, it should be rejected. If the paper has potential, the authors can be encouraged to re-work the paper and resubmit a different paper.
Revisions will normally be assigned to the same RE who handled the original paper. Major revisions should go to the same reviewers if possible; if not, new reviewers should be given the original reviews. Reviewers should be encouraged to comment on other reviews; for example, to help the editor recognize mistakes by reviewers. Minor revisions can be assessed by the RE, or the RE can ask any or all of the original reviewers to look at the revision. It is common, for example, to ask a reviewer who raised key concerns to look at a minor revision.
When analyzing reviews, your job is to make a fair, unbiased decision. This means that you have to arbitrate among differences, and when reviewers disagree, understand why and find the truth about the paper. Remember that reviews are ONLY inputs and acceptance is not based on a voting system. The editor makes the decision and can deliberately choose to ignore some comments. We are trying to strike a balance between not rejecting good (or potentially good) papers and not accepting poor papers. The journal has high standards, but do try to help the authors find a good paper if it’s there. Authors deserve the chance to improve papers.
It is perfectly acceptable, and even encouraged, to ask reviewers to modify their reviews. Reviews should be respectful, professional, impersonal, and objective, no matter how poor the paper is. It is also okay to and ask a reviewer a question such as “You said A but reviewer 2 said B, do you think it's possible that reviewer 2 may have seen something you did not?”. (Be sure to protect the anonymity of reviewers!)
Your assessment needs to be more than a simple recommendation of reject, major revision, minor revision, or accept. In particular, you need to specify the precise set of expectations to be fulfilled for the journal to accept the paper (for a revision), or the critical problems that led to rejection. This summary of the reviews is very helpful to the authors and very important to the journal’s reputation.
I would prefer your recommendations to me to be very direct, clear, and explicit. This is a private, confidential communication, and any suggestions will be appropriately edited before anything is sent to the authors.
Choosing reviewers is an art. STVR requires each paper to be reviewed by at least 3 people. It’s best if the 3 have some balance in terms of levels of expertise and experience. Every paper needs to be evaluated by a true expert on the topic, but it also helps to have a reviewer to represent the rest of STVR's readers to ensure that the paper is understandable outside of the small circle of experts. While experience certainly helps with reviewers, I often find that younger scientists to be more responsive and energetic.
You can find reviewers among your circle of professional colleagues, from the references in the paper, among authors (and editorial board members) from STVR, and from a web search on key words. We also occasionally recommend reviewers. I strongly encourage each of you to develop a system to track how responsive and helpful each reviewer is. As many of you know, one of the greatest frustrations among editors is chasing after reviewers. Try to allow for failure by doing things like selecting 4 or 5 names and initially inviting 3, and having backup emergency reviewers.
You should not be a reviewer. Your role is to independently and objectively arbitrate among three reviewers, and if you are one of the reviewers it becomes very hard to be impartial.
Time to making decisions on papers is a constant problem for all journals and we are no different, but it's certainly a problem that we can work together to solve.
Reducing time to decision is about managing the steps in the process above. MY GOAL IS TO HAVE DECISIONS ON PAPERS WITHIN 3 MONTHS OF SUBMISSION. This is an ambitious goal, which I will bracket by stating that any paper that takes more than 6 months represents a serious problem. Satisfying this goal is up to you. And it's not that hard, we just need to manage the process.
I expect to assign papers to REs within 5 days of submission. It is very important that you respond as quickly as possible, especially if you cannot handle the paper. The biggest source of delay is people who wait to say “no”—that creates two delays; the delay to respond and the delay to find someone else.
I hope you can get papers to reviewers within one week. It takes a couple of hours to select potential reviewers, and most reviewers will respond within a day or two. Again, the biggest source of delay is when potential reviewers take the 5 days we allocate, then say “no.” The second source of delay is when you have to contact many reviewers to get three.
We ask reviewers to respond within 6 weeks, which is very reasonable and gives plenty of time. I strongly suggest that after 8 weeks, you contact a 4th reviewer and ask for a quick turnaround. Once you get the third review, be sure to inform the slow reviewer that the review is no longer necessary. We are all busy, but that person made a commitment and should be able to honor it. Telling the slow reviewer is a professional courtesy (he or she can remove the task from the task list), and also sends a message that we are serious. If both the 4th and 3rd reviewer submits a review simultaneously, then the author will benefit from extra help.
Once 3 reviews are in, I hope you can send me a recommendation within one week. I will plan to make a final decision within one additional week.
This is a total of about 10 weeks. Even with some small delays, there is enough slack so that we should reasonably be able to make a decision within 3 months of submission. Unusual delays should not do more than double the time—hence my 6 month rule.
I hope you realize that deciding which papers are published in a major international journal is a big responsibility. The decisions we make impact people's careers, just as previous decisions impacted our careers. As decision-makers for the major journal in software testing, we are care-takers for the field. We have a professional obligation to take this responsibility seriously and respectfully.
I look forward to continuing to work with you.
Jeff Offutt
Editor in Chief, STVR
27 May 2016 (first version 28 June, 2007)
=======================================
Wiley’s Software Testing, Verification and Reliability
Useful research in making better software.
http://www.interscience.wiley.com/journal/softwaretesting
=======================================