Why should I review papers?

Published in volume 17, issue 4, December 2007

This issue has three fascinating papers. The first, Software component composition: A subdomain-based testing foundation, by Hamlet, proposes a theory of composing software components that is based on testing. Instead of modeling or specifying software systems directly, this paper suggests the novel approach of deriving abstractions of components from testing, and then using these abstractions to model the behavior of the entire system. The second paper, A combinatorial testing strategy for concurrent programs, by Lei, Carver, Kacker and Kung, addresses the hard problem of testing concurrent software. Instead of trying to test all possible synchronization sequences, this paper presents a method and algorithm for choosing a subset of synchronization sequences that will lead to effective fault detection. The third paper, Studying the separability relation between finite state machines, by Spitsyna, El-Fakih and Yevtushenko, explores relationships between finite state machines. It quantifies the "distance" between nondeterministic FSMs.

Due to the difficulty of soliciting reviews, one of these papers has actually been in STVR's reviewing process for more than two years, which brings me to the main subject of this editorial ...

One of the most important jobs of editors and associate editors is soliciting reviews. Reviewing takes time that we don't get paid for. The effort counts for almost nothing in promotion and tenure decisions. Not surprising, a perennial problem for most journals is getting reviews in a timely fashion. So I'm not surprised when I hear the question: "why should I review papers?"

Luckily I can offer several good answers.

The most obvious reason is that it is a service to the community. Service is good for the soul, and donating our time to our field makes us feel better about ourselves.

Another obvious reason is to return the favor. Every time we submit a paper to a journal, four or five people work for us: three reviewers, an editor, and possibly an editor-in-chief. We don't pay them in money, but we can pay for their time by reviewing papers.

A less obvious reason is to learn. Reviewing papers is a good way to learn new results early. Of course it's unethical to use results before they're published, but the reviewers know the results as soon as they're published. Moreover, the best ideas don't just give you ideas upon which to build your research, they change the way you think. And changing your thinking earlier than others always helps.

Another thing we learn is how to write papers. Seeing good (and bad!) examples help us develop and refine our own writing skills.

Perhaps the most subtle reason for reviewing is to enhance our reputations. This is especially true for young scientists. Reviewing papers lets us help the senior scientists who ask for the reviews, and we can build a reputation as for being a team player who support journals. If our reviews are high quality, the editors and associate editors notice and remember. A reputation is the most important intangible professional asset we have. Spending time to build that reputation pays off in unexpected ways.

The complement for a senior scientist is that reviewing allows us to help shape the field by giving back our experience. This is a way to teach the authors, many of whom are professionally younger.

The time required to review a paper varies tremendously by the subject, paper, and reviewer. Theoretical papers take longer, and if the reviewer has read many of the reference, the review takes less time. Conscientious reviewers will spend between one and four hours for a conference paper and between four and 12 hours for a journal paper. As we gain experience, we tend to spend less time. Some reviewers will focus on the big picture issues and some will focus more on details (which takes more time).

So we have many good reasons to review papers. Of course, it's always okay to say "no." Although refusing a review request also influences your reputation, nobody can review every paper offered and it's far better to say "no" at the beginning than to say "yes" and then be very slow. The most important thing is to respond to every request. A fast "no" allows the editor to go to the next person on the list, whereas a slow "no" (or no response!) introduces delay into the process, which hurts the authors and the journal. And returning that review within a reasonable time is crucially important. We've all sent off papers to journals only to have to wait ... and wait ... and wait. I'm glad to report that with the help of our wonderful editorial board and reviewers, STVR has made significant progress on timeliness. Time to review has gone from close to a year to about five months. My goal as editor is to reduce that further to three months.

Jeff Offutt
16 October 2007