ICST Steering Committee

for the International Conference on
Software Testing, Verification and Validation (ICST)
(Effective August 2007)
(Last update, October 2019)

This document describes the regulations and guidelines to help with organizing and managing the IEEE ICST conference. Both organizers and program committee members must abide by the regulations (sections 1, 2 and 3) and follow the guidelines to the extent possible.

I. Goals and scope for the ICST conference

The new IEEE International Conference on Software Testing Verification and Validation (ICST) offers an open forum for software testing, verification and validation research and its transfer to practice. One of the main goals of ICST is to bridge research and practice in software testing, verification, and validation. Furthermore, it aims at stimulating scientific research on model-based software testing, domain specific testing, empirical studies of testing techniques, and the technology transfer of research results to software development practices. The charter of ICST specifies rules ensuring the transparent, democratic, and open management of the conference.

II. Regulations

1. Steering Committee (SC)

1.1. Role and Composition

1.2. Election of Steering Committee Members

1.3. Election of the Steering Committee Chair (SCC)

1.4. Website

2. General Chair (GC) and Program Chair (PC)

3. Technical Program Committee Selection

The composition of a technical program committee is one of the most important factors in determining the quality of a conference and the quality of the papers it accepts. This in turn reflects on the long-term prestige of the conference and ultimately on the impact it is able to have. The ICST Steering Committee has thus adopted a set of guidelines for use by ICST Program Chairs in assembling a technical program committee (guidelines provided in next section). Program Chairs are asked to submit their list of prospective TPC members to the Steering Committee Chair at least one month prior to the desired date for sending out invitations to the TPC. If a prospective TPC member, or the list of members as a whole, does not meet these guidelines, the Program Chairs should indicate this, and provide arguments for these cases. The Steering Committee will consider, on a case by case basis, such arguments, as it evaluates the TPC list as a whole. The Steering Committee reserves the right to require changes in TPC composition based on these guidelines.

4. Guidelines to Form a Technical Program Committee

  1. ICST reviewers need to be able to judge whether research submitted to ICST is technically sound, contributes to the field, and is novel with respect to previous work. To ensure that reviewers have this qualification, ICST requires that to serve on the ICST TPC, a person must have had an active and documented role in the field of software testing, verification, and validation in the preceding 5 years. Although we could define a metric with which to assess qualifications, we prefer not to do so in order to allow flexibility. However, on prospective TPC members' whose qualifications in this regard are not clear by some obvious metric such as relevant publications, the TPC Chairs should briefly summarize those member's qualifications when presenting their list to the Steering Committee.
  2. ICST reviewers need to be familiar with ICST itself, and with the review process, from the point of view of an author. To ensure this, we require that first-time TPC members must have previously authored or co-authored at least one paper that has been accepted by ICST.
  3. Technical program committees require continuity to ensure that conference goals can continue to be met. It is also important, however, that TPCs make room for new members, and that Program Chairs do not feel obliged to retain, for historical reasons, committee members who do not fully participate in the process of preparing high-quality reviews. Thus, TPCs must explicitly incorporate a process of rotating members on and off of the TPC, as follows:
    1. No TPC member shall serve on more than three consecutive TPCs, following which they must be omitted from the TPC for at least one year.
    2. The PCs must explicitly seek permission from the SC to remove more than 50% of the previous year's TPC.
  4. To help ensure the success of future ICSTs, the Program Chairs and General Chair associated with a future ICST should be on the TPC for ICST for the year preceding that ICST. (In this instance it is permissible to violate guideline 3a.)
  5. Given the need to provide effective reviews in each of the many sub-areas of research covered by ICST, Program Chairs must ensure that the TPC includes members whose areas of expertise sufficiently cover those areas of research.
  6. Given the desire to continue to project ICST as an international and inclusive conference, Program Chairs should make every effort to achieve diversity on the TPC with respect to gender, geographic distribution, experience, and industry versus academic experience.
  7. The quality of a conference's reviews is central to the view authors have of the conference, and to the conference's subsequent prestige. This in turn affects the conference's ability to attract good work and have an impact. Therefore, Program Chairs should make every effort to invite TPC members who are expected to and agree to abide by the following proscriptions as a condition of serving on the TPC.
    1. TPC members are responsible for their own reviews. Although it is acceptable to obtain co-reviewers such as students, who may provide relevant expertise, it is not the student who is on the TPC, it is the TPC member, and ultimately that member is responsible for the review and must be personally able to argue for or against the merits of the paper.
    2. TPC members must play an active role in helping authors improve their papers. Reviews should contain details sufficient to support their conclusions, and reviews should be constructive, offering comments on how papers can be improved. To ensure that potential TPC members are aware of and agree with these proscriptions, Program Chairs should include them prominently in any invitation sent out to prospective TPC members, stating that acceptance of the invitation implies agreement with the proscriptions.
  8. SC members are de-facto members of the TPC. In consideration of their responsibilities to oversee the conference, they will normally be given a light reviewing load, usually of two papers or less.
  9. To determine the size of a TPC, calculate an estimate of paper submissions E given the submission numbers from the preceding two conferences, and determine the number of TPC members necessary to handle E papers, consistent with having 3 reviewers per paper, and a reviewing load of between 7 and 10 papers per member.
  10. When necessary, the Program Chairs should transmit a list of reviewers who did not perform their duties to acceptable standards to the SC in order to continuously improve the quality of reviews.
  11. The TPC could decide to shepherd some papers or to consider a two-rounds review process where the authors have a chance to respond to the reviews before final decisions are made.
  12. The TPC may not accept papers submitted as a full-length paper on the condition that they are cut to become a “short paper.” It is permissible to recommend authors to resubmit rejected papers to a different track, but only if that track was defined in the original CFP.

5. Location and Schedule

6. Industry - Academia Mix

7. Balancing Research and Practice

III. Special rules for initiating the conference

IV. Changelog

All changes must be discussed and approved by the Steering Committee by a majority vote.

Last update 16 October, 2019 by Jeff Offutt.