Charter
for the International Conference on
Software Testing, Verification and Validation (ICST)
(Effective August 2007)
(Last update, October 2019)

This document describes the regulations and guidelines to help with organizing and managing the IEEE ICST conference. Both organizers and program committee members must abide by the regulations (sections 1, 2 and 3) and follow the guidelines to the extent possible.

I. Goals and scope for the ICST conference

The new IEEE International Conference on Software Testing Verification and Validation (ICST) offers an open forum for software testing, verification and validation research and its transfer to practice. One of the main goals of ICST is to bridge research and practice in software testing, verification, and validation. Furthermore, it aims at stimulating scientific research on model-based software testing, domain specific testing, empirical studies of testing techniques, and the technology transfer of research results to software development practices. The charter of ICST specifies rules ensuring the transparent, democratic, and open management of the conference.

II. Regulations

1. Steering Committee (SC)

1.1. Role and Composition

• The SC has the responsibility to strategically lead the conference in the long term. This responsibility includes implementing this charter, selecting future conferences, and setting the overall goals for the directions of the conference and the technical community. While the primary task of the SC is governance of the conferences, it also takes on the role of mentoring.
• Member Tenure: 3 years
• Maximum number of consecutive terms: 2
• Recommended Size: 9
• New members are selected by election (see next subsection).
• Elections are administered by a committee appointed by the SC.
• Elected SC members are expected to attend at least two out of three editions of ICST during their terms.
• The SC membership should maintain a diverse group of people from both academia and industry.
• The SC should recruit members who are leaders and visionaries. They must be able to positively impact the development and growth of software testing and ICST.
• The General Chair from the most recent ICST conference is automatically a non-voting, ex officio member for the year following the conference.

1.2. Election of Steering Committee Members

• The SC candidates must have served in an official capacity at ICST conferences within the past 5 years. Official capacity is defined as those who have served in the roles of an ICST “conference committee” or “program committee” member.
• This selection can take place at the conference itself in an open meeting or be organized through email by the SC chair. The first is preferred.

1.3. Election of the Steering Committee Chair (SCC)

• The Steering Committee Chair must be a visionary leader who can positively impact the conference future.
• The SCC is elected by the Steering Committee.
• Steering Committee Chair Tenure: 3 years
• Maximum number of consecutive terms: 2

1.4. Website

• The SC is responsible for maintaining a website with relevant statistics on past conferences and an updated version of the present charter.

2. General Chair (GC) and Program Chair (PC)

• GC submits a proposal to chair a conference. The GC should preferably have prior leadership experience in the software testing community, or be appropriately teamed.
• The GC should recommend PCs to the SC. Final decision is by the SC.
• The Technical Program Committee (TPC) membership should change yearly, balancing a desire to bring in new people and maintain continuity (see next section on forming the TPC).
• The role of the GC and PC is to execute the specific conference they are chosen to lead. The GC and PC of a future year must be active in the current (or prior) year's conference, in some capacity, so as to gain experience.
• The GC and PC will be responsible and accountable for the planning and execution of an ICST conference.
• GC and PC should have served as a program committee member to qualify.
• Conference selection can be up to 3 years in advance, and no less than 18 months.
• The GC shall issue periodic reports on the status of the planning and execution of the conference to the SC. The GC and PC should use the SC for advice in planning for the conference.
• GCs and PCs cannot submit papers to any tracks of the main conference, but can submit to associated workshops. Chairs of tracks other than the main research track can submit to any track, with the exclusion of their own track.

3. Technical Program Committee Selection

The composition of a technical program committee is one of the most important factors in determining the quality of a conference and the quality of the papers it accepts. This in turn reflects on the long-term prestige of the conference and ultimately on the impact it is able to have. The ICST Steering Committee has thus adopted a set of guidelines for use by ICST Program Chairs in assembling a technical program committee (guidelines provided in next section). Program Chairs are asked to submit their list of prospective TPC members to the Steering Committee Chair at least one month prior to the desired date for sending out invitations to the TPC. If a prospective TPC member, or the list of members as a whole, does not meet these guidelines, the Program Chairs should indicate this, and provide arguments for these cases. The Steering Committee will consider, on a case by case basis, such arguments, as it evaluates the TPC list as a whole. The Steering Committee reserves the right to require changes in TPC composition based on these guidelines.

4. Guidelines to Form a Technical Program Committee

1. ICST reviewers need to be able to judge whether research submitted to ICST is technically sound, contributes to the field, and is novel with respect to previous work. To ensure that reviewers have this qualification, ICST requires that to serve on the ICST TPC, a person must have had an active and documented role in the field of software testing, verification, and validation in the preceding 5 years. Although we could define a metric with which to assess qualifications, we prefer not to do so in order to allow flexibility. However, on prospective TPC members' whose qualifications in this regard are not clear by some obvious metric such as relevant publications, the TPC Chairs should briefly summarize those member's qualifications when presenting their list to the Steering Committee.
2. ICST reviewers need to be familiar with ICST itself, and with the review process, from the point of view of an author. To ensure this, we require that first-time TPC members must have previously authored or co-authored at least one paper that has been accepted by ICST.
3. Technical program committees require continuity to ensure that conference goals can continue to be met. It is also important, however, that TPCs make room for new members, and that Program Chairs do not feel obliged to retain, for historical reasons, committee members who do not fully participate in the process of preparing high-quality reviews. Thus, TPCs must explicitly incorporate a process of rotating members on and off of the TPC, as follows:
   1. No TPC member shall serve on more than three consecutive TPCs, following which they must be omitted from the TPC for at least one year.
   2. The PCs must explicitly seek permission from the SC to remove more than 50% of the previous year's TPC.
4. To help ensure the success of future ICSTs, the Program Chairs and General Chair associated with a future ICST should be on the TPC for ICST for the year preceding that ICST. (In this instance it is permissible to violate guideline 3a.)
5. Given the need to provide effective reviews in each of the many sub-areas of research covered by ICST, Program Chairs must ensure that the TPC includes members whose areas of expertise sufficiently cover those areas of research.
6. Given the desire to continue to project ICST as an international and inclusive conference, Program Chairs should make every effort to achieve diversity on the TPC with respect to gender, geographic distribution, experience, and industry versus academic experience.
7. The quality of a conference's reviews is central to the view authors have of the conference, and to the conference's subsequent prestige. This in turn affects the conference's ability to attract good work and have an impact. Therefore, Program Chairs should make every effort to invite TPC members who are expected to and agree to abide by the following proscriptions as a condition of serving on the TPC.
   1. TPC members are responsible for their own reviews. Although it is acceptable to obtain co-reviewers such as students, who may provide relevant expertise, it is not the student who is on the TPC, it is the TPC member, and ultimately that member is responsible for the review and must be personally able to argue for or against the merits of the paper.
   2. TPC members must play an active role in helping authors improve their papers. Reviews should contain details sufficient to support their conclusions, and reviews should be constructive, offering comments on how papers can be improved. To ensure that potential TPC members are aware of and agree with these proscriptions, Program Chairs should include them prominently in any invitation sent out to prospective TPC members, stating that acceptance of the invitation implies agreement with the proscriptions.
8. SC members are de-facto members of the TPC. In consideration of their responsibilities to oversee the conference, they will normally be given a light reviewing load, usually of two papers or less.
9. To determine the size of a TPC, calculate an estimate of paper submissions E given the submission numbers from the preceding two conferences, and determine the number of TPC members necessary to handle E papers, consistent with having 3 reviewers per paper, and a reviewing load of between 7 and 10 papers per member.
10. When necessary, the Program Chairs should transmit a list of reviewers who did not perform their duties to acceptable standards to the SC in order to continuously improve the quality of reviews.
11. The TPC could decide to shepherd some papers or to consider a two-rounds review process where the authors have a chance to respond to the reviews before final decisions are made.
12. The TPC may not accept papers submitted as a full-length paper on the condition that they are cut to become a “short paper.” It is permissible to recommend authors to resubmit rejected papers to a different track, but only if that track was defined in the original CFP.

5. Location and Schedule

• The location should be practical to facilitate growth and quality of the conference.
• The conference should periodically be held in locations in North America and outside North America.
• The ICST conference should take place around the end of March to avoid conflicts with other conferences.

6. Industry - Academia Mix

• The conference should maintain or elevate high academic standards on the part of the research.
• The conference should foster strong industry-academic interaction and applications of testing research to industry.
• The conference should include programs and tracks that focus on academic needs.
• The conference should include programs and tracks that focus on industrial needs.
• Academic and industry tracks can be separate, on separate days, to increase interaction and synergy, rather than competition.

7. Balancing Research and Practice

• ICST programs should provide a balance of research and practice.
• Distinct programs for research and practice should be visible.
• Papers and presentations from both research and practice should excel in their own spaces and domains.
• The GC should appoint chairs and sub program committees to manage research and practice programs by their own standards.
• The conference should strategically avoid the tendency to become a “research only” conference as a channel for academics to publish “respectable” papers and satisfy only the needs of the academic community.
• The conference also should strategically avoid the alternate tendency to become a practice only conference.
• The conference should compare itself to the most successful conferences as a benchmark. Examples are OOPSLA and ICSE which have gained industry and academic respect and should be looked upon as models.

III. Special rules for initiating the conference

• Election of the first SC
  • Lionel Briand and Benoit Baudry are in charge of the election of the first SC.
  • Nine members from the group that initially supported the creation of the conference (Supporter Group - SG) will be voted in as SC members.
  • All initial supporters can vote.
  • The vote will take place by email. Each member of the SG will select a maximum of nine names. The nine people with the largest number of votes will then form the first steering committee.
  • To avoid renewing the SC entirely after 3 years after the first edition, we propose a specific process to renew one third of the SC each year. The group of people elected for the first SC will be divided into three tiers depending on the number of votes they receive: these three tiers will be elected for three years, two years, and one year, respectively.
• Legislation of the Charter
  • The first SC to be elected will be in charge of finalizing and voting the charter.
• Building the initial TPCs
  • The TPC is expected to be small the first year and grow progressively. As a result, we do not expect a third of the TPC members to be renewed after three years, despite the rule stated above. An alternative is to have a transition period after 3 years, over 2 or 3 years, where we progressively renew the members of the first TPC.

IV. Changelog

All changes must be discussed and approved by the Steering Committee by a majority vote.

• Changed October 2019, section II.1.1. Added rule that SC members should attend 2 of 3 editions of ICST.
• Changed March 2017, section II.4. Added rule 12, no short papers.
  Reason: Accepting a paper as a “short paper” corrupts the peer-review process.
• Changed April 2016, section II.1.1, first bullet. Changed “Both the electorates and SC candidates” to “The SC candidates”.
  Reason: It was never intended for voters to have served in an official capacity.
• Changed February 2015, section II.2, final bullet. Change “GCs and PCs cannot submit papers to the conference” to “GCs and PCs cannot submit papers to any tracks of the main conference, but can submit to associated workshops. Chairs of tracks other than the main research track can submit to any track, with the exclusion of their own track.”
  Reason: Clarify roles of non-research track chairs.
• Deleted March 2010, section II.4.3.b: On each TPC, between 10% and 30% of the members must be new with respect to the preceding year's TPC.
  Reason: Rule conflicted with II.4.3.b, and having precise percentages overly constraint PCs.
• Added March 2010, section II.4.3.b: The PCs must explicitly seek permission from the SC to remove more than 50% of the previous year's TPC.
  Reason: To preserve continuity.
• Added May 2009, section II.1.1: The General Chair from the most recent ICST conference is automatically a non-voting, ex officio member for the year following the conference.
  Reason: To provide current advice and insight.