Globalization—References and Citations

Published in volume 24, issue 1, January 2014

This issue has three exciting papers that show how test tools can help in areas ranging from model-based testing to embedded software to web application software. The first, Tool Support for the Test Template Framework, by Cristiá, Albertengo, Frydman, Plüss, and Rodríguez Monetti, describes a new tool to support model-based testing with Z specifications. Their tool, Fastest, is open-source and available online. (Recommended by Paul Strooper.) The second, Model Checking Trampoline OS: A Case Study on Safety Analysis for Automotive Software, by Choi, presents a study of the use of model checking to check safety properties in automotive operating systems. The author was able to find evidence of safety problems in the Trampoline operating system. (Recommended by Jeff Offutt.) The third, Design and Industrial Evaluation of a Tool Supporting Semi-Automated Website Testing, by Mahmud, Cypher, Haber, and Lau, presents experience from using a test automation tool, CoTester, in practical situations. They found that the tool was useful for both professional testers and non-professional testers, but most useful for the non-professional testers. (Recommended by Per Runeson.)


I wrote about The Globalization of Software Engineering in a previous editorial [1], and followed up with a discussion on language skills to support globalization [2]. Another difference I have noticed is in how the scientific community uses citations and references, and more interestingly, which citations they use. Many of these differences are personal and individual, but some seem cultural.

I will first discuss citations in general with some thoughts I share with my PhD students, then talk about some cultural differences from my experience.

The first principle is that references must help readers understand the paper. Of course, this is so broad that it's not much help, but it's an important starting point. References have an important role in research papers. They need to explain what the paper is based on (context), they indicate what the authors know about the subject, and they summarize what the readers should know to understand this paper. Reviewers also use references as a proxy for the measure of care the authors take with their research.

Reviewers also expect certain rules to be followed. The most important is ethical: Never reference something you haven't read. A secondary citation, where we write something like (Parnas [4] "as cited by Burdell [15]"), indicates that you read Burdell's paper and he referenced Parnas' paper. This should only be used when absolutely necessary if the original reference is unavailable. It's also important to list all authors in a reference list; "et al." is okay in the text, but if you leave off the name in the references of a person who reviews your paper, it will not help your chances of being accepted.

Another expectation is that you write the authors names as they appeared in the published paper. So if someone changes his or her name, you should not update the old papers. The final note is about grammar. The citations are parenthetical elements, not nouns. That is "as said in [52]" is grammatically wrong, and should be written as "as said by Liskov [52]." This last may be the most common mistake, made even by established scientists, probably because it is a convenient shortcut.

These ideas are basic, and generally taught in high school and early college writing classes. So our PhD students should already know these rules, and if not, should certainly absorb them before they "leave the nest" and start independent research.

A major question about references is how many? The correct answer is, of course, to use exactly the references the paper needs and no more. But defining what "references the paper needs" is clearly subjective. My general philosophy is that it's better to over-reference than under-reference. We are more likely to confuse readers with too much information than with too little. And as a famous scientist once told me, even famous scientists like to see their names in print. We don't get paid much for publishing papers, so referencing papers is a way to thank those who helped get you to the point where you can write this current paper.

Generally, I have noticed that North Americans tend to include more references than scientists in other parts of the world, whereas scientists from Asia tend to include fewer. I don't know why.

Of greater concern is that some scientists tend to cite many papers from other scientists in their region, but not as many from other regions. So Americans often cite a lot of papers by American authors but not so many from Europe, French scientists sometimes cite lots of French papers but not many from Asia, and so on. In fact, I recently reviewed a paper by French authors where 17 of 22 references were by themselves or other French authors, and omitted key papers by non-French authors. This is probably extreme, but in my experience, fairly common.

Generally, the lesson is that we need to make sure we are not being parochial in our study of the literature. Globalization means that quality research is done all over and good papers can come from anywhere.

[1] Jeff Offutt. The Globalization of Software Engineering (Editorial), Wiley's journal of Software Testing, Verification, and Reliability, 23(3), May 2013. http://www.cs.gmu.edu/~offutt/stvr/23-3-May2013.html (last access December 2013).

[2] Jeff Offutt. Globalization—Language and Dialects (Editorial), Wiley's journal of Software Testing, Verification, and Reliability, 23(4), June 2013. http://www.cs.gmu.edu/~offutt/stvr/23-4-June2013.html.

Jeff Offutt
9 December 2014