This issue has two exciting papers. The first, Modelling methods for web application verification and testing: state of the art, by Alalfi, Cordy and Dean, provides a comprehensive survey of models that are used to help us evaluate the quality of web applications. The second, System testing for object-oriented systems with test case prioritization, by Kundu, Sarma, Samanta and Mall, presents a method to generate tests from UML sequence diagrams.
I recently moved into a new building, and not surprisingly, it made me think about testing. (Right, everything does.) Although I am happy here, it has its flaws like any building. Lots of people have drawn analogies between software engineering and civil engineering, and I took this opportunity to reflect on how the analogy applies to testing.
One of the first things I noticed is that my blinds can only be full open or full closed, not part way. Missing functionality! The first day I noticed a space at the bottom of a stairwell with no access. Dead code!
Our small seminar room has a built-in projector and electric screen. Very nice! But the ceiling lights are either all on or all off, and there is no way to turn off just the lights in front of the screen. Usability flaw! Missing requirement? Connecting the computer to the projector requires running a wire across the carpet. Another usability flaw! The acoustics in the large seminar room downstairs is so bad that speakers must use a microphone. Design flaw, usability problem!
Several bathroom stall doors do not latch properly because they were hung incorrectly. A classic integration fault! The elevator lights indicate the wrong elevator is arriving, but only on one floor. Unit fault! (Since this is the CS floor, the students claim a software fault, but my guess is crossed wiring.)
Several rooms abut a large glass wall. Because the carpenters did not know how to connect dry wall to glass, they left 6 inch gaps between the office walls and the glass wall. Architectural mistake!
When we moved into our offices, we found that some rooms had thermostats in the middle of the wall where we planned to put our whiteboards, and all of us had electrical outlets behind our furniture. Integration faults!
Another fault that turns out to be a feature is the elevators are very slow. The designers claim this was intentional to encourage use of stairs (it's a "green building," after all). Just like in software, "it's a feature, not a flaw!"
My favorite is our back door. It has a security control and is meant to be locked at all times. Unfortunately, if we simply release the door, it doesn't latch. So the door is seldom closed! The door is fine, the frame is fine, but the door was hung incorrectly. This is an integration fault that leads to a classic emergent property failure: a security flaw.
This analysis is fun, but it also tells us that we have the same problems in building construction as in software construction. They've been putting up buildings for thousands of years and they still don't get it right! It also reminds us that there is no such thing as correctness in any but the tiniest of engineering constructs. We would never consider asking whether a building is "correct." Or a car, or an airplane. So why do we ask if computer programs, which are orders of magnitude more complicated than a building, are correct? As George Box said, "All models are wrong, some are useful."
A more interesting question is who is the tester? Who "tests" buildings? When I had a deck added to the back of my house, the county sent inspectors to check the design, the postholes, the concrete bases, the framing, the final deck, and the electrical wiring. The inspectors are testers! And how does somebody get to be an inspector for construction projects? By being very knowledgeable; often one of the best engineers, carpenters, electricians, or plumbers!
Software engineering will not be considered a mature field until we match all traditional engineering disciplines that operate in the physical domain. That is, until the best programmers and designers hope to be promoted to testing.
Jeff Offutt
offutt@gmu.edu
11 November 2009