| Course Schedule |
Instructor: Dr. Jeff Offutt, offutt@gmu.edu Prerequisite: SWE 637 (or permission of instructor; experience is also relevant)
Many large software companies report that 50% or more of software development resources are devoted to testing. In broad terms, the goal of testing software is to find, and fix, software faults in-house before they are revealed to users as failures. Studies have found that user-revealed faults cost up to 100 times more to debug and repair as faults found during development. As software grows in complexity and cost, is used in more applications, and is based on more technologies, this cost is fast becoming unsustainable.
Updated schedule as of 13 February.
| Week | Topic | Readings |
| Week 1, 26 January | Introductions and discussion of topics | |
| Week 2, 2 February | Finalizing topics & schedule
Project selection discussions Web app testing introduction (slides) | Piazza
Quality Attributes of Web Software Applications, Jeff Offutt, IEEE Software: Special Issue on Software Engineering of Internet Software, March/April 2002 Modeling Presentation Layers of Web Applications for Testing, Jeff Offutt and Ye Wu, Springer's Software and Systems Modeling, 9(2):257-280, April 2010 |
| Week 3, 9 February | Testing web applications
Bypass testing (slides) The atomic section model (slides) |
A Case Study on Bypass Testing of Web Applications,
Jeff Offutt, Vasileios Papadimitriou, and Upsorn Praphamontripong,
Springer's Empirical Software Engineering journal, 19(1):69-104, February 2014
Modeling Presentation Layers of Web Applications for Testing, Jeff Offutt and Ye Wu, Springer's Software and Systems Modeling, 9(2):257-280, April 2010 Prioritizing User-session-based Test Cases for Web Applications Testing, Sreedevi Sampath, Renée Bryce, Gokulanand Viswanath, Vani Kandimalla, and Güneş Koru, First International Conference on Software Testing, Verification, and Validation, April 2008 Applying Mutation Testing to Web Applications, Upsorn Praphamontripong and Jeff Offutt, Sixth Workshop on Mutation Analysis, April 2010, Paris, France |
| Week 4, 16 February | Testing web applications
User-session-based testing (slides) Mutation testing of web apps (slides) | |
| Week 5, 23 February | Testing object-oriented software
Speaker: Anastasia Harsch Overview of OO language features (slides) Criteria for testing OO features (slides) |
Testing Object Oriented Software: A Survey,
Robert V. Binder,
Wiley's Software Testing, Verification and Reliability, 6(3-4):125-252, December 1996
Testing Object-oriented Software Systems, Harry M. Sneed, 24th European Conference on Object-Oriented Programming (ECOOP), Maribor, Slovenia, June 2010 (Anastasia) Testing Coupling Relationships in Object-Oriented Programs, Roger T. Alexander, Jeff Offutt, and Andreas Stefik Wiley's Journal of Software Testing, Verification, and Reliability, 20(4):291-327, December 2010 |
| Week 6, 2 March | Testing mobile applications
Speaker: Lin Deng |
Dynodroid: an input generation system for Android apps,
Aravind Machiry, Rohan Tahiliani, and Mayur Naik,
9th ACM Joint Meeting on Foundations of Software Engineering (ESEC/FSE), Saint Petersburg Russia, 2013, pages 224-234
Automated Test Input Generation for Android: Are We There Yet?, Shauvik Roy Choudhary, Alessandra Gorla, and Alessandro Orso, 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), Bergamo, Italy, 2015, pages 429-440. Sapienz: multi-objective automated testing for Android applications, Ke Mao, Mark Harman, and Yue Jia, 25th International Symposium on Software Testing and Analysis (ISSTA), Saarbrücken Germany, 2016, pages 94-105 Mutation Operators for Testing Android Apps, Lin Deng, Jeff Offutt, Paul Ammann, and Nariman Mirzaei, Elsevier's Information and Software Technology, 81:154-168, January 2017 |
| Week 7, 9 March | Test automation
Speaker: Eric Mingo Test oracle strategies (slides) |
Software Test Automation Practices in Agile Development Environment: An Industry Experience Report,
Eliane Figueiredo Collins and Vicente Ferreira de Lucena, Jr.,
7th International Workshop on Automation of Software Test, Pages 57-63, Zurich, Switzerland, June 2012
(Eric)
Test Automation in the Real World—2016, (results from a survey monkey poll) New survey reveals test automation trends, Cecilia Rehn, Software Testing News August 2015 (summary of a longer article behind a pay wall) Test Oracle Strategies for Model-based Testing, Nan Li and Jeff Offutt, IEEE Transactions on Software Engineering, To appear, 2017 |
| 16 March | Spring break—no class | |
| Week 9, 23 March | Testing for security
Speaker: Ryan Johnson Overview (slides) Java applications (slides) Web applications (slides) DoS in Android (slides) |
Software Security Testing,
Bruce Potter and Gary McGraw,
IEEE Security & Privacy, 2(5):81-85, September-October 2004
Finding Security Vulnerabilities in Java Applications with Static Analysis, V. Benjamin Livshits and Monica S. Lam, 14th conference on USENIX Security Symposium—Volume 14, Berkeley CA, USA, 18-18, 2005 Method Using Command Abstraction Library for Iterative Testing Security of Web Applications, Seiji Munetoh and Nobukazu Yoshioka, International Journal of Secure Software Engineering (IJSSE), 2015, Vol.6(3), pp.26-49 Why Software DoS is Hard to Fix: Denying Access in Embedded Android Platforms, Ryan Johnson, Mohamed Elsabagh, and Angelos Stavrou, International Conference on Applied Cryptography and Network Security (ACNS), Guildford, UK, June 2016, pp 193-211 |
| Week 10, 30 March | Testing the internet of things
Speaker: Nipun Monga Writing Getting papers rejected (slides) Details for writing good (slides) Paper writing and formatting standards (PDF) |
Functional Testing for IoT,
Chris Riley, DeveOps.com blog post, February 2015.
(Nipun)
Security Testing the Internet of Things IoT, author and date not given, Beyond Scurity. (Note this is essentially an advertisement for a company's product, nevertheless, it contains useful ideas.) (Nipun) Testing the Internet of Things: The Human Experience, Gerie Owen, November 2014, InfoQ (online) Breaking Down Mirai: An IoT DDoS Botnet Analysis, Ben Herzberg, Dima Bekerman, and Igal Zeifman, Imperva Incapsula blog (online) |
| Week 11, 6 April | Load testing
Speaker: Rasika Mohod Speaking Giving good presentations (slides) |
A Survey on Load Testing of Large-Scale Software Systems,
Zhen Ming Jiang and Ahmed E. Hassan,
IEEE Transactions on Software Engineering, 41(11):091-1118, November 2015
(Rasika)
Load Testing of Web Sites, Daniel A. Menasce, IEEE Internet Computing, 6(4):70-74, July-August 2002 Automatic Identification of Load Testing Problems, Zhen Ming Jiang, Ahmed E. Hassan, Gilbert Hamann, and Parminder Flora, 2008 IEEE International Conference on Software Maintenance, Beijing, 2008, pp. 307-316 |
| Week 12, 13 April | Usability testing
Speaker: Joffrey Pannee |
Design and Implementation of a toolkit for Usability Testing of Mobile Apps,
Xiaoxiao Ma, Bo Yan, Guanling Chen, Chunhui Zhang, Ke Huang, Jill Drury, and Linzhang Wang.
Mobile Networks and Applications, 18(1):81-97, February 2013 (Joffrey)
Web Evaluation: Heuristic Evaluation vs. User Testing, Wei-Siong Tan, Dahai Liu, and Ram Bishu, International Journal of Industrial Ergonomics, 2009, 39(4), pp.621-627 |
| Week 13, 20 April | Prof out of town, work on projects | N/A |
| Week 14, 27 April | Project presentations • Lin Deng • Ryan Johnson • Anastasia Mingo | • An Empirical Study of Identifying Redundant Mutation Operators in Android Mutation Testing • Input Validation for Android Intents • JavaScript Testing Framework Comparison |
| Week 15, 4 May | Project presentations • Eric Mingo • Rasika Mohod • Nipun Monga • Joffrey Pannee | • Evaluation of E2E Automation Testing • Usability Testing: Its Impact and Methodologies • In-Depth Load Testing Tool Comparison • Transitioning From Manual To Automated Tests In Web Applications |
| Week 16, 11 May | Final exam period—no class | |
| My Home Page |