Syllabus * Schedule * Piazza
                          Course Schedule                          

SWE 737: Advanced Software Testing
Spring 2017, Thursday 4:30-7:10 pm, AB 2026

Instructor: Dr. Jeff Offutt,
Prerequisite: SWE 637 (or permission of instructor; experience is also relevant)

Many large software companies report that 50% or more of software development resources are devoted to testing. In broad terms, the goal of testing software is to find, and fix, software faults in-house before they are revealed to users as failures. Studies have found that user-revealed faults cost up to 100 times more to debug and repair as faults found during development. As software grows in complexity and cost, is used in more applications, and is based on more technologies, this cost is fast becoming unsustainable.

Updated schedule as of 13 February.

Week 1, 26 January Introductions and discussion of topics  
Week 2, 2 February Finalizing topics & schedule
Project selection discussions
Web app testing introduction (slides)
Quality Attributes of Web Software Applications, Jeff Offutt, IEEE Software: Special Issue on Software Engineering of Internet Software, March/April 2002
Modeling Presentation Layers of Web Applications for Testing, Jeff Offutt and Ye Wu, Springer's Software and Systems Modeling, 9(2):257-280, April 2010
Week 3, 9 February Testing web applications
Bypass testing (slides)
The atomic section model (slides)
A Case Study on Bypass Testing of Web Applications, Jeff Offutt, Vasileios Papadimitriou, and Upsorn Praphamontripong, Springer's Empirical Software Engineering journal, 19(1):69-104, February 2014
Modeling Presentation Layers of Web Applications for Testing, Jeff Offutt and Ye Wu, Springer's Software and Systems Modeling, 9(2):257-280, April 2010
Prioritizing User-session-based Test Cases for Web Applications Testing, Sreedevi Sampath, Renée Bryce, Gokulanand Viswanath, Vani Kandimalla, and Güneş Koru, First International Conference on Software Testing, Verification, and Validation, April 2008
Applying Mutation Testing to Web Applications, Upsorn Praphamontripong and Jeff Offutt, Sixth Workshop on Mutation Analysis, April 2010, Paris, France
Week 4, 16 February Testing web applications
User-session-based testing (slides)
Mutation testing of web apps (slides)
Week 5, 23 February Testing object-oriented software
Speaker: Anastasia Harsch
Overview of OO language features (slides)
Criteria for testing OO features (slides)
Testing Object Oriented Software: A Survey, Robert V. Binder, Wiley's Software Testing, Verification and Reliability, 6(3-4):125-252, December 1996
Testing Object-oriented Software Systems, Harry M. Sneed, 24th European Conference on Object-Oriented Programming (ECOOP), Maribor, Slovenia, June 2010 (Anastasia)
Testing Coupling Relationships in Object-Oriented Programs, Roger T. Alexander, Jeff Offutt, and Andreas Stefik Wiley's Journal of Software Testing, Verification, and Reliability, 20(4):291-327, December 2010
Week 6, 2 March Testing mobile applications
Speaker: Lin Deng
Dynodroid: an input generation system for Android apps, Aravind Machiry, Rohan Tahiliani, and Mayur Naik, 9th ACM Joint Meeting on Foundations of Software Engineering (ESEC/FSE), Saint Petersburg Russia, 2013, pages 224-234
Automated Test Input Generation for Android: Are We There Yet?, Shauvik Roy Choudhary, Alessandra Gorla, and Alessandro Orso, 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), Bergamo, Italy, 2015, pages 429-440.
Sapienz: multi-objective automated testing for Android applications, Ke Mao, Mark Harman, and Yue Jia, 25th International Symposium on Software Testing and Analysis (ISSTA), Saarbrücken Germany, 2016, pages 94-105
Mutation Operators for Testing Android Apps, Lin Deng, Jeff Offutt, Paul Ammann, and Nariman Mirzaei, Elsevier's Information and Software Technology, 81:154-168, January 2017
Week 7, 9 March Test automation
Speaker: Eric Mingo
Test oracle strategies (slides)
Software Test Automation Practices in Agile Development Environment: An Industry Experience Report, Eliane Figueiredo Collins and Vicente Ferreira de Lucena, Jr., 7th International Workshop on Automation of Software Test, Pages 57-63, Zurich, Switzerland, June 2012 (Eric)
Test Automation in the Real World—2016, (results from a survey monkey poll)
New survey reveals test automation trends, Cecilia Rehn, Software Testing News August 2015 (summary of a longer article behind a pay wall)
Test Oracle Strategies for Model-based Testing, Nan Li and Jeff Offutt, IEEE Transactions on Software Engineering, To appear, 2017
16 March Spring break—no class
Week 9, 23 March Testing for security
Speaker: Ryan Johnson
Overview (slides)
Java applications (slides)
Web applications (slides)
DoS in Android (slides)
Software Security Testing, Bruce Potter and Gary McGraw, IEEE Security & Privacy, 2(5):81-85, September-October 2004
Finding Security Vulnerabilities in Java Applications with Static Analysis, V. Benjamin Livshits and Monica S. Lam, 14th conference on USENIX Security Symposium—Volume 14, Berkeley CA, USA, 18-18, 2005
Method Using Command Abstraction Library for Iterative Testing Security of Web Applications, Seiji Munetoh and Nobukazu Yoshioka, International Journal of Secure Software Engineering (IJSSE), 2015, Vol.6(3), pp.26-49
Why Software DoS is Hard to Fix: Denying Access in Embedded Android Platforms, Ryan Johnson, Mohamed Elsabagh, and Angelos Stavrou, International Conference on Applied Cryptography and Network Security (ACNS), Guildford, UK, June 2016, pp 193-211
Week 10, 30 March Testing the internet of things
Speaker: Nipun Monga

Getting papers rejected (slides)
Details for writing good (slides)
Paper writing and formatting standards (PDF)
Functional Testing for IoT, Chris Riley, blog post, February 2015. (Nipun)
Security Testing the Internet of Things IoT, author and date not given, Beyond Scurity. (Note this is essentially an advertisement for a company's product, nevertheless, it contains useful ideas.) (Nipun)
Testing the Internet of Things: The Human Experience, Gerie Owen, November 2014, InfoQ (online)
Breaking Down Mirai: An IoT DDoS Botnet Analysis, Ben Herzberg, Dima Bekerman, and Igal Zeifman, Imperva Incapsula blog (online)
Week 11, 6 April Load testing
Speaker: Rasika Mohod

Giving good presentations (slides)
A Survey on Load Testing of Large-Scale Software Systems, Zhen Ming Jiang and Ahmed E. Hassan, IEEE Transactions on Software Engineering, 41(11):091-1118, November 2015 (Rasika)
Load Testing of Web Sites, Daniel A. Menasce, IEEE Internet Computing, 6(4):70-74, July-August 2002
Automatic Identification of Load Testing Problems, Zhen Ming Jiang, Ahmed E. Hassan, Gilbert Hamann, and Parminder Flora, 2008 IEEE International Conference on Software Maintenance, Beijing, 2008, pp. 307-316
Week 12, 13 April Usability testing
Speaker: Joffrey Pannee
Design and Implementation of a toolkit for Usability Testing of Mobile Apps, Xiaoxiao Ma, Bo Yan, Guanling Chen, Chunhui Zhang, Ke Huang, Jill Drury, and Linzhang Wang. Mobile Networks and Applications, 18(1):81-97, February 2013 (Joffrey)
Web Evaluation: Heuristic Evaluation vs. User Testing, Wei-Siong Tan, Dahai Liu, and Ram Bishu, International Journal of Industrial Ergonomics, 2009, 39(4), pp.621-627
Week 13, 20 April Prof out of town, work on projects N/A
Week 14, 27 April Project presentations
 • Lin Deng
 • Ryan Johnson
 • Anastasia Mingo

 • An Empirical Study of Identifying Redundant Mutation Operators in Android Mutation Testing
 • Input Validation for Android Intents
 • JavaScript Testing Framework Comparison
Week 15, 4 May Project presentations
 • Eric Mingo
 • Rasika Mohod
 • Nipun Monga
 • Joffrey Pannee

 • Evaluation of E2E Automation Testing
 • Usability Testing: Its Impact and Methodologies
 • In-Depth Load Testing Tool Comparison
 • Transitioning From Manual To Automated Tests In Web Applications
Week 16, 11 May Final exam period—no class
My Home Page