•   When: Thursday, February 06, 2020 from 11:00 AM to 12:00 PM
•   Speakers: Kevin Moran
•   Location: Engineering Building 4201
•   Export to iCal

Abstract: Given the ubiquity of software in modern society and its applications in increasingly complex problem domains, today’s developers require practical automation in order to effectively and efficiently build, test, and maintain software systems. At the same time, the proliferation of software has led to the creation of an unprecedented amount of freely available data that describes a diverse array of software systems. Artifacts such as source code files, screenshots, videos, and bug reports provide a wealth of information from which patterns can be learned and leveraged to enable useful automation for developers. In this talk, I will describe two of my recent research projects that use machine learning techniques to harness the data contained within software repositories to automate different components of the development lifecycle for mobile applications. I focus on mobile apps both because their popularity among developers and users means this research has a large potential for impact, and because of the unique development challenges posed by the mobile domain (such as change-prone APIs and platform fragmentation). These projects aim to improve developer productivity while alleviating the effects of these challenges.

First, I will introduce an approach that completely automates the process of prototyping GUIs for mobile apps. This approach, called ReDraw, is able to transform an image of a mobile app GUI into runnable code by detecting discrete GUI-components using computer vision techniques, classifying these components into proper functional categories (e.g., button, dropdown menu), and assembling these components into realistic code. Second, I will present a technique that is capable of translating a screen-recording of a mobile application into a replayable scenario. This technique, called V2S, is based primarily on computer vision techniques and adapts recent solutions for neural object detection and image classification to detect and classify user actions on a screen. Finally, I will conclude my talk by providing a brief overview of related projects, as well as my research vision and planned future work.

Bio: Dr. Kevin Moran is a Research Assistant Professor in the Department of Computer Science at William & Mary.  He graduated with his B.A. in Physics and a Computer Science Minor from the College of the Holy Cross in 2013. He graduated with his M.S. in Computer Science from William & Mary in 2015, and his Ph.D. in Computer Science from William & Mary in 2018, advised by Dr. Denys Poshyvanyk. His main research interests include software engineering, maintenance, evolution, and security with a focus on mobile platforms. Dr. Moran’s current work explores applications of machine learning techniques to data mined from software repositories in order to create practical automation for developers. He has published at several top software engineering and computer security venues including ICSE, FSE, ASE, ICSME, IEEE TSE, USENIX Security, and CODASPY. Dr. Moran’s work has recently earned an ACM SigSoft Distinguished Paper Award at FSE’19 and the Best Paper Award at CODASPY’19.