•   When: Thursday, March 30, 2023 from 01:00 PM to 02:00 PM
•   Speakers: Emma Tosch
•   Location: ENGR 4201
•   Export to iCal

True reproducibility of empirical software research involves significant effort: in addition to being able to run new tools, researchers must also reproduce experimental results in order to validate findings. Unfortunately, descriptions of data collection procedures in scientific papers can be underspecified. Artifact evaluation has arisen to address some of these issues, with current best practices focusing on replicability via VMs or containers, in the hopes of allowing evaluators to repeat experiments under near-identical conditions to those reported in scientific papers. These replications fall short of reproducibility because they may still hold constant conditions that would otherwise vary in practice.

 My research on formal language support for data collection can address these concerns. In this talk, I will focus on the formal specification of hypotheses and experimental procedures. This specification can be used to abstract over empirical evaluations, lifting their lower-level specifications into the domain of causal reasoning. Such specifications open new possibilities for future research: they provide an underlying structure that can aid in experiment search, while also serving as a kind of regression testing for new experiments. Furthermore, researchers could share their experimental designs, preconditions, and assumptions, aiding in both scientific communication and training. I will end on a reflection of how experimentation differs from optimization and testing and connect this work to broader initiatives in designing accountable software systems.

Bio: 

Emma Tosch works in applied programming languages (PL) research, where she treats the process of language formalization — especially the design of domain-specific languages — as a methodological approach to problems not ordinarily considered the domain of PL. Her work has been recognized externally with a Distinguished Paper Award (OOPSLA 2014), an ACM SRC first place prize (PLDI 2014), a SIGPLAN research highlight in (2020), and a CACM Research Highlight (Sept. 2021), as well as a Formal Methods in the Field grant (NSF #2220422). 

 Tosch is currently a Postdoctoral Researcher at Northeastern University working with Dr. Chris Martens on generating narratives to explain privacy policies. Tosch was formerly an Assistant Professor of Computer Science in the College of Engineering and Mathematical Sciences at the University of Vermont. She earned her B.A in English Literature from Wellesley College in 2008 before working at a healthcare IT start up. She defended her PhD at the University of Massachusetts Amherst in 2020.