Software Testing in the Machine Learning Era

Call for Papers - a Special Issue of the Empirical Software Engineering (EMSE) journal.

Editors of the Special Issue

Onn Shehory, Bar Ilan University, Israel, onn.shehory@biu.ac.il
Andrea Stocco, Software Institute, USI - Lugano, Switzerland, andrea.stocco@usi.ch
Gunel Jahangirova, Software Institute, USI - Lugano, Switzerland, gunel.jahangirova@usi.ch
Vincenzo Riccio, Software Institute, USI - Lugano, Switzerland, vincenzo.riccio@usi.ch
Guy Barash, Western Digital, Israel, guy.barash@wdc.com
Eitan Farchi, IBM Haifa Research Lab, Israel, farchi@il.ibm.com
Diptikalyan Saha, IBM Research, India, diptsaha@in.ibm.com

Description of the Special Issue

Machine Learning (ML) and Deep Learning (DL) are widely adopted in modern software systems, including safety-critical domains such as autonomous cars, medical diagnosis, and aircraft collision avoidance systems. Thus, it is crucial to rigorously test such learning-based applications to ensure high reliability and dependability. However, standard notions of software quality and reliability become irrelevant when considering learning-based systems, due to their non-deterministic nature and the lack of a transparent understanding of the models’ semantics. The impact of ML and DL goes beyond simply offering new applications and case studies for testing techniques. In fact, they are revolutionizing how the software is developed and tested. Indeed, ML and DL are increasingly being applied for devising novel program analysis and software testing techniques related to malware detection, fuzzy testing, bug-finding, and type-checking.

This EMSE special issue seeks research contributions targeting the intersection of SE and ML/DL. The purpose is twofold:

• Novel techniques to ensure the quality of learning-based applications
• Novel techniques that use ML or DL to support software engineering tasks

The special issue invites submissions reporting new and innovative research results and industrial experience papers in the area of learning-based applications (e.g., machine learning, deep learning, transfer learning, and meta-learning) for software quality, and quality assurance techniques for learning-based applications.

Topics of Interest

Relevant topics include, but are not limited to:

Quality

• The quality implication of learning-based algorithms on large-scale software systems
• Application of classical statistics to learning-based systems’ quality
• Training and payload data quality
• The correctness of data abstraction, data trust
• High-quality benchmarks for evaluating learning-based approaches

Testing and Verification

• Test data synthesis for testing learning-based systems
• White-box and black-box testing strategies
• ML models for testing programs
• Adversarial machine learning and adversary based learning
• Test coverage
• Vulnerability, sensitivity, and attacks against learning-based
• Metamorphic testing as software quality assurance
• New abstraction techniques for verification of Mlearning-based systems
• ML techniques for software verification
• Dev-ops for learning-based software (MLOps, AIOps)

Fault Localization, Debugging and Repairing

• Quality Metrics for learning-based systems, e.g., Correctness, Accuracy, Fairness, Robustness, Explainability
• Sensitivity to data distribution diversity and distribution drift
• Failure explanation and automated debugging techniques
• Runtime monitoring
• Fault Localization and anomaly detection
• Model repairing
• The effect of labeling costs on solution quality (semi-supervised learning)
• ML for fault prediction, localization, and repair
• ML to aid program comprehension, program transformation, and program generation

Schedule

Submission Deadline: January 31st, 2022

Submission Instructions

Papers should be submitted through the Empirical Software Engineering editorial manager website. At the site: (1) In “Article Type Selection”, select “Research Papers”. (2) In “Additional Information”: (3) answer “Yes” to “Does this paper belong to a special issue?”; (4) select “Software Testing in the Machine Learning Era” for “Please select the issue your manuscript belongs to”.

For formatting guidelines as well as submission instructions, please visit Empirical Software Engineering

Submission guidelines.