Innovations in Software System Testing with Deep Learning

Call for Papers - a Special Issue of the Empirical Software Engineering (EMSE) journal.

Editors of the Special Issue

Andrea Stocco, Technical University of Munich & fortiss - Germany andrea.stocco@tum.de | stocco@fortiss.org

Matteo Biagiola, Software Institute - USI, Switzerland, matteo.biagiola@usi.ch

Vincenzo Riccio, University of Udine, Italy vincenzo.riccio@uniud.it

Foutse Khomh, Polytechnique Montréal - Canada foutse.khomh@polymtl.ca

Nicolás Cardozo, Universidad de los Andes, Colombia n.cardozo@uniandes.edu.co

Donghwan Shin, University of Sheffield, United Kingdom d.shin@sheffield.ac.uk

Description of the Special Issue

Machine Learning (ML) and Deep Learning (DL) are widely adopted in modern software systems, including safety-critical domains such as autonomous cars, medical diagnosis, and aircraft collision avoidance systems. Thus, it is crucial to rigorously test such learning-based applications to ensure high reliability and dependability. However, standard notions of software quality and reliability become irrelevant when considering learning-based systems, due to their non-deterministic nature and the lack of a transparent understanding of the models’ semantics. The impact of ML and DL goes beyond simply offering new applications and case studies for testing techniques. In fact, they are revolutionizing how the software is developed and tested. Indeed, ML and DL are increasingly being applied for devising novel program analysis and software testing techniques related to malware detection, fuzzy testing, bug-finding, and type-checking.

This EMSE special issue seeks research contributions targeting the intersection of SE and ML/DL. The purpose is twofold:

• Novel techniques to ensure the quality of learning-based applications
• Novel techniques that use ML or DL to support software engineering tasks

The special issue invites submissions reporting new and innovative research results and industrial experience papers in the area of learning-based applications (e.g., machine learning, deep learning, transfer learning, and meta-learning) for software quality, and quality assurance techniques for learning-based applications.

Topics of Interest

Relevant topics include, but are not limited to:

Quality

• The implication of using large language models to aid software engineering tasks
• The quality implication of learning-based algorithms on large-scale software systems
• Application of classical statistics to learning-based systems’ quality
• Training and payload data quality
• The correctness of data abstraction, data trust
• High-quality benchmarks for evaluating learning-based approaches

Testing and Verification

• Testing large language models
• Large language models for Testing
• Test data generation for testing learning-based systems
• White-box and black-box testing strategies
• ML/DL models for testing programs
• Test coverage
• Metamorphic testing
• Vulnerability, sensitivity, and attacks against learning-based systems
• New abstraction techniques for verification of learning-based systems
• ML techniques for software verification
• Dev-ops for learning-based software (MLOps, AIOps)

Fault Localization, Debugging and Repairing

• Quality Metrics for learning-based systems, e.g., Correctness, Accuracy, Fairness, Robustness, Explainability
• Sensitivity to data distribution diversity and distribution drift
• Failure explanation and automated debugging techniques
• Runtime monitoring
• Fault Localization and anomaly detection
• Model repairing
• The effect of labeling costs on solution quality (semi-supervised learning)
• ML/DL for fault prediction, localization, and repair
• ML/DL to aid program comprehension, program transformation, and program generation

Schedule

Submission Deadline: January 31st, 2024 February 28th, 2024

First Notification: March 31st, 2024 April 30, 2024

Submission Instructions

Papers should be submitted through the Empirical Software Engineering editorial manager website. At the site: (1) In “Article Type Selection”, select “Research Papers”. (2) In “Additional Information”: (3) answer “Yes” to “Does this paper belong to a special issue?”; (4) select “Innovations in Software System Testing with Deep Learning” for “Please select the issue your manuscript belongs to”.

For formatting guidelines as well as submission instructions, please visit Empirical Software Engineering

Submission guidelines.