Attacks against Machine Learning in Structured Domains

• Applicant: Professor Dr. Konrad Rieck
• Subject Area: Security and Dependability, Operating-, Communication- and Distributed Systems
• Term: since 2022
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 492020528

Machine learning techniques are increasingly used in security-critical applications, such as for the detection of malicious code and attacks. However, current learning algorithms are often vulnerable themselves and can be deceived by manipulated inputs. In recent years, a large number of new attack techniques against machine learning has thus been developed. With few exceptions this research has focused on a simplified scenario: The attacks are conducted in the feature space of the learning algorithms only. By making small changes to vectors in this space, it becomes possible to to influence the algorithms' decisions and provoke incorrect predictions. In practice, however, these attacks are only applicable if the manipulated vectors can be mapped back to real objects. For structured data, such as program code, file formats and natural language, this inverse mapping from vectors to structures is almost never defined. Thus, the robustness of many security-critical applications cannot be investigated and tested with the majority of existing attacks.The goal of this project is to explore the security of learning algorithms in structured domains and close a gap of current research. In contrast to previous work, a systematic understanding of the relationship between the problem space of the original data and the feature space will be developed. Two strategies will be pursued for this purpose: First, new inverse mappings for structured data will be explored and developed that replicate missing semantics and syntax in the problem space. Second, new attacks will be devised that operate directly on structured data and thus are not affected by feature mappings. Based on both strategies, new defenses can emerge that build on the interleaving of the problem space and feature space to realize more robust learning systems for computer security.

• DFG Programme: Research Grants