Research

Research Focus: Mathematical Methods in Computer Science:

While machine learning methods continue to advance, the mathematical understanding of these methods usually lags behind (often quite severely). We believe that such mathematical understanding is important for the precise calibration and employment of these methods, which ultimately determines whether they can be used in an efficient, reliable and safe way. Our group aims to contribute to these goals, often from the direction of (computational) probability theory.

Computational Understanding of Probabilistic Structures:

Probabilistic structures like data sets or probability measures are crucial parts of machine learning methods. To account for the inherent variability (like changing dynamics from training and deployment) of probabilistic structures, one often models these structures as elements in a space. The characteristics of this space can be specified through statistical distances (e.g., optimal transport). We work on (mathematical understanding of) algorithmic procedures on this space, as well as flexible new tools to both analyze and computationally deal with probabilistic structures in practice. Here are some example papers in this direction we contributed to:

• Estimating the Rate-Distortion Function by Wasserstein Gradient Descent. NeurIPS, 2023.
• Hilbert's projective metric for functions of bounded growth and exponential convergence of Sinkhorn's algorithm. Preprint, 2024.

Causal reasoning and graphical causal models:

To reliably apply machine learning methods in real-world scenarios, it is crucial that these methods rely on a causal understanding of the world rather than merely statistical associations, which can easily fail in new environments.

A common approach to incorporate causality into probabilistic settings is through graphical causal models. We explore how general probabilistic tools can be made suitable for graphical causal models. Among others, we are investigating how to derive statistical guarantees for causal questions (e.g., through causal optimal transport), how to obtain optimal statistical estimators under graphical causal knowledge, and under which assumptions interventional or even counterfactual assertions are justified in probabilistic settings, which also contributes to the interpretability of such assertions. Some example papers in this direction are:

• What Do Counterfactuals Say About the World? Reconstructing Probabilistic Logic Programs from Answers to “What If?” Queries. ILP, 2024.
• Optimal transport and Wasserstein distances for causal models. Bernoulli, 2024.

Internal Mechanisms and Reasoning of Neural Networks:

When data is processed by neural networks, all neurons fire with a certain magnitude, leading to so-called activation vectors (which are again a probabilistic structure). It is thus relatively easy to get a fine-grained view of the reasoning processes of a neural network. However, for instance due to the large number of neurons, it is still extremely difficult to translate this fine-grained view in a way which allows for real human understanding. We explore whether neural networks exhibit recurring patterns in their reasoning. To this end, we investigate whether different neural networks exhibit similar reasoning and if so, how to measure this similarity. In this regard, we explore the use of statistical distances which can be suitably adjusted to different neural network architectures.

Funding: We gratefully acknowledge the support of the DFG as the Emmy Noether Junior Research Group "Algorithms and Structure on the Space of Probability Measures" and our association with and support from the excellence cluster "Machine Learning: New Perspectives in Science”.