Differentiable programming for flexible dynamic modeling with small data

Harald Binder

Institute of Medical Biometry and Statistics (IMBI), University of Freiburg

Abstract

Knowledge-driven modeling and data-driven machine learning techniques have been characterized as two rather different approaches to data analysis. While data-hungry techniques, such as deep learning, have seen huge successes with large data sets, in particular with image data, small data settings, characterized by a limited number of observations, are considered to be the primary domain of modeling with strong structural assumptions, such as in regression modeling or with differential equation systems. Still, a combination of both strategies not only is feasible but promises considerable benefits. I will exemplarily illustrate how such a combination of knowledge-driven and data-driven modeling, enabled by differential programming techniques, can be useful for the analysis of longitudinal rare disease data, i.e., also in a small data setting. In particular, combining deep learning for dimension reduction and differential equations for dynamic modeling enables comprehensive modeling even in a situation with a multitude of longitudinal characteristics and a large number of potentially relevant baseline characteristics. The latter allow to identify individual-specific dynamic patterns, which could then also inform treatment decisions. Based on this example, I will also more generally discuss how such combinations of approaches enrich the data science toolbox for tackling challenging novel questions, in particular in small data settings.