Deep Learning for Tabular and Categorical Data

So far, tree-based models like XGBoost are still the state-of-the-art for tabular data, despite the success of deep learning in many areas. Our goal is to develop competitive deep learning methods for tabular data, as common in many real-world data science tasks. One problem is the fact that tabular data typically contains high-cardinality categorical features, which are challenging for deep learning methods. Therefore, a focus of this project is to develop scalable methods that can handle such features. One option are so-called mixed-effects neural networks.

In addition, we are working on gradient descent methods for learning tree-based models, combining the strengths of deep learning models and tree-based models.

Key Publications

Sascha Marton, Stefan Lüdtke, Christian Bartelt, Heiner Stuckenschmidt. GRANDE: Gradient-Based Decision Tree Ensembles. International Conference on Learning Representations (ICLR) 2024.

Sascha Marton, Stefan Lüdtke, Christian Bartelt, Heiner Stuckenschmidt. GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent. Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI) 2024. [web]