---
_id: '59794'
abstract:
- lang: eng
  text: The depth of networks plays a crucial role in the effectiveness of deep learning.
    However, the memory requirement for backpropagation scales linearly with the number
    of layers, which leads to memory bottlenecks during training. Moreover, deep networks
    are often unable to handle time-series data appearing at irregular intervals.
    These issues can be resolved by considering continuous-depth networks based on
    the neural ODE framework in combination with reversible integration methods that
    allow for variable time-steps. Reversibility of the method ensures that the memory
    requirement for training is independent of network depth, while variable time-steps
    are required for assimilating time-series data on irregular intervals. However,
    at present, there are no known higher-order reversible methods with this property.
    High-order methods are especially important when a high level of accuracy in learning
    is required or when small time-steps are necessary due to large errors in time
    integration of neural ODEs, for instance in context of complex dynamical systems
    such as Kepler systems and molecular dynamics. The requirement of small time-steps
    when using a low-order method can significantly increase the computational cost
    of training as well as inference. In this work, we present an approach for constructing
    high-order reversible methods that allow adaptive time-stepping. Our numerical
    tests show the advantages in computational speed when applied to the task of learning
    dynamical systems.
author:
- first_name: Sofya
  full_name: Maslovskaya, Sofya
  id: '87909'
  last_name: Maslovskaya
- first_name: Sina
  full_name: Ober-Blöbaum, Sina
  id: '16494'
  last_name: Ober-Blöbaum
- first_name: Christian
  full_name: Offen, Christian
  id: '85279'
  last_name: Offen
  orcid: 0000-0002-5940-8057
- first_name: Pranav
  full_name: Singh, Pranav
  last_name: Singh
- first_name: Boris Edgar
  full_name: Wembe Moafo, Boris Edgar
  id: '95394'
  last_name: Wembe Moafo
citation:
  ama: Maslovskaya S, Ober-Blöbaum S, Offen C, Singh P, Wembe Moafo BE. Adaptive higher
    order reversible integrators for memory efficient deep learning. Published online
    2025.
  apa: Maslovskaya, S., Ober-Blöbaum, S., Offen, C., Singh, P., & Wembe Moafo,
    B. E. (2025). <i>Adaptive higher order reversible integrators for memory efficient
    deep learning</i>.
  bibtex: '@article{Maslovskaya_Ober-Blöbaum_Offen_Singh_Wembe Moafo_2025, title={Adaptive
    higher order reversible integrators for memory efficient deep learning}, author={Maslovskaya,
    Sofya and Ober-Blöbaum, Sina and Offen, Christian and Singh, Pranav and Wembe
    Moafo, Boris Edgar}, year={2025} }'
  chicago: Maslovskaya, Sofya, Sina Ober-Blöbaum, Christian Offen, Pranav Singh, and
    Boris Edgar Wembe Moafo. “Adaptive Higher Order Reversible Integrators for Memory
    Efficient Deep Learning,” 2025.
  ieee: S. Maslovskaya, S. Ober-Blöbaum, C. Offen, P. Singh, and B. E. Wembe Moafo,
    “Adaptive higher order reversible integrators for memory efficient deep learning.”
    2025.
  mla: Maslovskaya, Sofya, et al. <i>Adaptive Higher Order Reversible Integrators
    for Memory Efficient Deep Learning</i>. 2025.
  short: S. Maslovskaya, S. Ober-Blöbaum, C. Offen, P. Singh, B.E. Wembe Moafo, (2025).
date_created: 2025-05-05T09:25:28Z
date_updated: 2025-09-30T15:16:09Z
ddc:
- '510'
department:
- _id: '636'
external_id:
  arxiv:
  - '2410.09537'
file:
- access_level: closed
  content_type: application/pdf
  creator: sofyam
  date_created: 2025-05-05T09:28:02Z
  date_updated: 2025-05-05T09:28:02Z
  file_id: '59795'
  file_name: 2410.09537v2.pdf
  file_size: 1830758
  relation: main_file
  success: 1
file_date_updated: 2025-05-05T09:28:02Z
has_accepted_license: '1'
language:
- iso: eng
status: public
title: Adaptive higher order reversible integrators for memory efficient deep learning
type: preprint
user_id: '85279'
year: '2025'
...