{"citation":{"chicago":"Maslovskaya, Sofya, Sina Ober-Blöbaum, Christian Offen, Pranav Singh, and Boris Edgar Wembe Moafo. “Adaptive Higher Order Reversible Integrators for Memory Efficient Deep Learning,” 2025.","apa":"Maslovskaya, S., Ober-Blöbaum, S., Offen, C., Singh, P., &#38; Wembe Moafo, B. E. (2025). <i>Adaptive higher order reversible integrators for memory efficient deep learning</i>.","short":"S. Maslovskaya, S. Ober-Blöbaum, C. Offen, P. Singh, B.E. Wembe Moafo, (2025).","ama":"Maslovskaya S, Ober-Blöbaum S, Offen C, Singh P, Wembe Moafo BE. Adaptive higher order reversible integrators for memory efficient deep learning. Published online 2025.","bibtex":"@article{Maslovskaya_Ober-Blöbaum_Offen_Singh_Wembe Moafo_2025, title={Adaptive higher order reversible integrators for memory efficient deep learning}, author={Maslovskaya, Sofya and Ober-Blöbaum, Sina and Offen, Christian and Singh, Pranav and Wembe Moafo, Boris Edgar}, year={2025} }","mla":"Maslovskaya, Sofya, et al. <i>Adaptive Higher Order Reversible Integrators for Memory Efficient Deep Learning</i>. 2025.","ieee":"S. Maslovskaya, S. Ober-Blöbaum, C. Offen, P. Singh, and B. E. Wembe Moafo, “Adaptive higher order reversible integrators for memory efficient deep learning.” 2025."},"title":"Adaptive higher order reversible integrators for memory efficient deep learning","_id":"59794","year":"2025","external_id":{"arxiv":["2410.09537"]},"file_date_updated":"2025-05-05T09:28:02Z","ddc":["510"],"user_id":"85279","department":[{"_id":"636"}],"status":"public","author":[{"first_name":"Sofya","id":"87909","full_name":"Maslovskaya, Sofya","last_name":"Maslovskaya"},{"id":"16494","first_name":"Sina","full_name":"Ober-Blöbaum, Sina","last_name":"Ober-Blöbaum"},{"first_name":"Christian","id":"85279","last_name":"Offen","orcid":"0000-0002-5940-8057","full_name":"Offen, Christian"},{"last_name":"Singh","full_name":"Singh, Pranav","first_name":"Pranav"},{"id":"95394","first_name":"Boris Edgar","full_name":"Wembe Moafo, Boris Edgar","last_name":"Wembe Moafo"}],"language":[{"iso":"eng"}],"abstract":[{"lang":"eng","text":"The depth of networks plays a crucial role in the effectiveness of deep learning. However, the memory requirement for backpropagation scales linearly with the number of layers, which leads to memory bottlenecks during training. Moreover, deep networks are often unable to handle time-series data appearing at irregular intervals. These issues can be resolved by considering continuous-depth networks based on the neural ODE framework in combination with reversible integration methods that allow for variable time-steps. Reversibility of the method ensures that the memory requirement for training is independent of network depth, while variable time-steps are required for assimilating time-series data on irregular intervals. However, at present, there are no known higher-order reversible methods with this property. High-order methods are especially important when a high level of accuracy in learning is required or when small time-steps are necessary due to large errors in time integration of neural ODEs, for instance in context of complex dynamical systems such as Kepler systems and molecular dynamics. The requirement of small time-steps when using a low-order method can significantly increase the computational cost of training as well as inference. In this work, we present an approach for constructing high-order reversible methods that allow adaptive time-stepping. Our numerical tests show the advantages in computational speed when applied to the task of learning dynamical systems."}],"file":[{"access_level":"closed","creator":"sofyam","file_size":1830758,"file_name":"2410.09537v2.pdf","success":1,"date_updated":"2025-05-05T09:28:02Z","file_id":"59795","date_created":"2025-05-05T09:28:02Z","relation":"main_file","content_type":"application/pdf"}],"date_created":"2025-05-05T09:25:28Z","date_updated":"2025-09-30T15:16:09Z","has_accepted_license":"1","type":"preprint"}