---
_id: '66209'
abstract:
- lang: eng
  text: <jats:p>For timing-sensitive edge applications, the demand for efficient lightweight
    machine learning solutions has increased recently. Tree ensembles are among the
    state-of-the-art in many machine learning applications. While single decision
    trees are comparably small, an ensemble of trees can have a significant memory
    footprint leading to cache locality issues, which are crucial to performance in
    terms of execution time. In this work, we analyze memory-locality issues of the
    two most common realizations of decision trees, i.e., native and if-else trees.
    We highlight that both realizations demand a more careful memory layout to improve
    caching behavior and maximize performance. We adopt a probabilistic model of decision
    tree inference to find the best memory layout for each tree at the application
    layer. Further, we present an efficient heuristic to take architecture-dependent
    information into account thereby optimizing the given ensemble for a target computer
    architecture. Our code-generation framework, which is freely available on an open-source
    repository, produces optimized code sessions while preserving the structure and
    accuracy of the trees. With several real-world data sets, we evaluate the elapsed
    time of various tree realizations on server hardware as well as embedded systems
    for Intel and ARM processors. Our optimized memory layout achieves a reduction
    in execution time up to 75 % execution for server-class systems, and up to 70
    % for embedded systems, respectively.</jats:p>
author:
- first_name: Kuan-Hsun
  full_name: Chen, Kuan-Hsun
  last_name: Chen
- first_name: Chiahui
  full_name: Su, Chiahui
  last_name: Su
- first_name: Christian
  full_name: Hakert, Christian
  last_name: Hakert
- first_name: Sebastian
  full_name: Buschjäger, Sebastian
  last_name: Buschjäger
- first_name: Chao-Lin
  full_name: Lee, Chao-Lin
  last_name: Lee
- first_name: Jenq-Kuen
  full_name: Lee, Jenq-Kuen
  last_name: Lee
- first_name: Katharina
  full_name: Morik, Katharina
  last_name: Morik
- first_name: Jian-Jia
  full_name: Chen, Jian-Jia
  last_name: Chen
citation:
  ama: Chen K-H, Su C, Hakert C, et al. Efficient Realization of Decision Trees for
    Real-Time Inference. <i>ACM Transactions on Embedded Computing Systems</i>. 2022;21(6):1-26.
    doi:<a href="https://doi.org/10.1145/3508019">10.1145/3508019</a>
  apa: Chen, K.-H., Su, C., Hakert, C., Buschjäger, S., Lee, C.-L., Lee, J.-K., Morik,
    K., &#38; Chen, J.-J. (2022). Efficient Realization of Decision Trees for Real-Time
    Inference. <i>ACM Transactions on Embedded Computing Systems</i>, <i>21</i>(6),
    1–26. <a href="https://doi.org/10.1145/3508019">https://doi.org/10.1145/3508019</a>
  bibtex: '@article{Chen_Su_Hakert_Buschjäger_Lee_Lee_Morik_Chen_2022, title={Efficient
    Realization of Decision Trees for Real-Time Inference}, volume={21}, DOI={<a href="https://doi.org/10.1145/3508019">10.1145/3508019</a>},
    number={6}, journal={ACM Transactions on Embedded Computing Systems}, publisher={Association
    for Computing Machinery (ACM)}, author={Chen, Kuan-Hsun and Su, Chiahui and Hakert,
    Christian and Buschjäger, Sebastian and Lee, Chao-Lin and Lee, Jenq-Kuen and Morik,
    Katharina and Chen, Jian-Jia}, year={2022}, pages={1–26} }'
  chicago: 'Chen, Kuan-Hsun, Chiahui Su, Christian Hakert, Sebastian Buschjäger, Chao-Lin
    Lee, Jenq-Kuen Lee, Katharina Morik, and Jian-Jia Chen. “Efficient Realization
    of Decision Trees for Real-Time Inference.” <i>ACM Transactions on Embedded Computing
    Systems</i> 21, no. 6 (2022): 1–26. <a href="https://doi.org/10.1145/3508019">https://doi.org/10.1145/3508019</a>.'
  ieee: 'K.-H. Chen <i>et al.</i>, “Efficient Realization of Decision Trees for Real-Time
    Inference,” <i>ACM Transactions on Embedded Computing Systems</i>, vol. 21, no.
    6, pp. 1–26, 2022, doi: <a href="https://doi.org/10.1145/3508019">10.1145/3508019</a>.'
  mla: Chen, Kuan-Hsun, et al. “Efficient Realization of Decision Trees for Real-Time
    Inference.” <i>ACM Transactions on Embedded Computing Systems</i>, vol. 21, no.
    6, Association for Computing Machinery (ACM), 2022, pp. 1–26, doi:<a href="https://doi.org/10.1145/3508019">10.1145/3508019</a>.
  short: K.-H. Chen, C. Su, C. Hakert, S. Buschjäger, C.-L. Lee, J.-K. Lee, K. Morik,
    J.-J. Chen, ACM Transactions on Embedded Computing Systems 21 (2022) 1–26.
date_created: 2026-07-03T21:18:32Z
date_updated: 2026-07-05T14:45:55Z
doi: 10.1145/3508019
intvolume: '        21'
issue: '6'
language:
- iso: eng
page: 1-26
publication: ACM Transactions on Embedded Computing Systems
publication_identifier:
  issn:
  - 1539-9087
  - 1558-3465
publication_status: published
publisher: Association for Computing Machinery (ACM)
status: public
title: Efficient Realization of Decision Trees for Real-Time Inference
type: journal_article
user_id: '128464'
volume: 21
year: '2022'
...