Design and Implementation of a ReconOS-based TensorFlow Lite Delegate Architecture

Modern machine learning (ML) techniques continue to move into the embedded system space because traditional centralized compute resources do not suit certain application domains, for example in mobile or real-time environments. Google’s TensorFlow Lite (TFLite) framework supports this shift from cloud to edge computing and makes ML inference accessible on resource-constrained devices. While it offers the possibility to partially delegate computation to hardware accelerators, there is no such “delegate” available to utilize the promising characteristics of reconfigurable hardware. This thesis incorporates modern platform FPGAs into TFLite by implementing a modular delegate framework, which allows accelerators within the programmable logic to take over the execution of neural network layers. To facilitate the necessary hardware/software codesign, the FPGA delegate is based on the operating system for reconfigurable computing (ReconOS), whose partial reconfiguration support enables the instantiation of model-tailored accelerator architectures. In the hardware back-end, a streaming-based prototype accelerator for the MobileNet model family showcases the working order of the platform, but falls short of the desired performance. Thus, it indicates the need for further exploration of alternative accelerator designs, which the delegate could automatically synthesize to meet a model’s demands.