Master thesis: Hardware-aware benchmarking of analog gated recurrent neural networks

Project description

At EIS lab, we are interested in developing efficient computing systems that combine machine-learning algorithms with hardware architectures tailored to their computational structure. This hardware-algorithm co-design perspective is especially important for temporal sequence processing, where recurrent neural networks can maintain compact internal states and are therefore attractive for embedded, always-on, and energy-constrained applications.

Recently, we developed MINIMALIST [1], a mixed-signal in-memory computing architecture for efficient computation of minimal gated recurrent neural networks [2]. The approach combines a strongly simplified recurrent model with switched-capacitor circuits that can implement both in-memory computation and gated state updates. Initial results validate the basic software-to-hardware mapping and demonstrate the feasibility of the architecture on a first benchmark task. However, to guide the next stage of hardware development, it is important to understand whether the approach generalizes to a broader range of sequence-processing problems, and where potential algorithmic or hardwarerelated bottlenecks may appear.

In this project, you will expand the benchmarking of MINIMALIST-style recurrent networks. The goal is to further validate the proposed approach and hardware architecture and at the same time to identify limitations that can aid future optimization of the planned hardware. You will evaluate the model on additional temporal tasks, including synthetic benchmarks such as the selective copying task, as well as sensor-processing, audio, and time-series classification datasets. A particular focus will be placed on hardware-aware training constraints, including quantization, quantization-aware training, gating behavior, binary activations between layers, and the choice of activation functions.

 

In this thesis you will:

• - implement and extend a benchmarking framework for hardware-amenable gated recurrent neural
  networks
• - evaluate the model on a broader set of temporal sequence tasks, including synthetic memory benchmarks and real-world time-series datasets
• - study the impact of quantization, quantization-aware training, and reduced numerical precision on
  task performance and robustness
• - compare different activation functions and gating nonlinearities with respect to both accuracy and
  hardware compatibility
• - analyze performance bottlenecks related to model size, hidden-state dynamics, activation sparsity,
  communication activity, and switching activity
• - derive practical recommendations for future optimization of the MINIMALIST hardware architecture

 

What should you bring to the table

• - curiosity about efficient neural network models and their implementation in hardware
• - interest in hardware-algorithm co-design for low-power machine learning systems
  good programming skills in Python
• - experience with PyTorch or similar machine-learning frameworks
  basic understanding of recurrent neural networks, quantization, or embedded machine learning is
  a plus
• - interest in mixed-signal circuits or in-memory computing is helpful, but not required

 

What will you get out of this

• - work on a timely research topic at the intersection of machine learning, neuromorphic computing,
  and mixed-signal hardware design
• - gain practical experience with hardware-aware neural network training and benchmarking
• - develop a hardware-algorithm co-design mindset by interacting with both algorithm and circuit
  designers
• - contribute to the evaluation and optimization of a new recurrent neural network accelerator architecture

 

Further reads

[1] S. Billaudelle, L. Kriener, F. Moro, T. Torchet, and M. Payvand, “MINIMALIST: switchedcapacitor circuits for efficient in-memory computation of gated recurrent units,” arXiv preprint
arXiv:2505.08599, 2025.
[2] L. Feng, F. Tung, M. O. Ahmed, Y. Bengio, and H. Hajimirsadeghi, “Were rnns all we needed?,”
arXiv preprint arXiv:2410.01201, 2024.