Design of input layer to a neuromorphic recurrent neural network

The need for low-power machine learning computing solutions has triggered significant interest in energy-efficient neuromorphic systems. In our previous work [1], we proposed a recurrent neural network model that achieved remarkable performance in a number of benchmarks. The recurrent neural network model used in [1] is designed to interface to a variety of sensor types such as an audio chip, an accelerometer or a biomedical sensor. In a number of use-cases, for example, audio or accelerometer recording input patterns, the raw data is first passed through a spectrogram before passing through the neural network layers. In this project, the goal is to design an input stage that can adapt to a variety of such use-case scenarios. This project may cover all or a subset of the following:

1. Exploration of alternative feature extraction techniques (for example: wavelets [2]) that are compatible with the lpRNN module and compare the performance of the network in various scenarios.

2. Design of an adaptable feature extractor that is suitable for a number of different sensor use-cases.

Project goals

In this project, we will work on implementing an end-to-end spoken phrase detection task using the lpRNN model. The version described in [1] does not take into account the computational cost and energy cost of the input stage used when training the lpRNN model. The objective of this project is to include these constraints on the model and to study the performance of the resulting system.

Primary goal

1. Study the effect of various input stage feature extraction techniques in the performance and computational cost of the system

   1. On the Google Commands dataset [3]

   2. On a Human activity recognition from accelerometers and gyroscopes such as [4]

Secondary goals

1. Develop an FPGA implementation of the designed system.

2. Explore the possibility of training the parameters of the feature extractor stage as part of the neural network optimization procedure.

Necessary skill-set

1. Python with experience in deep learning libraries such as PyTorch/Tensorflow.

2. Desirable: Experience in FPGA programming and/or analog/digital chip design.

Bibliography

1. Nair, Manu V., and Giacomo Indiveri. "Mapping high-performance RNNs to in-memory neuromorphic chips." arXiv preprint arXiv:1905.10692 (2019).

2. Kaiser, Gerald. A friendly guide to wavelets. Springer Science & Business Media, 2010.

3. Warden, Pete. "Speech commands: A dataset for limited-vocabulary speech recognition." arXiv preprint arXiv:1804.03209 (2018).

4. Casale, P. Pujol, O. and Radeva, P., 'Personalization and user verification in wearable systems using biometric walking patterns', Personal and Ubiquitous Computing, 16(5), 563-580, 2012