Event-based (Neuromorphic) radar signal encodings

General information
Daily Supervisor IMEC-NL: Manolis Sifalakis
Second supervisor IMEC-NL (optional): Federico Corradi, Amirreza Yousefzadeh
Department (pick one): IoT
Team: NLICDESIGN
Interviewers for this project: M.S., F.C., A.Y., M.K. (R&D Manager)

Assignment
Small introduction project:
At IMEC’s (Holst-centre lab) in Eindhoven, The Netherlands, we are developing a novel neuromorphic radar sensor backend called event-radar that targets always-on low-power sensing, sparse data streaming, and on-sensor processing. In-line with this work we seek for a motivated student to undertake a project, which will focus on exploring and developing temporally and spatially sparse (event based) encodings of radar signals for short-range radar application tasks (gesture recognition, vital sign detection, room activity classification). The objective will be that these signals can be generated and used for inference right at the sensor (low-power budget and real-time application inference).
Duration assignment: 9 to 12 months

Student profile
Level of education: M.Sc.
Required program:
- Electrical/Computer Engineering
- Computer Science
- Neuromorphic engineering

Project description:
[To be considered for this position: The European candidates must be enrolled in a Master program. Non- European master students who are enrolled in a Dutch university are also welcomed to apply] Typically, most sensors today (camera/microphone/radar/etc.) generate a lot of data that need to be communicated for processing/inference by a model. This allows the sensor to do little processing work at the expense of the bandwidth that is needed to communicate the data to the downstream processing pipeline. By contrast neuromorphic sensors (dynamic vision sensor [1], cochlea audio sensor [2], e-skin sensor [3]), exploit bio-inspired sensory processing principles, generates sparser temporal signals and consume significantly less power. A big advantage of this paradigm is that it leaves resources for application-related processing right at the sensor as well. Towards a similar objective in the neuromorphic group of IMEC (Holst-Centre lab, Netherlands), we have been developing an analogous neuromorphic radar-sensor backend, for indoor or short distance sensing applications (human machine interface using gestures commands, human activity classification, vital signs, and other applications for smart spaces).
The goal of this project will be to explore various temporal encodings and sparse distributed representations of the radar signals, their suitability for embedded low-power processing and their efficacy in machine learning related application tasks.
For example, a baseline exploration point can be a differential encoding (delta or sigma-delta modulator), and one may move on to introduce reverberating dynamics with neural networks such as recurrent spiking neural networks that can be “nudged” to resonate according to the radar front-end detections or move to trainable sparse signature representations [4] of the activity taking place in front of the sensor.
The results of this exploration will be compared with more common-place traditional radar DSP pipelines (e.g., FFT based) and evaluated in various application tasks such as those listed above.
Project duration is set to 9 or 12months (e.g., internship and MSc project) and depending on outcomes, there will be opportunity to patent or publish the results in high-visibility conference or journal in the field.
While the work is primarily algorithmic, depending on competence and interest, the student is also expected to work directly with the radar sensor and dynamic vision sensor hardware prototype and novel neuromorphic accelerators (available at IMEC), for collecting relevant dataset and running experiments.
Candidates are expected to be highly motivated, with relevant background in one or more of the following fields:
sensor signal processing, neuromorphic computing/engineering, optimization and learning in neural networks, statistical pattern recognition / probabilistic learning models. The candidate must have good programming skills in Python and reasonable exposure to C/C++ (there will not be opportunity to learn elementary programming during the project). Interested applicants are welcome to submit their CV, and academic transcripts (courses taught, and scores or level attained wherever applicable).
References:
[1] Galego et al. (2020). Event-based vision: a Survey. IEEE transactions on Pattern Analysis and Machine Intelligence.
[2] S.Liu et al. (2014). Asynchronous Binaural Spatial Audition Sensor With 2x64x4 Channel Output. IEEE Transactions on Biomedical Circuits and Systems.
[3] F.Bergner et al. (2020). Design and Realization of a Resistive Efficient Large-Area Event-Driven E-Skin. MDPI Sensors.
[4] W. Brendel et al (2020). Learning representations spike-by-spike. PLOS Computational Biology
[5] https://www.imec-int.com/en/79GHz-and-140GHz-radar-solutions/8-GHz-radar-for-smart-buildings
[6] https://drupal.imec-int.com/sites/default/files/2019-04/8GHz%20UWB%20RADAR_FINAL.pdf

Tasks (specific)
- Literature review on neuromorphic sensing and processing
- Plan exploration for a small set of designed encodings/representations (define criteria of interest and application of interest)
- Implementation of resp. representation algorithms
- Performance testing and evaluation, comparison with contemporary radar DSP pipelines
- Thesis writing and documentation in IMEC Holst-Centre

Required skills
- Very good/excellent programming in python and at least intermediate programming in C/C++
- Good background in one or more of:
- o Sensor digital signal processing (radar DSP preferable)
- o Neuromorphic computing/engineering
- o Optimization for learning in Neural networks
- o Statistical pattern recognition
- A structured way of reporting, both orally and written
- Motivated student eager to work independently and expand knowledge in the field.
- Good written and verbal English skills