Cortical wiring motifs as Inductive Biases for improved Few-shot Learning (MSc Semester/Thesis project)

Figure: a) Candidate neural architectures and their b) Connectivity matrix

At the EIS lab, we are neuromorphic engineers who are curious to explore how motifs from the brain’s morphology and architecture can serve as useful inductive biases to (i) improve energy and area of the neural network hardware substrates, and (ii) improve generalization and task performance.

In this project, we are focusing on the specific connectivity in the cortex which is highly local and sparsely global called a small-world connectivity. We have shown how such block diagonal connectivity with sparse global connections significantly reduces the communication energy on neuromorphic hardware [1].

Currently, we are curious to understand whether this connectivity structure is also a good algorithmic prior to improve task performance and generalization. 

In this specific project, we want to explore whether this connectivity improves few-shot learning. In a previous project, we have used Model-Agnostic-Meta-Learning (MAML), to pre-train a block-diagonal neural network in a few-shot learning setting, where the online few-shot learning only updates the linear read-out [2]. 

We have some evidence that points towards the fact that this network connectivity requires a lower number of shots to learn the same task, compared to a fully connected recurrent network. 

Your task would be to study this further based on an already available code, on different tasks. 

You are required to have good programming skills, and should be familiar with PyTorch or JAX, have a background and understanding of modern Machine Learning models, and have a passion for brain-inspired computing.

Please contact melika@ini.uzh.ch and karthik@ini.uzh.ch if you are interested. 

[1] https://www.nature.com/articles/s41467-023-44365-x

[2] https://ieeexplore.ieee.org/abstract/document/10548824