Track 3 - Embedded AI for Biosignal Processing

Hands on tutorials to work on porting embedded AI applications for biosignal processing to state of the art heterogeneous IoT processors from Greenwaves Technologies (GAP8/GAP9).

You will learn:

• An introduction to shared-memory many-core architectures
• Issues in explicit memory management, tiling, DMA transfers
• Working with 16-8bit data formats to fit data into memory, quantization
• Programming the devices
• Interfacing sensors, collecting data
• Running your programs, debugging

Course will be in English with support from POLITO, ETH Zurich

Nine 2-hour modules with > 50% hands on exercises in computer laboratory.

Limited to 25 participants

Prerequisites: General knowledge on ML algorithms, basic programming in C/C++ or similar

Agenda:

We will have 18 hours of lectures and hands on exercises focused on the deployment of AI models for biosignal processing on the GAP9 platform. Below, you can find the detailed agenda.

• Session 1: Motivation + SW-Optimization 1 (Generic DNN Optimization concepts, Efficient Layers)
  • Duration: 2 hours
  • Type: lecture
  • Room: ETZ E9
• Session 2: Biosignal processing applications, Introduction to GAP9 (PULP architecture etc) and Acquisition Platform
  • Duration: 2 hours
  • Type: lecture
  • Room: ETZ E9
• Session 3: SW-Optimization 2 (NAS, Pruning, Quantization)
  • Duration: 2 hours
  • Type: lecture
  • Room: ETZ E9
• Session 4: DNN Creation and optimization (NAS, Pruning) on pre-collected dataset
  • Duration: 2 hours
  • Type: hands-on
  • Room: ETZ D61.2
• Session 5: Personalized Dataset Acquisition, Visualization
  • Duration: 2 hours
    
  • Type: hands-on
  • Room: ETZ D61.2
    
• Session 6: Finetuning of the optimized DNNs on personalized data; Quantization
  • Duration: 2 hours
  • Type: hands-on
  • Room: ETZ D61.2
• Session 7: Compilers/Deployment (AI compilers, TVM, Zigzag, MATCH)
  • Duration: 2 hours
  • Type: lecture
  • Room: ETZ E9
• Session 8: Compilation of DNN models and programming of BioGAP, test of DNNs found in Session #4 on the board, using random inputs
  • Duration: 2 hours
  • Type: hands-on
  • Room: ETZ D61.2
• Session 9: Streaming data from BioGAP to PC and real-time visualization of predicted gestures
  
  • Duration: 2 hours
  • Type: hands-on
  • Room: ETZ D61.2

Reading material:

DNNs, Quantization, Pruning:
Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. "Deep learning". MIT press, 2016.

Hoefler, Torsten, et al. "Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks." Journal of Machine Learning Research 22.241 (2021): 1-124.
Nagel, Markus, et al. "A white paper on neural network quantization." arXiv preprint arXiv:2106.08295 (2021).

NAS:
White, Colin, et al. "Neural architecture search: Insights from 1000 papers." arXiv preprint arXiv:2301.08727 (2023).
Pagliari, Daniele Jahier, et al. "Plinio: a user-friendly library of gradient-based methods for complexity-aware DNN optimization." 2023 Forum on Specification & Design Languages (FDL). IEEE, 2023.
Risso, Matteo, et al. "Lightweight neural architecture search for temporal convolutional networks at the edge." IEEE Transactions on Computers 72.3 (2022): 744-758.

Biotask:
Zanghieri, Marcello, et al. "Robust real-time embedded EMG recognition framework using temporal convolutional networks on a multicore IoT processor." IEEE transactions on biomedical circuits and systems 14.2 (2019): 244-256.

AI Compilation and Deployment:
T. Chen et al., "TVM: an automated end-to-end optimizing compiler for deep learning", OSDI18, 2018
https://tvm.apache.org/
https://mlir.llvm.org/
A. Burrello et al., "DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs," in IEEE Transactions on Computers, 2021
Hsin-I Cindy Liu et al., "TinyIREE: An ML Execution Environment for Embedded Systems from Compilation to Deployment", arXiv, 2022
Martin Maas et al., "TelaMalloc: Efficient On-Chip Memory Allocation for Production Machine Learning Accelerators", ASPLOS 2023
https://greenwaves-technologies.com/gap9_processor/
https://github.com/eml-eda/match