Titre de projet de fin d’études : Détection et classification en temps réel d’événements rares dans des flux vidéo à l’aide de réseaux de neurones quantifiés et d’implémentation optimisée sur FPGA

Project Abstract

This work presents a real-time detection and classification framework for rare events in video streams, leveraging quantized neural networks (QNNs) and hardware-accelerated inference on field-programmable gate arrays (FPGAs). The core motivation is to enable accurate recognition of infrequent, high-impact events (e.g., anomalous activities, unusual objects, or rare motion patterns) within continuous video feeds while meeting stringent latency, power, and resource constraints typical of embedded and edge deployments. The proposed approach integrates model quantization, architecture optimization, and FPGA-specific design strategies to deliver robust performance under real-time conditions. First, we investigate quantization techniques to compress state-of-the-art deep learning models without sacrificing detection accuracy on rare-event datasets. This includes uniform and non-uniform weight and activation quantization, with precision configurations ranging from 4-bit to 8-bit representations. We introduce a calibration-driven retraining workflow that minimizes accuracy loss on skewed class distributions and leverages specialized loss functions to emphasize rare-event samples during fine-tuning. Complementary techniques such as batch normalization folding, channel-wise quantization, and weight pruning are explored to further reduce memory footprints and arithmetic complexity. Second, we design FPGA-friendly network architectures tailored to real-time video analysis. The architectures emphasize temporal consistency through lightweight recurrent or temporal convolutional modules, while maintaining a predominantly feed-forward structure to exploit parallelism on FPGA fabric. We implement fixed-point arithmetic with carefully chosen word lengths and pruning patterns to maximize throughput. A co-design methodology aligns neural network layers with FPGA resources, ensuring balanced utilization of DSPs, BRAMs, and logic blocks, and minimizes off-chip memory bandwidth. Third, we develop an end-to-end hardware-software stack for streaming inference. This includes a modular accelerator engine capable of handling frame-level feature extraction, temporal aggregation, and the final lightweight classifier. We introduce a streaming data path with zero-copy data transfers, ultra-low-latency scheduling, and dynamic reconfiguration to adapt to varying video resolutions and frame rates. The system supports on-device learning for domain adaptation and incremental updates to protect against data drift in real-world deployments. Fourth, we address the challenge of rare-event data scarcity by adopting data augmentation, synthetic minority oversampling, and anomaly-aware evaluation metrics. We propose evaluation protocols that reflect the operational importance of rare events, including precision-recall trade-offs, false alarm rates, and time-to-detection metrics under constrained latency budgets. Empirical validation is conducted on multiple benchmark datasets with annotated rare events and on a custom FPGA prototyping board to demonstrate end-to-end performance. Results indicate a favorable balance between accuracy and efficiency high detection rates for rare events with low false-alarm rates, while achieving millisecond-level latency and reduced power consumption relative to conventional GPU-based baselines. The findings underscore the viability of quantized, FPGA-accelerated deep learning pipelines for reliable real-time surveillance, industrial inspection, and safety-critical monitoring tasks. Potential deployment scenarios, limitations, and avenues for future enhancement, including adaptive quantization and more aggressive sparsity strategies, are discussed to guide subsequent work.

Project Overview

What This Project Is About

---

A practical study on detecting and classifying rare events in real-time video streams using quantified neural networks and a fast, hardware-friendly implementation on FPGA chips. The project focuses on making rare event detection quick and reliable while running on specialized hardware for efficiency.

---

---

The Problem It Addresses

---

In many situations, rare events (such as unusual activities or anomalies) are important but hard to spot quickly in video feeds. Traditional models are slow or require large hardware, making real-time use impractical. This project aims to deliver fast, accurate detection with lower resources by compressing models and running them on FPGA hardware.

---

---

Objectives of the Project

---

1. Build a simple pipeline to detect rare events in video data.
2. Apply neural network quantization to reduce model size and speed up inference.
3. Implement the model on an FPGA platform and test real-time performance.
4. Evaluate accuracy, latency, and resource use on hardware.
5. Provide guidelines for deploying similar systems in practice.

---

---

What You Will Do Step by Step

---

1) Review basic concepts of video processing and neural networks. 2) Collect or simulate video data containing rare events. 3) Train a lightweight model and apply quantization. 4) Map the model to an FPGA and optimize for speed. 5) Run experiments to measure accuracy and latency. 6) Compare with non-quantized baselines. 7) Document findings and best practices.

---

---

---

---

Expected Outcome

An efficient, real-time video analysis system that detects rare events with acceptable accuracy and low hardware requirements. The project should produce a working FPGA implementation, performance metrics, and practical deployment guidance.