Smart Traffic Signal Control using Multi-Agent Reinforcement Learning

Project Abstract

This study presents a novel framework for optimizing urban traffic flow through a distributed multi-agent reinforcement learning (MARL) approach to traffic signal control. By decomposing a city-scale intersection network into a coordination graph, autonomous agents manage local signal phases while leveraging cooperative strategies to minimize overall travel time, stop frequency, and emissions. The proposed system integrates state-of-the-art MARL algorithms with domain-specific enhancements, including queue-length aware reward shaping, congestion-aware action masking, and transfer learning to accelerate adaptation across varying traffic regimes. A hybrid observation space combines real-time vehicle counts, lane-level occupancy, and estimated arrival times obtained from urban sensing infrastructure, with coarse-grained environmental context such as weather and public-event indicators to improve robustness. Agents are trained in a simulated mesoscale environment that captures realistic traffic dynamics, signal timing constraints, and pedestrian interactions, allowing scalable experimentation without disrupting live traffic. To address non-stationarity and partial observability, we implement a centralized critic during training with decentralized execution, augmented by attention-based communication channels that enable adaptive information sharing among neighboring intersections while preserving privacy and scalability. The methodology includes offline pretraining on diverse synthetic scenarios, followed by continual online learning with safe exploration guarantees to accommodate evolving traffic patterns. Performance is evaluated against fixed-timing, adaptive-timing, and centralized optimization baselines using metrics such as average travel time, network-wide throughput, average queue length, stopping rate, and energy consumption. Our results demonstrate substantial improvements in throughput and latency reductions under peak and off-peak conditions, with gains that persist across different city topologies and incident conditions. Sensitivity analyses reveal the system’s resilience to sensor noise, communication delays, and partial observer failures, while ablation studies identify the contribution of each MARL component, including reward shaping, action masking, and inter-agent communication. In addition, the framework supports interpretability features that provide insights into learned policies, such as prioritization heuristics for green phases and emergent coordination patterns among adjacent intersections. The dissertation contributes to the field by (1) presenting a scalable MARL-based traffic signal control architecture suitable for real-world deployment, (2) delivering a comprehensive evaluation framework for urban-scale traffic control with realistic simulators and benchmarks, (3) proposing robust training strategies for non-stationary and partially observable environments, and (4) offering practical guidelines for integration with existing vehicular and pedestrian infrastructure. The outcomes have the potential to reduce congestion-related emissions and energy use, improve travel reliability for commuters, and enhance the efficiency of urban transportation networks through autonomous, cooperative signal management.

Project Overview

What This Project Is About

---

A plain-language overview of how traffic lights can be controlled by smart software that learns from real-world traffic to reduce waiting times and congestion. The project studies how multiple traffic signals can work together using learning algorithms to adapt to changing traffic patterns in real time. It aims to demonstrate a practical, safer, and more efficient approach to city traffic management using computer simulations and simple hardware prototypes when possible.

---

---

The Problem It Addresses

---

Many intersections use fixed timing plans or simple rules that don’t respond well to fluctuations in traffic. This leads to longer waits, increased emissions, and accidents. The project explores how coordinated decision-making among several signals can improve flow and reduce delays in busy urban networks.

---

---

Objectives of the Project

---

1. Understand basic traffic signal control concepts and reinforcement learning ideas at a high level.
2. Design a simple multi-agent system where each signal acts as an agent that learns to optimize its own and nearby intersections.
3. Simulate traffic scenarios to compare learned control with traditional fixed-timing methods.
4. Evaluate improvements in average waiting time and queue lengths.

---

---

What You Will Do Step by Step

---

1. Study introductory materials on traffic signal control and reinforcement learning.
2. Set up a basic simulation environment representing a small road network with multiple intersections.
3. Implement a simple multi-agent learning model where each signal adjusts timing based on traffic feedback.
4. Run experiments under different traffic patterns and record results.
5. Analyze data to assess performance gains and limitations.
6. Document findings, challenges, and potential real-world extensions.

---

---

Expected Outcome

A demonstration that a lightweight multi-agent learning approach can reduce average wait times and smoother traffic flow in simulated networks, with clear guidance on scenarios where it works best and what pitfalls to watch for.