English | 简体中文

Q-Learning Maze

A small Python maze demo that trains a Q-learning agent and visualizes both the training loop and the final greedy-path replay with pygame.

Overview

Q-Learning Maze is a compact reinforcement learning demo built in Python. It generates a solvable maze, trains an agent with Q-learning, renders the learning process with pygame, and replays the greedy path discovered after training.

The project is intentionally small enough to read end-to-end while still showing the full loop of environment design, value updates, epsilon-greedy exploration, and visual feedback.

Features

• Random solvable maze generation with walls and traps
• Visualized Q-learning training loop rendered with pygame
• Greedy path replay after training completes
• Runtime tuning through MAZE_EPISODES and MAZE_TRAINING_DELAY_MS
• Automated tests for the maze environment and Q-learning agent

Quick Start

Create a virtual environment, activate it, and install the dependencies:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

Run

Start the desktop demo with:

python main.py

If your shell exposes only python3, use:

python3 main.py

Headless Mode

For non-GUI environments, run the demo with SDL's dummy video driver:

SDL_VIDEODRIVER=dummy MAZE_TRAINING_DELAY_MS=0 MAZE_EPISODES=5 python main.py

This is useful for CI, remote shells, or quick runtime verification after environment changes.

Tests

Run the full test suite with:

python -m pytest

Project Structure

• main.py: Training loop, runtime configuration, pygame rendering, and greedy-path replay
• maze_env.py: Maze generation, solvability checks, state transitions, rewards, and drawing helpers
• rl_brain.py: Q-learning agent, epsilon-greedy action selection, value updates, and path extraction
• tests/: Unit tests for environment behavior, Q-learning updates, and runtime configuration

Notes

Python virtual environments store absolute paths internally. If the project is moved to a new directory, the existing .venv may become stale and activation can fall back to the system interpreter.

If that happens, recreate the environment and reinstall dependencies:

rm -rf .venv
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt