Reinforcement learning · Gymnasium
An agent that learns to buy low and sell high.
An interactive reinforcement learning project where an AI agent learns to swing trade on a scrolling price chart. Built with Python, Gymnasium and Pygame, it shows the learning process live, on synthetic curves or real historical prices from Yahoo Finance.
WebGL · move your cursor
0
Actions (Hold, Buy, Sell)
0
RL agents
0
State features
0
Default training episodes
// highlights
Whatmakesittick
A proper Gymnasium environment
Discrete actions (hold, buy, sell) and a continuous state of current price, previous price and inventory.
Reward design
Realised profit is rewarded, losses and illegal moves such as selling with no inventory are penalised, and a small step penalty discourages sitting in cash.
Tabular Q-Learning
Discretises price trends into bins and updates a Q-table with the Bellman equation. Ideal for understanding RL fundamentals.
Deep Q-Network in PyTorch
Neural policy with a replay buffer and target network, handling continuous states natively on MPS, CUDA or CPU.
Real market data
Train on synthetic sine-plus-noise curves or fetch real prices (AAPL, BTC-USD and more) through yfinance.
Trading AI Desk
A Pygame terminal with dynamic axes, real dates, an area-filled chart, a P&L heads-up display and live buy and sell triangles.
// architecture
Howit'sbuilt
- 01Pygame Trading DeskChart · HUD · buy/sell markers
- 02AgentsQ-Learning · DQN (PyTorch)
- 03Gymnasium environmentSynthetic curve or yfinance data
step 01
Choose
Pick an agent (Q-Learning or DQN) and an environment (synthetic or real) from the command line.
step 02
Train
Run many episodes while epsilon decays and the loss settles. Models save automatically.
step 03
Test
Open the Trading AI Desk and watch the trained agent trade.
// under the hood
Thedetails
# Train a DQN agent on real data python main.py --mode train --agent dqn --env real --ticker BTC-USD --episodes 500 # Watch it trade in the Pygame desk python main.py --mode test --agent dqn --env real --ticker BTC-USD
// stack
Builtwith
- Python
- Gymnasium
- PyTorch
- Pygame
- yfinance
- NumPy