Python · from GitHub
Lab Activities for the course Reinforcement Learning (CAI-2508)
Lab work for CAI2508 (Reinforcement Learning): a series of standalone Python scripts implementing classical RL algorithms from scratch and via Gymnasium environments, covering dynamic programming, Monte Carlo methods, temporal-difference learning, and multi-armed bandits.
WebGL · move your cursor
// readme
01
gymnasium (with the toy_text extras for FrozenLake/Blackjack)numpypandasmatplotlibseabornInstall everything with:
pip install "gymnasium[toy-text]" numpy pandas matplotlib seaborn
Some lab sheets render environments in a window (render_mode="human") or record episodes to video (RecordVideo), so a display/OpenGL backend is needed to run them interactively.
02
| File | Topic |
|---|---|
| LabSheet01.py | Introduction to Gymnasium — explore FrozenLake's state/action spaces and transition probabilities, run a random policy |
| LabSheet02.py | CartPole with a random agent, recording episodes to video via RecordVideo |
| LabSheet03.py | Solving FrozenLake with Value Iteration and optimal policy extraction |
| LabSheet04.py | Solving FrozenLake with Policy Iteration |
| LabSheet05.py | Every-Visit Monte Carlo prediction on Blackjack |
| LabSheet06.py | First-Visit (Single-Visit) Monte Carlo prediction on Blackjack |
| LabSheet07.py | On-Policy Monte Carlo Control with an epsilon-greedy policy on Blackjack |
| LabSheet08.py | TD(0) Learning (temporal-difference prediction) on FrozenLake |
| LabSheet09.py | SARSA (on-policy TD control) on FrozenLake |
| LabSheet10.py | Q-Learning (off-policy TD control) on FrozenLake |
| LabSheet11.py | Epsilon-Greedy exploration for a 2-armed bandit |
| LabSheet12.py | Softmax Exploration for a 2-armed bandit |
| LabSheet13.py | Upper Confidence Bound (UCB) algorithm for a 2-armed bandit |
| LabSheet14.py | Thompson Sampling for a 2-armed bandit |
| LabSheet15.py | Contextual bandit case study: banner ad selection with epsilon-greedy on synthetic click-through data |
LabSheets 11–14 use a small hand-rolled BanditTwoArmedHighLowFixed environment (replacing the unmaintained gym_bandits package) with a high-payout arm (p=0.8) and a low-payout arm (p=0.2).
03
Each lab sheet is self-contained — run it directly with:
python LabSheet03.py
Scripts that use render_mode="human" will open a live environment window; scripts using RecordVideo (e.g. LabSheet02) write .mp4 files to videos/.
04
The videos/ directory contains sample episode recordings (rl-video-episode-*.mp4) generated from the CartPole random-agent lab (LabSheet02).
05
pytorch-pod.yaml is a Kubernetes manifest for provisioning a PyTorch development pod (with a NodePort service) on a shared DGX cluster — used for any labs requiring GPU compute, not required for the core RL exercises above.