Datathon · Imbalanced classification
Decision strategy beats model complexity.
An end-to-end machine learning pipeline that predicts accident severity (Slight, Serious or Fatal) from about 12,000 mostly categorical records. Only around 1% of cases are fatal, so the work is about not letting the rare, critical class get ignored.
WebGL · move your cursor
0.000
Macro F1 (LightGBM)
~0%
Accuracy
~0
Records
~0%
Fatal cases
// highlights
Whatmakesittick
Optimise for the right metric
Accuracy rewards ignoring rare events, so the pipeline maximises Macro F1 to keep minority classes in play.
Six models benchmarked
Logistic Regression, Random Forest, XGBoost, LightGBM, CatBoost and a neural network. LightGBM won.
Resampling lost
SMOTE produced invalid synthetic data, class weights were unstable and undersampling threw away information. No resampling was best.
Weighted decision threshold
Predicting argmax(probabilities × weights) with weights like [10, 3.5, 1] lifts minority recall without retraining.
Rule-based overrides
Domain-aware rules, such as predicting Fatal above a probability threshold, squeeze out further gains.
Shipped as an app
A Streamlit interface with an input form, real-time predictions and a live view of the class probabilities.
// architecture
Howit'sbuilt
- 01Streamlit appForm · live class probabilities
- 02Decision layerThreshold weights + rule overrides
- 03LightGBMProbabilities over 3 classes
step 01
Clean
Clean raw data and one-hot encode 30+ mostly categorical features.
step 02
Model
Train LightGBM and output class probabilities.
step 03
Weight
Apply the optimised threshold weights to the probabilities.
step 04
Override
Apply rule-based overrides for the critical class.
step 05
Predict
Return the final severity with probabilities in Streamlit.
// under the hood
Thedetails
# Instead of argmax(probabilities):
weights = [10, 3.5, 1]
prediction = argmax(probabilities * weights)
if fatal_probability > threshold:
prediction = "Fatal"Model comparison (Macro F1, no resampling)
| Model | Macro F1 |
|---|---|
| LightGBM | 0.506 |
| XGBoost | 0.4728 |
| CatBoost | 0.4711 |
Key learnings
- ◆Decision strategy matters more than model complexity.
- ◆SMOTE fails on high-cardinality categorical data.
- ◆Threshold tuning is critical for imbalanced problems.
- ◆Feature interactions matter more than raw features.
// stack
Builtwith
- Python
- scikit-learn
- LightGBM
- XGBoost
- CatBoost
- Streamlit
- pandas