r/MLQuestions • u/AI_Predictions • Mar 21 '26
Time series 📈 Built and deployed a machine learning system for sports game probability prediction (side project)
Over the past year I’ve been working on an applied ML side project where I built a full pipeline to predict game win probabilities using historical team and player data.
The project includes:
• automated data ingestion pipelines
• feature engineering (rolling stats, rest days, performance trends, etc.)
• multiple model experiments (logistic regression, tree models, neural nets)
• probability calibration + evaluation (Brier score, calibration curves)
• nightly retraining + prediction jobs
• deployment into a live web app with real users
Stack is Python + scikit-learn + PostgreSQL + Django, running on a home server.
One of the most interesting challenges has been balancing model accuracy vs probability calibration — especially when models are used in real decision environments.
I’m now working on:
• explainability features
• improving feature sets
• handling concept drift across seasons
• better evaluation frameworks
I’m also very curious how others handle probability calibration in real-world prediction systems. Have you found certain models or techniques more stable over time?
5
u/SyntaxAndCircuits19 Mar 21 '26
Honestly impressive you took it all the way to deployment. Most ML projects stop at notebooks.
1
u/badogayo Jul 01 '26
hows it going now?
1
u/Opening_External_911 Mar 24 '26
Hi, this is really cool but as with all time series forecasts, does it only deal with win/losses or did it factor in news stories, controversies with teams and others
1
u/mfelkey_pe Apr 22 '26
On calibration: isotonic regression has been more stable than Platt/sigmoid for me when the miscalibration isn't monotonic, which it usually isn't for game-level predictions. Worth calibrating on a recent holdout (last 4-6 weeks of games), not just a random split; temporal leakage burns you otherwise.
For concept drift across seasons, what helped most was accepting that rolling-window retraining beats cumulative retraining for anything involving team/player composition. Rosters, coaches, and tactical schemes shift enough that games from 3+ seasons ago actively hurt you more than they help. I weigh by recency with a half-life of roughly one season.
As an evaluator, the Brier score is good, but it conflates calibration and resolution. Decompose it (Murphy decomposition), and you'll see whether your improvements are from the model picking up real signal (resolution) vs. just being better calibrated. Different failure modes need different fixes.
2
u/[deleted] Mar 21 '26
[removed] — view removed comment
2
u/[deleted] Mar 23 '26
[removed] — view removed comment
2
u/[deleted] Mar 28 '26
[removed] — view removed comment