Play The NumbersNRL ANALYTICSRound 26
← ArticlesMETHOD · 5 min read

The case for publishing model limitations

A clean percentage can look more certain than the information underneath it. PTN treats limitations as part of the product, not fine print to hide.

Every model leaves something out

Models simplify reality. That is what makes them useful, but it also creates blind spots. A team-strength model may not know that a key spine player has been ruled out an hour before kickoff.

The responsible response is not to pretend those gaps do not exist. It is to identify what is included, what is missing and how that should change the way the output is read.

Transparency improves the model

Publishing assumptions makes future improvements measurable. If PTN later adds injury value, strength of schedule or venue-specific effects, the change can be backtested against the earlier baseline.

That is much stronger than quietly changing a model after poor results and presenting the new output as if it had always existed.

The standard PTN should aim for

The goal is not to make the model look perfect. It is to make it understandable, testable and progressively better.

  • Date every model snapshot
  • Track predictions before results
  • Publish core inputs
  • Backtest changes
  • Separate observed statistics from PTN-derived ratings
PLAY THE NUMBERSObserved data and PTN-derived analysis are kept separate.

Where PTN uses official NRL statistics, the raw inputs are identified as source data. Ratings, percentiles, model probabilities and interpretation are Play The Numbers calculations.

KEEP READING

More PTN analysis