Recruitment models perform noticeably worse in smaller competitions, which is unfortunate because those are exactly the markets where value is supposed to be found. The failure is structural rather than a matter of tuning.
Short seasons produce thin evidence
Many competitions play considerably fewer fixtures than the largest leagues, and a young player may feature in only part of them. The resulting sample is small before anything else is considered.
Rate statistics calculated on small samples swing widely, so a player can appear elite or ordinary depending on a handful of events.
Models that assume a full season of evidence overstate their confidence when applied to half of one.
Weak opposition distorts the difficulty
A talented player in a low-quality division faces defending that a stronger competition would not tolerate. Actions that succeed there may not succeed elsewhere.
The data records the outcome without recording how hard it was, so the same completed dribble carries the same weight regardless of opponent.
Opposition adjustments help but require reliable estimates of team strength, which small leagues also struggle to supply.
Data collection is thinner at the same time
Coverage in smaller competitions often lacks tracking data and uses a reduced event specification. The measurements available are the coarsest ones.
Camera positions may be poor and some matches may not be captured at all, which introduces gaps that are not random.
Models trained on rich data from major leagues degrade when fed a sparser and noisier version of the same inputs.
Selection bias runs through the whole exercise
The players who move from small leagues to large ones were chosen because someone believed they would succeed. Outcome data therefore describes an already-filtered group.
Training a model on those transitions teaches it about players scouts liked, not about players from that league generally.
This makes historical transfer success rates a misleading guide to how well a model would perform on the unfiltered population.
The workable approach is narrower
Clubs that recruit successfully from these markets tend to use models to generate a longlist and rely on repeated live observation to make the decision.
They also concentrate on a small number of competitions where they have accumulated their own historical record, which partly substitutes for the missing data.
Depth in a few markets beats coverage of many, and the clubs with reputations for finding players in obscure places almost always look in the same few places repeatedly.