Matching 10 million business records: what worked in the Amazon ML Challenge
How our team got from 0.927 to 0.964 F0.5 by fixing recall first, letting records pick entities, and judging every match against its rivals.
The task in the Amazon ML Challenge 2026 was entity resolution: decide which records from two noisy sources belong to which of 2.2 million businesses in a third. About 10.3 million records, names in English and Indic scripts, French addresses that only appeared in the test set, and plenty of deliberate decoys.
Our team, Team Minutes, finished with a best public leaderboard score of 0.964 macro F0.5. These are the four ideas that moved the score most.
1. You can't match what you never compare
Comparing every pair is impossible at this size, so the first step is blocking: build an inverted index on name tokens, prefixes, address pieces and house numbers, and only compare records that share a key.
The trap is measuring blocking on a sample. A sample hides crowding, because common names have far fewer neighbours. We always queried validation entities against the full 10.3 million-record pool, which made the recall number honest.
2. Let records choose entities too
Forward blocking (each entity keeps its top 50 records) left good matches buried behind similar-looking candidates. Adding reverse blocking, where each record also proposes its top 10 entities, lifted full-density recall from 94.2% to 97.1%.
3. Judge a match against its competitors
A pairwise model looks at one entity and one record at a time. That is exactly what decoys exploit: same address, name changed by one syllable.
Our second-stage model looked at how each record scored against every entity that retrieved it: the margin to the best rival, its rank, how many entities wanted it. The margin became the single most important feature, and each record was then assigned to at most one owner.
4. Tune for the metric you are scored on
F0.5 rewards precision more than recall, so we picked the decision threshold by grid search on a held-out set of 25,000 entities. The curve was flat between 0.60 and 0.75, which told us the model was stable rather than lucky.
The progression
- Baseline blocking + LightGBM: 0.927 public
- Richer blocking keys and decoy features: 0.940
- Indic transliteration and candidate statistics: 0.945
- Reverse blocking and one-to-one ownership: 0.964
The lesson I took away: most of the gains came from fixing what the model could see, not from a bigger model.