Amazon ML Challenge 2026
Business entity resolution across 10.3 million records
Match the same business across three noisy sources: 2.2M entities against 10.3M records.
Normalisation, forward and reverse blocking, a two-stage LightGBM model and one-to-one assignment.
0.964 F0.5 on the public leaderboard; 0.978 on validation.
- Role
- Machine learning engineer
- Year
- 2026
- Team
- Team Minutes, with Sidhant Malik and Pranav Suri
- Stack
- Python, LightGBM, pandas, RapidFuzz, Parquet
Overview
A two-stage blocking and gradient-boosting pipeline that matches the same business across three noisy data sources, including names in Indic scripts and French addresses.
The problem
The same business appears in different sources with abbreviations, typos, reordered addresses, native-script names and deliberate decoys at the same address. Comparing every pair is impossible at 2.2 million entities against 10.3 million records.
What I built
- 1Normalised names and addresses: Unicode transliteration for Indic scripts, legal-form dictionaries for the US, India and France, state codes and a phonetic skeleton.
- 2Built forward and reverse blocking on a multi-key inverted index, raising candidate recall at full data density from 94.2% to 97.1%.
- 3Trained a LightGBM model on about 80 pair features, then a second-stage model that compares how each record scores against every competing entity.
- 4Assigned each record to one owner and tuned the threshold for macro F0.5 on a held-out set.
Highlights
- Validation macro F0.5 of 0.978 at full density; best public leaderboard score 0.964.
- Competition-aware matching targeted decoys and same-name siblings, the main source of errors.
How it works
- Normalise names and addresses
- Block: forward + reverse index
- Stage 1: LightGBM, ~80 features
- Stage 2: entity-vs-entity competition
- One-to-one assignment + threshold
Key decisions and trade-offs
Measure recall at full density
A sampled validation set hides crowding. Validation entities were always queried against the full 10.3M-record pool.
Let records choose entities too
Reverse blocking lets each record propose its best entities, lifting candidate recall from 94.2% to 97.1%.
Judge a match against its competitors
The second-stage model compares each record across every entity that retrieved it; the margin to the best rival became the top feature.
Optimise for the metric
The threshold was tuned for macro F0.5, which rewards precision; the curve is flat between 0.60 and 0.75.