The training-data pipeline — adapters, alignment, synthesis, Parquet files.
View All TagsI taught the address parser three new countries overnight. Two of them landed. The third failed a regression check I trusted, and the reason was concerning: the new addresses never included their own country, and the model was learning to stop labeling countries at all.
Most sequence labelers throw away the parts of an address they don't understand, and the output format is built to hide it. We made ours hand every dropped character back, then aggregated them into a ranked list. The top of the list turned out to be the Polish alphabet.
Open the demo and type an address in Kabul, and the demo returns no result. It does not place a pin a few streets off or fall back to the city center, because the gazetteer has no entry for Kabul. Hong Kong, Tirana, Chişinău, and every place in the Democratic Republic of the Congo return no result as well. We had been calling this gazetteer "world coverage" for weeks, and it covered 97 of the world's ~195 countries. The other ninety-odd countries were absent from the file.
We're trained to be suspicious of a score that looks too good. The scores that say you failed get a free pass — and this week three of them were lying, including one that had us reporting kilometers for a product that lands in meters.
We taught our parser the hardest street cases it gets wrong, the aggregate score climbed, and we almost shipped it. Then a held-out probe caught it forgetting what a town is.
Our failure taxonomy now ranks boundary instability first among the parser's weaknesses. We
Our training corpus is built from source. The build combines hundreds of millions of address