Rent an API and the address is now someone else's business. Every lookup is a billable request and a row of your users' home addresses leaving your infrastructure, and the free tier caps out at one request a second, so a table of any size takes days. Self-host Nominatim instead and you've signed up for PostgreSQL, an osm2pgsql import measured in hours and tens of gigabytes, and a service you run on a box you can never embed inside your own app. Either way, the thing you wanted — geocode(address) — sits on the far side of a week of setup.

Mailwoman is a parser and geocoder that runs from a SQLite file and a ~40 MB model, entirely on your own machine (it makes no outbound network request). None of that helps if wiring it in is its own week, though. So the work of the last stretch wasn't the model. It was making the thing install the way the rest of your dependencies do.

Your existing client already works​

Mailwoman ships endpoints that speak the wire formats your stack was already built against. @mailwoman/nominatim answers /search and /reverse and /status the way Nominatim does. @mailwoman/photon returns the GeoJSON that Photon clients expect for autocomplete. @mailwoman/libpostal parses and expands on the /parse and /expand routes. The client library you wrote against those services doesn't know it's talking to something else.

geopy is the clearest case. Its Nominatim class takes a domain, so pointing it at a local instance is two keyword arguments and no code change:

from geopy.geocoders import Nominatim

geo = Nominatim(domain="localhost:8080", scheme="http", user_agent="my-app")
loc = geo.geocode("1600 Pennsylvania Avenue NW, Washington, DC 20500")
print(loc.latitude, loc.longitude)   # 38.8977 -77.0365

Same call you already had. It now resolves against a rooftop-level result on your own box, The API returns an OpenCage-style annotations block — timezone, coordinate formats, the things upstream Nominatim leaves empty — hanging off loc.raw.

One docker run for the server​

The container installs the published packages and bakes the model weights in, so a fresh pull parses addresses with no data at all. The container needs a gazetteer at runtime, so it mounts one at /data:

docker run --rm -p 8080:8080 \
  -v /path/to/mailwoman-data:/data:ro \
  ghcr.io/sister-software/mailwoman:latest \
  node node_modules/@mailwoman/nominatim/out/cli.js serve

You have an endpoint. Curl it, point geopy at it, put it behind your load balancer. The image also runs the native /v1 API and the Photon and libpostal drop-ins on their own commands, so one artifact covers the formats your services speak.

In-process, when a round trip is silly​

A Node service that needs a coordinate shouldn't have to talk to itself over HTTP. @mailwoman/fastify registers the routes and a fastify.mailwoman decorator over the local pipeline, built once at startup:

import Fastify from "fastify"
import mailwoman from "@mailwoman/fastify"

const app = Fastify()
await app.register(mailwoman, { resolveDatabasePath: "/data/wof/candidate.db" })

const { lat, lon } = await app.mailwoman.geocode("350 5th Ave, New York")

For the frontend, @mailwoman/react is the parse, geocode, and POI components that drive our own docs demos, split into small pieces and headless hooks so you can take the usePOISearch logic without our markup. Python and Rust callers have generated clients on PyPI and crates.io (mailwoman-client). And for agents, @mailwoman/mcp exposes parsing, geocoding, POI search, and spatial export as MCP tools over stdio, so a model can look an address up as a tool call instead of guessing at coordinates.

What it costs you​

You bring the gazetteer. The model is in the image, but geocoding needs the admin database. You download it or build it. The worldwide candidate file is about 1.4 GB, and it's read-only, so it mounts :ro and you never think about it again. Until it's in place, the server parses but won't geocode. That is a degrade with a clear message rather than a crash. If you're not sure what's present, mailwoman doctor checks the weights, the data root, the gazetteer, and the runtime and tells you the one command to fix each gap.

The other cost is the shape. This is SQLite rather than a cluster, so one instance is one node. That's the point on a single box or inside an app, and it means you scale by running more copies behind a balancer rather than by tuning a shared database. If your plan was a managed Postgres fleet, this isn't that, on purpose.

And the license is AGPL, with a commercial option. If you're embedding it in something you ship closed, that's the conversation to have first.

Where it stops​

None of this changes what the geocoder does — it's the same parser and the same resolver from a month of eval posts. What changed is the distance between reading this and having an endpoint. That step is now a docker run or an npm install rather than a migration. The address still turns into a coordinate. It does it inside the stack you already have.

The easy part​

The parser reads a raw string such as 2835 Woodhaven Rd, Shirley, V9Z1G6 and labels every piece: house number, street, city, and postcode. It is a neural sequence model, and it learns a new country from a few hundred thousand real examples. We had a new source for those examples: Overture's open addresses theme, with millions of rows per country, a permissive license, and no API key. Canada returned sixteen million addresses, Mexico thirty million, and Brazil ninety million.

Canada and Mexico trained without problems. Mexico showed the main improvement: the model's accuracy on Mexican cities went from mediocre to good. It now reads Calle Zeverino León 749, 28809 Manzanillo and labels Manzanillo correctly instead of guessing. The model already handled Canada reasonably well, because Canadian addresses resemble American ones, so adding Canada neither cost nor gained anything measurable. Both countries went into one training run, and the whole battery passed. I then prepared Brazil as the third country.

The guard that said no​

Every model change runs a check before it can reach production. Part of the check confirms that the new data worked. The required part checks whether anything else got worse. We keep a frozen set of graded addresses and require that no category regress by more than two points. The check is simple, and it is what lets us move quickly without shipping unnoticed regressions.

Brazil built and trained without errors. The check then reported one failure. Country recall, the model's ability to extract the country from an address that includes one, had dropped more than five points. Every other category (cities, streets, postcodes, and house numbers) stayed flat. Only country dropped, and by enough to fail.

The earlier run explained the pattern. The run with only Canada and Mexico had lowered country recall by about a point and a half. That drop was under the threshold and easy to dismiss as noise. Brazil enlarged the existing problem. The two runs showed a lower country-label rate after more addresses were added.

What the addresses forgot to say​

I then inspected the training rows themselves instead of the model. None of the 666,000 new Canadian, Mexican, and Brazilian rows contained a country.

That is normal for the source. An Overture address is street, city, postcode. The country is implied by the file it came from, in the same way that a domestic letter omits the country from the envelope. Each row is correct on its own. A model, however, learns from the whole set. I had given it two-thirds of a million addresses, and none of them included a country. Trained on that data at the weight I had given it, the model learned that addresses do not include their country, in addition to learning three new countries. When the graded set gives it 1600 Pennsylvania Ave NW, Washington, DC, USA, which does include a country, the model has learned to skip the USA token.

I had checked that the new rows had the tags I wanted: street, city, and postcode, all present and correct. I never checked for the tag that was missing, and the missing tag caused the regression.

The fix is old, and we already own it​

We built the fix for this a year ago and did not apply it to the new pipeline. Our older locale recipe, the one behind the European countries, renders addresses in their native form and also renders a share of them in an international form, the kind written on a parcel crossing a border: 12 Rue de Rivoli, Paris, France. That form includes the country deliberately, and it is why adding France never taught the model to drop France. The new Overture pipeline skipped that step and used the addresses exactly as given, without countries.

Brazil therefore needs only a small change, and the design stays as it is. The change is to render a fraction of the new rows with the country included, so the training set stops teaching the model to omit it. Canada and Mexico ship as they are, because that run passed. Brazil waits for the next build, and the fix is smaller than it seemed at 4 a.m.

The lesson, in a sentence​

June's lesson was to grade the coordinate the user sees rather than the layer you happened to build. This one is related: a model also learns from what your training data consistently omits. Every field you consistently leave out is a field you are teaching the model to leave out. Coverage includes what you put in the corpus and also what you leave out. The model can learn an omission as a rule when the omission repeats across a million rows.

A simple two-point regression check caught this before any user saw it. The same check has now caught two different problems of this kind.

So you ship that parser. It looks great on the addresses you tested it against. And you have no idea what it's dropping in production, because the output format was built to hide exactly that.

We got tired of not knowing. So we made the decoder keep a promise: every byte of the input lands in exactly one typed span, including the bytes we can't classify. The runs the model leaves untagged stop being absence and become a thing you can hold — an unknown span, with a start, an end, and the literal text the model couldn't place. Concatenate all the spans back together and you get the original string, character for character. No byte gets lost, because there's nowhere left for it to go.

That's a small change to a decoder. It turns into a diagnostic you can point at production addresses.

The shopping list​

Once the gaps are typed, you can count them. We ran the parser over a few thousand real addresses and folded every unknown span into a frequency table: a ranked list of what the model systematically fails to read. We started calling it the shopping list, because that's what it is. The next retrain, in priority order, written by the model's own failures instead of our hunches.

The first thing the list did was catch us lying to ourselves. Ninety-eight percent of inputs had an unknown span. Ninety-eight! That number is worthless, and it took about four minutes to see why: the gaps were almost all whitespace and commas, the spaces between tokens that no component owns. A metric that's mostly punctuation will tell you the sky is falling on every single address. Once we separated the trivial delimiter gaps from gaps holding an actual letter or digit, the direct figure dropped to nine percent. Nine percent of addresses contain a run of real content the model can't place. That one you can act on.

What it can't read is Polish​

Then we aimed the same tool at OpenAddresses data from nine countries, and the shopping list stopped being abstract.

The United States and Australia came back spotless: zero content gaps. France, ten percent. Italy, two. And then Czechia at eighty-four percent and Poland at seventy-seven, towering over everything else on the board.

The top entries said why in the plainest possible terms: ł, á, ó, í, ě, ś, ę, č. The whole Latin-Extended block. Our tokenizer isolates a Slavic diacritic into its own little subword, the model was never shown enough of them to learn what they are, so it files them under background noise. Grudziądz comes apart at the ą: the model labels Grudzi, drops the accent, and picks the city back up at dz, as if the letter in the middle were a coffee stain. Bohaterów splits at the ó and takes the neighboring house number down with it.

We had a comfortable theory that this was a decoder bug, a character-offset mismatch around multi-byte characters, the kind of thing you fix in an afternoon without touching a GPU. We checked before we wrote it down. It wasn't the decoder. The offsets were fine. The model only doesn't know these letters, and there is no clever preprocessing that fixes not-knowing. It's a retrain. The difference is that now we know precisely which characters to weight and which locales to leave unchanged, because Australia and the US don't need a thing.

Honesty you can query​

A model that guesses without a word is a liability, because you learn what it can't do from a customer. A model that hands you a typed, countable record of its own blind spots is something you can plan around. We didn't make the parser smarter this week. We made it direct about what it doesn't know, and that honesty turned out to be a feature you can run a GROUP BY over.

The shopping list is checked in, and it runs before and after every retrain now. Next time you build something that classifies text, make it account for the text it couldn't classify. You'll be surprised how loudly it starts telling you where to look.

Update — July 2​

We wrote "it's a retrain" and then we tested it, because writing a thing down is not the same as it being true. The retrain held English and made Czech worse. What the ranked list had found was one layer lower: the tokenizer's vocabulary stored every one of those characters as its own lonely subword and not a single piece containing them, and a unigram model cannot emit a subword that isn't in its table. No volume of training data teaches a model to say a word its alphabet can't spell.

So the fix wasn't a retrain. We trained a small SentencePiece model on Czech and Polish addresses, spliced its 10,582 diacritic-containing pieces into the vocabulary, and initialized each new embedding from the mean of the pieces it used to shatter into. Zero GPU hours. Wrong-city rates fell from 22.4% to 14.8% in Czechia and from 27.9% to 7.9% in Poland, and Slovak and Slovenian improved with no further work because they share the codepoints. It shipped the next day.

The shopping list went from measurement to a shipped fix in under twenty-four hours. The fix was cheap because the blind spots were countable. Grudziądz is whole now.

It does this with Sydney too (a hamlet in Ohio), and Zurich (Kansas), and Toronto (also Ohio, as it happens). The pattern is always the same: you name a famous foreign city, you name the country it's in, and the geocoder hands you back its smallest American namesake like it's doing you a favor.

The fix that felt like progress​

When you find a bug like this, the first idea is almost always the expensive one. Ours was: the country router doesn't know enough countries. Teach it that "Austria" means Austria — feed it more examples, retrain the model, ship a new set of weights. It's a satisfying plan. It feels like progress, because training a model is the kind of work that looks like work.

We had the GPU budget queued and the issue written up as a model gap. And then, because we've been burned before by fixing the wrong layer, we did the cheap thing first: we asked the model what it saw.

Here is Vienna, Austria straight out of the parser:

Vienna   → locality
Austria  → country

The model knew. It tagged Austria as a country, directly, every time — Australia, Switzerland, Canada, all of them. There was no model gap. The thing we were about to spend a retrain on was already working perfectly. The bug lived one layer down, in the part of the system that takes a correct parse and turns it into a place on Earth.

Forty-five Viennas and a popularity contest​

The resolver's job is to take the word "Vienna" and decide which Vienna. Our gazetteer knows about forty-five of them in the United States, plus the real one in Austria, plus a few others. Faced with that pile, the resolver did the reasonable-sounding thing: it picked the most prominent match by population and name — and then it threw the country token away. It never asked the one question the address had already answered for it.

So Vienna resolved on its own merits, the populous American candidates outranked the Austrian one in the bag, and Austria sat there in the parse, ignored, like an unread return address. The model had said which country in plain text, and the resolver wasn't listening.

The direct version of the bug: the parse and the resolve each made a locally sensible decision, and no stage checked that they agreed.

The fix we didn't make​

There's a tempting shortcut here, and we've watched it sink geocoders before. You could keep a list. "If the address contains Austria, force the country to AT." It would work for Vienna, and Sydney, and Zurich. It would also be the first paving stone on the road to a thousand-entry exception table that someone maintains forever and no user trusts — the same road that turned older rules-based geocoders into archaeology. We don't pin, and we don't keep lists of special cases the model is supposed to defer to.

The fix instead asks the parse and the resolve to be consistent with each other. When the address includes an explicit country and the locality it resolved to lives somewhere else, re-pick the locality to the one that sits inside the country the address states. The country code comes from the model's own country emission, normalized through an ISO table we already had. The gazetteer's own country column does the rest. No list. No prior. It generalizes to every country at once, because it isn't about any particular country — it's about the address agreeing with itself.

The fix has a guard rail, because this kind of fix typically goes wrong at that point. It only fires when the country is the most specific geographic evidence in the address. Write Springfield, IL, USA and the state is right there doing the disambiguating, so the new pass stays out of the way and Springfield stays in Illinois — not the largest Springfield in America. The fix warrants its keep precisely by knowing when not to act.

What it bought, and what it cost​

We measure these things against coordinates rather than labels, because a name-match metric would have told us this bug didn't exist. The direct yardstick is a bare City, Country query — the exact thing a user types into the box — scored by how close the answer lands to the real place.

Forty-five countries that used to need a hint and now don't, a twenty-four-point jump on the query shape people use, and the cost on the GPU meter was zero. The thing we'd budgeted a retrain for was a hundred-odd lines in the resolver.

The fix does have a direct cost: it can only re-pick to a place the gazetteer has. Where the gazetteer lacks a city, the country in the address can't summon a place the gazetteer lacks, and the fix correctly declines to guess. That's a coverage problem, and it's a different change for a different day.

The lesson​

We almost retrained a model to fix a bug the model didn't have. The diagnostic that saved us was thirty seconds of parsing seven addresses and reading the output — cheaper than the issue we'd written, never mind the GPU hours behind it.

The reflex is to train. The discipline is to look first. The model knew it was Austria the whole time. We had to teach the rest of the system to listen.

(One more for the honesty file: the test that proves Sydney now lands in Australia failed the first time I ran it — by 7,532 kilometers. The fix was fine. I'd dropped the minus sign on Sydney's latitude and parked it in the northern hemisphere. The check caught my typo before it caught a real regression. That is exactly the order you want.)

The geocoder couldn't find Tirana​

Start with the symptom. Ask mailwoman's drop-in API for Tirana, Albania and you got no result. Same for Yerevan, Kabul, Ulaanbaatar, Sarajevo — 147 countries where the candidate gazetteer, the table the demo and the drop-ins query, held exactly zero rows. You didn't get a wrong coordinate, you got silence, as if the country didn't exist.

That gap is easy to miss, because the cities that do resolve — Berlin, Paris, Tokyo — resolve beautifully, and those are the ones you check by hand. The hole was in the long tail of countries that no one on the team types into a demo at 2am.

If you've shipped a data pipeline you know how this happens. A from-scratch rebuild enriched every existing place with alternate-language names, so Berlin now answers to Berlino, Берлин, 柏林, all 134 of its surface forms. Wonderful. But enriching the places you have adds no coverage for the places you don't, and the gap-fill that was supposed to add the missing countries lived in a side index the candidate table never folded in. Every piece worked. The join between them was empty, and no component failed loudly enough to notice.

The plan was wrong, and a ten-minute probe said so​

The plan blamed exonyms — the Warsaw-versus-Warszawa problem, where the address uses one name and the gazetteer holds another. Reasonable theory, and an expensive fix: a model retrain to learn the aliases.

So before betting the night on it, we ran the cheap version first. For the top cities of every failing country, is the local name already in the table? If it is, exonyms are the problem. If it isn't, it's coverage. The answer came back in ten minutes: exonym reliance was about one percent. The alt-names were already there. The cities themselves were missing.

That killed the expensive theory and pointed at a boring data fix: pull the real GeoNames town-level dumps for all 147 countries and rebuild the table. It worked the way data fixes do — completely. Before, zero of twenty-five sampled capitals resolved. After, twenty-five of twenty-five, every one inside six kilometers, Tirana to the meter. The countries that already worked came through byte-for-byte identical, because every new row sat in a country that previously had none. No regression to argue about.

That's the result, and we staged it to ship. It's the part you'd put in the changelog. I learned more from what came next.

Then the eval lied to me​

With coverage handled, we asked a different question: when mailwoman does parse a foreign address, does it land in the right place? We ran the assembled-coordinate eval on Portugal, Poland, and Australia — real addresses, scored on distance to truth.

The numbers were ugly. Portugal's median error came back at 47 kilometers. Poland 116. Australia 798. The parser was reading the street name as the city and resolving to a town with that name three regions over. Feed it a perfect parse and the error dropped under a kilometer. Clean story: the parser is the bottleneck, this warrants a GPU retrain, write it up as a GO.

I almost did. Then I checked which model the eval had loaded.

It was a six-week-old build. The dev symlink that points at "the shipped model" had drifted to an old one — a test re-creates that symlink as a side effect, and it had been pinning the wrong file for weeks. Every eval that loaded the model by its default path was grading something we retired in May.

Point the eval at what we ship and Portugal's 47 kilometers becomes 0.8. Poland 2.3. Australia 1.2. The disaster evaporated. The shipped parser was fine the whole time. The only broken thing was the ruler I measured it with. The retrain I was about to recommend would have fixed a problem that didn't exist — and the multilocale parser that does fix this, for the cases where it's real, already sits in a validated checkpoint waiting on a promotion that never came. We didn't need a GPU. We needed to read a symlink.

What went wrong, both times​

Twice in one night the thing being measured wasn't the thing we thought. The plan's diagnostic ran without the candidate database loaded, so it blamed coverage on exonyms — the same missing-database confound that put Warsaw in Indiana and faked a whole scoreboard a few weeks back. My parser eval ran against a stale model, so it blamed the shipped parser for a retired one's mistakes. Same failure wearing two costumes: the harness reported its numbers and stayed quiet about its assumptions.

The only thing that caught either one was the same dull habit — before you trust an aggregate, check what produced it. Dump the rows. Read the config. Print the path the symlink resolves to. It is never the interesting work, and it is most of the job. A model that scores 47 kilometers and a model that scores 0.8 can be the same model graded two ways, and the gap between "ship a fix" and "burn a week on no real problem" is whether you looked.

We fixed the symlink, bumped its default to the shipped weights, and left a louder note, since the script's own comments admit this drift cost an eval shift once before. It will drift again the first time someone forgets to bump it. That's the lesson, and the unsatisfying one: the bug is rarely in the model. It's in what you handed the eval, and the eval will not volunteer it.

from geopy.geocoders import Nominatim

geo = Nominatim(domain="localhost:8080", scheme="http")
geo.geocode("1600 Pennsylvania Ave NW, Washington DC", addressdetails=True)

That query returns the rooftop, with the house number, the road, and a country code. It also returns an annotations block that Nominatim does not provide: timezone, calling code, and the coordinate in six formats. OpenCage charges for the same block. The US results looked good, so we tried European addresses.

It put Warsaw in Indiana.

Warsaw, Poland came back at 41.2, -85.8. That point is Warsaw, Indiana, across an ocean from the Warsaw in Poland with one-point-eight million people. Berlin went to New Hampshire. Vienna went to Virginia. Sydney went to Nova Scotia, in Canada and outside the United States altogether. Every city we tried resolved with confidence to a smaller namesake in North America.

Which Berlin? covered this kind of collision before. Bare city names repeat across borders, and a later stage has to decide which place you meant. This time was different, because we had built a system that chose the American one every time.

The line that hardcoded America​

The cause is one line. When we wire the geocoder, there's a country fallback:

const defaultCountry = resolveCandidateDBPath() ? undefined : "US"

The line looks harmless: "If the ranking database is not loaded, assume the United States." It was harmless at one point, because the model could not route countries on its own, so defaulting to the US was the best available guess for a US-centric demo.

The problem is what defaultCountry does downstream. It is a hard override that takes precedence over every other signal, including the model's own reading of the address. Its name suggests a fallback, but the resolver treats it as final. We have a small classifier whose job is to read "Warsaw, Poland" and output Poland, and it does so correctly and with confidence. The override discarded that answer and substituted "US" before the resolver saw it. We had trained a model to read the country from the address and then ignored its output.

The fix was to delete the line. The model already produced the right country, so the fix was to stop overriding it. Without the override, the country written in the address reaches the resolver. Berlin lands in Germany and Madrid lands in Spain. The five European countries our resolver covers well resolved correctly, and our drop-in went from resolving four of ten test cities to nine of ten, with no change to the American ones.

That fix is correct, and it has shipped.

Four cities that won't move​

The tenth city was Vienna, and closer inspection found three more like it. With the override deleted, four cities still resolve to the wrong country: Vienna to Virginia, Sydney to Nova Scotia, Warsaw to Indiana, and Toronto to Ohio.

These errors come from the name collision itself, with no override involved. The resolver ranks candidates by population, and for most cities that is enough, because no other Tokyo is big enough to outrank Tokyo. For Vienna, Austria to win, it has to outrank a Vienna the resolver reaches first. Without a country weight in the ranking, the ranker never asks which Vienna is biggest. The model assigns the country tag Austria, but on this query the resolver never receives that signal as a binding constraint.

A manual workaround exists today, the same one Nominatim offers. A caller can set countrycodes=au to make Sydney land in Australia, because the resolver honors the restriction. That fixes the simpler collisions. It does not fix the harder ones: Warsaw plus a Poland filter returns no place, because the gazetteer files the record under Warszawa and the English exonym matches no Polish place. That is a separate problem with a separate fix, and we filed it separately.

At this point we had a one-line cause, a one-line fix, and a known residual with a workaround. We wanted to measure the residual: how much of the world's error comes from the model failing to route the country, and how much from other causes? We built a measurement that geocodes each country's most populous cities as bare names and scores them by distance from the truth, across one hundred and eighty-seven countries.

The measurement was wrong, because we did not check which configuration it ran on.

The database we never switched on​

The first sign was a contradiction in our own notes. The breakdown classified Poland as coverage-absence, meaning the gazetteer had no record at all. Two hours earlier, while investigating the exonym, we had shown by hand that Warszawa, Poland resolves to 52.2, 21.0, exactly right. The record exists. We had a report saying "no Warsaw" next to a transcript that resolved Warsaw.

Both could not be true, so one was measuring the wrong thing, and it was the report. The whole sweep had run with the candidate ranking database switched off. The same resolveCandidateDBPath() ? … : "US" branch from the top of this post applied. This time the missing database did more than change a fallback. It changed which resolver answered every non-US query. We had measured the world on a code path that was never meant to serve it and then recorded the numbers as if they were final.

With the database switched on, the results changed substantially. The sweep had called London a coverage gap. London now resolves two kilometers from the truth, and Beijing does too. Those cities were present all along, in the index we had not loaded.

We retracted the numbers. The report now has a caveat at the top, the two issues we had filed based on it have correction notes, and the retrospective states that the percentages were measured on the wrong configuration. We want this mistake on the record. It is the same kind of mistake as Which Berlin?, a metric grading the wrong thing, one layer lower. In that post, the number was computed correctly but measured the wrong quantity. Here the number measured the right quantity on the wrong configuration. The same practice catches both: before trusting a measurement of the engine, fix and state the data and configuration it ran on. We caught this later than we would have liked, and only because we had written down the contradicting result.

Where this leaves us​

Here is where things stand.

The drop-in works and is shipping. You can point your Nominatim client at it, keep your code, and drop the database stack. The America-by-default bug is fixed: the model routes the country it reads, and countrycodes is available when you want to force one. Those results hold regardless of configuration.

The namesake gap is also real, and it remained after the correction. With the ranking database on, Vienna, Sydney, Warsaw, and Toronto still resolve to the wrong country, because population-first ranking cannot break a same-name tie without a prior on the country. We re-checked this finding with the database loaded, because of the earlier mistake, and verified those four wrong-country results. The model work must improve confident predictions for more countries and make the resolver treat those predictions as binding.

We do not yet know the size of the remaining error categories, such as exonyms like Warsaw versus Warszawa or missing coverage. The only meaningful way to measure them is on the production configuration. We must run that measurement on the production configuration before estimating the category sizes.

Nominatim covers the whole planet on community data and sets the standard we are trying to meet. We can now answer its queries from a file, and on much of the map we answer them well. We also learned that grading a geocoder on the wrong configuration, and forgetting that you did, produces misleading results. We will keep grading on the coordinate and will record the configuration each grade ran on.

That's the gap we set out to close. Every geocoder chases accuracy, be-right-more-frequently, and so do we. The piece that few geocoders hand you is the one underneath it: a number on each answer that tells you which ones to trust, so you can keep the good ones and send the rest to a human.

The forecaster's interface​

Think about a weather forecaster. A good one who forecasts "70% chance of rain" is right on about seven of every ten days they forecast it. That's the whole interface: the number means what it states. A neural model makes the same promise implicitly every time it hands you a softmax probability, and for a long while ours without output broke it, confident in some bands and timid in others, with no single number you could lean on.

We fixed that part first (it's its own story). The short version: a small post-hoc correction maps the model's raw confidence onto a calibrated one, so when a parsed span reads conf="0.94" it's right about 94% of the time, measured on addresses the fit never saw. When the model reports high confidence, you can now believe it.

A calibrated number per span still isn't a number you can act on per answer, though. So we extended it the last step, to the resolved coordinate.

Set a precision threshold​

mailwoman returns a coordinate and a confidence you can put a threshold on. Set a bar (call it τ) and accept only the answers the model is at least that sure about. Route everything below it to review.

On a deliberately messy held-out set (lowercased, abbreviated, postcodes stripped, the real-world soup), graded by whether the resolved point lands within 25 km of the truth, the dial does what you'd want:

Raise the bar and the precision of what you keep climbs from 84% to 97%. The price is coverage: you answer fewer rows, and you pay it knowingly, because the number is direct about where it stands. A search index returns one best guess and one confidence. That gives you no dial at all. You take what it gives and hope.

The part that makes this trustworthy rather than a nice-looking curve is that it holds on data we didn't draw the curve on. Split the messy set in half, fit no new parameters, and the high-confidence answers still beat the low-confidence ones out-of-sample by fourteen points. The model is most unsure exactly where it ought to be: the locales whose coverage we're still building. It has some sense of what it doesn't know.

What you do with it​

Go back to the hundred thousand records. With a threshold you can finally split them in two: the answers the model stands behind and the ones it's hedging on. You auto-accept the first group and move past it. A person looks at the second. The threshold means what you set it to. Set it at 0.95 and your auto-accepted set is right about that frequently, a rate you can put in a report and defend rather than a hope you re-check by hand.

That's the difference between a vibe and a number. A pin with a vibe attached is a thing you re-litigate on every row. A pin with a calibrated confidence is a thing you build a pipeline around.

It's a change rather than a wand. Crank the threshold and you hand more rows to humans. That's the cost, stated directly, and it's yours to set against your own tolerance for a wrong match. What we've done is make the trade legible and put a real number on both sides of it.

It's live now in mailwoman@4.14.0. Pass a calibrator to the parser, read the conf= off the resolved tree, pick your bar. Then stop guessing which answers to trust, and start knowing.

A loss to Nominatim would have been acceptable, because Nominatim covers the whole planet on community-contributed data and is the standard to beat. We could not explain the loss, though. We knew our parser was not ten points worse in Europe than in America, so we investigated what the gap consisted of.

This post covers the cause, the fixes, and the problems the fixes did not solve. For the European comparison we added a third system: Pelias, through geocode.earth. geocode.earth hosts Pelias on Elasticsearch. Pelias turned out to be the stronger competitor, and this post states where it still beats us.

How we kept score​

We graded on where the address lands on Earth, as we argue geocoders should be graded, and ignored whether the model's labels matched ours. OpenAddresses ships a truth coordinate with every row. For each address we parse it, resolve it to a point, and measure the great-circle distance to the true location. A result within 25 kilometers counts as correctly placed, and a no-result counts as a miss, the same as a wrong answer. Each system gets one number on the same real addresses, and each gets the same country hint.

This standard is stricter than the one we had been using internally. Our internal "resolve rate" counted any resolution as a success, including results that landed a region away. Re-graded at 25 km, the internal metric had overstated our European results by fifteen to twenty points. The benchmark's first effect was to correct our own scoreboard, and we now grade the coordinate.

The US, briefly — because it's why Europe stung​

In the US, mailwoman correctly places 99% of these addresses at 25 km, and Nominatim places 84%. Almost all of the difference is no-result: ours is near zero, and Nominatim's is about one in six. That difference reflects our purpose-built data, with TIGER and a fifty-state rooftop layer covering areas where community OSM data is still incomplete.

We get that result from a 30 MB model that runs in your browser, without Elasticsearch or a server round-trip. The same engine, deployed the same way, ran on Europe. That is why a double-digit loss there was confusing.

Europe: we weren't wrong, we were silent​

Graded the same way across six European locales, mailwoman started at about 66%. Nominatim scored 78%. Pelias uses Elasticsearch and a mix of sources, and it scored 89%. We were behind both. The obvious explanation is that the parser is worse in Europe, but the data did not support it. When mailwoman does resolve a European address, the point is within a kilometer or two of the truth. Almost the whole gap came from the other failure mode: more than one in four messy European addresses came back empty. The model did not place them wrong. It returned no place at all.

Empty results are easier to fix than wrong ones. We investigated why the results were empty and found two separate causes.

Why it went quiet, part one: a word it mispronounced​

The first cause is fragmentation. The subword tokenizer splits a town like Grudziądz at the accented ą into Grudzi + dz. Neither half is a place, so neither resolves, and the whole address fails. The full town name was present in the input the entire time. The tokenizer split a word that it had received intact.

The unresolved results include Polish, Czech, and Portuguese towns that the model split at a diacritic and then could not find.

Why it went quiet, part two: postcodes we never had​

The second cause was missing data. We had no postcodes for Poland and Czechia. WhosOnFirst, our gazetteer's main source, has rich postcode coverage for some countries and none for others, and PL and CZ had zero rows. A Polish address whose town the model did read correctly could still fail, because the postcode it could have resolved by was absent from the database.

The fixes: read the word, and fill the gap​

For the first cause, when a parse resolves no place, the resolver now returns to the raw text, using the actual characters with diacritics intact instead of the model's subwords. It enumerates the whitespace tokens, matches each against the same-country gazetteer, and takes the longest exact match because a specific town name holds more information than an ambiguous prefix. Before trusting it, we checked that it recovered real coordinates. The recovered towns land about 1.8 km from the truth, and not in a place with the same name elsewhere in the country.

For the second cause, we imported the missing postcodes from GeoNames: about 73,000 for Poland and 15,000 for Czechia, each with its own centroid and an open license. We added them directly to the gazetteer without a model change or retrain.

We did not expect the next result. The two fixes mostly solve the same cases by different routes. Where we now have the postcode, the address resolves directly and word recovery has no work to do. Where we lack it, recovery resolves the address. Graded the same way, the postcode fill by itself raised Europe from 66% to 84%. Word recovery by itself also raised it from 66% to 84%. Together they reached 88%, and the empty-result rate fell from more than one in four to under one in twelve.

The third fix: stop being confidently wrong​

At 88%, a set of misses remained, and these were wrong results instead of empty ones. "06260 Saint-Pierre, La Placette 7" came back 617 kilometers from the truth. Saint-Pierre is one of the most common town names in France, and we had picked the wrong one. The postcode, 06260, had resolved correctly to the Alpes-Maritimes, where the address is. The coordinate picker trusted the town name and never checked it against the postcode in the same parse, so it discarded the correct answer.

We added that check. When a town lands far from its own postcode, the picker chooses the town of that name nearest the postcode instead. If none is close, it falls back to the postcode's own point. This applies the consistency check from recovery to the main path, without a model change. It raised Europe from 88% to 94%. After we extended the GeoNames postcode fill to Portugal, Austria, and Australia, it also raised Australia from 35% to 65%, because a resolved postcode gave the disambiguation a reference point.

The direct part​

We have been wrong before by declaring success early, so here are the numbers without interpretation. Graded the same way on the European set, all three systems side by side: mailwoman 94%, Nominatim 78%, and Pelias 89%. We now lead both, by five points over a hosted Elasticsearch stack and sixteen over Nominatim, with a 30 MB model running in a browser tab. We lead both systems in Italy, Portugal, France, and Czechia.

Across all seven locales including Australia, the results are 90% for us, 81% Nominatim, and 88% Pelias. We still lead, but the margin is thin, and Australia accounts for all of the narrowing: we score 65% there to Nominatim's 97. We have now worked out that Australia is a parsing problem. Australian addresses in this set lead with the postcode ("3053 Carlton, Barry Street 50"). Our model, trained on American and European formats where the number comes first, reads the leading 3053 as a house number and mislabels everything after it. Given the same address reordered in the American format, it parses correctly, and 65% becomes 87%. The fix belongs in the model: we will train it on the Australian format using the national address file. Australia remains unfixed. We know the cause, and it lies in the parser. The reordered addresses resolve with the existing gazetteer, so the gazetteer needs no change.

There is also a precision cost. At 25 km we're competitive, and at one kilometer we are not. We resolve to admin and postcode centroids, so we place the area correctly, within a couple of kilometers, where Nominatim and Pelias place a rooftop pin. The headline result therefore means the right neighborhood and not the right building. A neighborhood-level result answers an area-level question. A rooftop result requires building-level data.

Word recovery stays off by default, because about one in six of the newly recovered addresses lands more than 25 km away when recovery matched the wrong place with that name. Where we have postcode coverage, a consistency check rejects those before they are returned. The GeoNames fill means that check now runs for Poland and Czechia, where a month ago it could not. Australia is where it still cannot run. A batch pipeline therefore leaves recovery off, because a wrong coordinate corrupts a join without any visible error. The demo turns recovery on, because in an interactive tool a visible, correctable result is more useful than no result. Every recovered place is tagged with whether the postcode check ran, so each caller can decide how to treat it.

The thing a scoreboard can't show​

A single resolve rate also leaves out confidence. A search index returns a result without saying how much to trust it. Mailwoman's confidence is calibrated: a span it marks 90% is correct about 90% of the time, measured on held-out data. You can build a rule on that number, such as auto-accepting above a threshold and routing the rest to a human reviewer, and the threshold behaves as you set it. The recovered-but-unverified coordinates above are flagged separately and are not reported as calibrated probabilities, because the calibrated number is only useful if it stays accurate.

Here is where we stand. Pelias is ahead of us across the full panel, and it places rooftops where we place neighborhoods. In Europe we beat Nominatim by a wide margin and matched Pelias, using 30 megabytes in a browser, without Elasticsearch, and with a calibrated confidence number. We still trail on parts of the map, and Australia is unsolved. We mostly closed a double-digit gap by reading town names that the tokenizer had been splitting and by adding postcodes we should have shipped from the start. We will keep grading on the coordinate.

That metric is measuring the wrong thing. The moment we measured the right one, the panic fell apart.

We've told this story once before, about a resolver that scored 93.7% on a name-match metric while landing 326 kilometers from the truth. We thought we'd learned the lesson — grade the coordinate rather than the word — and filed it under "things we now know." This is that same lesson knocking again, in a new accent, and catching us flat-footed in a new way. We are writing it up because the second time you fall for a trick is the one that should embarrass you into never falling for it again.

A test that grades the words rather than the place​

Here is the number that started it. We took 120 real Spanish addresses — pulled from OpenAddresses rather than invented — and ran them through the model, then scored its component labels against the truth: did it tag the street as street, the city as locality, and so on. Macro-F1 came back at 28.5%. Street by itself: 24.9%. By that yardstick, the model can barely read Spanish.

You can talk yourself into a panic from a number like that. We almost did.

The trouble is what the number is counting. A Spanish street is Calle Mayor, Avenida de la Constitución — a type word, then a name. Our gold tags the whole span as the street, and the strict match wants the model's span to line up with it exactly. It mostly doesn't, and we wanted to know why: is the model directly splitting the type word off — calling Calle a prefix and Mayor the street, a harmless tagging-convention difference — or is it in fact losing the street? Therefore, we re-graded against a bare-name gold, Mayor with the type word stripped from both sides. If the model were splitting Calle off, that should reward it. It did the opposite: street-F1 fell from 25% to 2%. The model's span matches the whole Calle Mayor far more frequently than the bare name — so it's keeping the type word in rather than splitting it off. Its boundaries only don't sit where our gold puts them, and the strict label match scores every such mismatch a flat zero, whether the address was understood or mangled.

A test like that can only tell you whether the model used your exact words. It makes no statement about whether it found the building.

So we graded the coordinate​

The direct question isn't "did it use my labels." It's "did the address land in the right place on Earth." OpenAddresses ships a truth coordinate with every row, so we can ask it directly: parse the address, resolve it to a point, measure the great-circle distance to where it really is. We built held-out sets of 150 real addresses each across the locales where we have that coordinate truth, and graded the production model on the metric we ship.

One subtlety changes the interpretation, so it gets its own column. We separate how frequently the model produces a resolvable answer from how good that answer is when it does. Lump them together and a locale that fails half the time but nails the other half reads as mediocre everywhere. That reading is exactly wrong. Pull them apart and the model tells you something useful.

Spain isn't in that table — its open-address data is cadastral, with no clean coordinate to grade against, so for Spanish we can only score labels. We can't put a Spanish address on the map in this test and measure it directly, and we won't pretend otherwise. What we can do is take the exact street-boundary quirk that tanked the Spanish label score and watch what it costs next door, in the Romance-language locales we can measure. The answer is: no measurable cost. Where the model resolves an Italian or Portuguese address at all, it lands within a few kilometers — city grain, sometimes the rooftop. Luxembourg comes back at 300 meters. The model gets the locality and the postcode right frequently enough to find the right town while it draws the street boundary somewhere the label test hates. The 28% was never measuring geocoding. It was measuring vocabulary — and the redemption holds everywhere we have a coordinate to check it against. That is the strongest claim the data lets us make for Spanish without Spanish coordinates.

The gap is how frequently it answers rather than how well​

So that's the good half, and we'll take it. The direct half is the left column, and it's where the real work is.

Look down the resolves rate and a pattern falls out that is unrelated to accuracy and is entirely about us. France and Italy — the two locales we've poured training data into — resolve about 80% of the time. Everywhere we haven't, the rate drops by a third or more: Portugal, Poland, and Austria resolve around half the time, and Czechia resolves at 43. Australia resolves at 28 and misplaces what little it does resolve. We've barely touched Australia, and its own town names collide across states. The model isn't worse at understanding those addresses. It's worse at producing an answer for them at all, and the cliff lines up exactly with where our corpus stops.

That's the uncomfortable, useful shape of it. The wall isn't precision — when the model commits to an answer abroad, the answer is good. One asterisk applies, and it's a candid one: those are the answers the model chose to give, and the addresses it drops skew toward the harder ones, so read the resolved distances as a ceiling rather than an average. The wall is recall, and recall is a map of our own training data drawn in someone else's country.

Not the gazetteer's fault​

There's a comfortable version of this story where the model parses everything correctly and the gazetteer is missing half of Poland, and the fix is a data import that reflects badly on no one. We checked, because that's the version we'd have preferred. It isn't true.

When a Portuguese address fails to resolve, it's not because the town is absent from our index. We folded the whole world's place names into the index months ago, and a spot check confirmed every Polish and Portuguese locality GeoNames knows is already there. It's that the model never directly emits the locality in the first place. The label test agrees, for once: Portuguese locality-recall sits at 39%, right alongside its 52% resolve rate. If the gazetteer were the bottleneck, the model would extract the town and the index would fail to find it. Instead the model and the resolver fail together, in lockstep. The places are in the index. The model doesn't reliably ask for them. We ran that check directly for Portuguese. Across the rest of the mid-tier the same shape repeats: low resolve, low locality-recall, and towns that the spot check found in the index. That pattern is what you'd expect if the cause is the same one.

Which tells us what not to do. There's no clever rule, no Calle-handling special case, that fixes this — bolt on a Spanish street splitter and you've taught the parser one convention by hand and learned no general rule for the next forty. The model didn't fail because it lacked a rule. It failed because it hasn't seen enough of the place. The change is the training distribution, and we now have a coordinate-graded yardstick that will tell us, locale by locale, whether feeding it more moves the recall it's supposed to.

What we're keeping​

The lesson is the same one as last time. We had it, written down, and the label metric still walked us to the edge of the wrong conclusion before the coordinate pulled us back. Grade the thing you ship. Distance to the truth can't be talked out of, can't be gamed by matching a word, and doesn't care which language the street type is in. Every other metric is a proxy, and proxies flatter you in the languages you trained on and slander you in the ones you didn't.

And then there's the quieter half, the one the two columns made impossible to ignore: a model can be right and unavailable at the same time, and if you only measure one number you'll never know which problem you have. Pull recall apart from precision before you decide what's broken. Ours turned out to be accurate everywhere it answers — and the whole job now is getting it to answer more frequently, in more places, by showing it more of the world.

This gap did not come from a single dramatic mistake. It built up without any error, the way most data gaps do, and the most useful part of this post is the audit that finally exposed it.

A missing place never files a complaint​

Coverage gaps are hard to see because a missing place does not raise an error. When a parser mislabels a token, the wrong field shows up in a diff. When a resolver picks the wrong Springfield, the pin lands in the wrong state and a test fails. When a place is absent from the gazetteer, the system behaves exactly as it does for a typo or a fictional street: it returns no result. Afghanistan returning no result looks identical to "Hgsdfgh Street" returning no result.

You can run the eval suite, see it pass, ship, and announce world coverage while every number you report is accurate. The US accuracy is real, and so is the German coordinate error. Those numbers are measured against the places that are in the file, so the file determines what gets measured. We were grading ourselves on the 97 countries we had, and the missing countries never entered the average.

This applies to anyone building on open geodata: your coverage is your sources' coverage, and you inherit its gaps without any warning. We build the gazetteer from Who's On First plus Overture's administrative divisions. Both are high-quality sources, and we would choose them again. High quality does not mean complete, though, and the admin DB built from those two sources contains localities for 97 countries. We never decided to drop Libya. It was never in the data, and no stage of the pipeline was designed to detect the absence.

The trick: audit against a yardstick you didn't build​

Your own data cannot reveal gaps that are absent from it. Compare it against an independent source and check where the two disagree.

We did the cheapest version of that comparison. GeoNames publishes cities15000, a flat file of every populated place on Earth with more than 15,000 people. We already had it on disk. For each country, we counted the localities in our gazetteer and the cities in GeoNames and printed the ratio. The audit took about a minute to write. Most countries had reasonable ratios, and a long column of countries had zero:

country  ours  GeoNames cities>15k
AF       0     54
HK       0     141
KP       0     97
CD       0     114
AL       0     25
...

Hong Kong has a hundred and forty-one cities in the reference and zero in ours. Model settings cannot fill a gap of that size. We checked whether the candidate build was filtering out data that existed, and it was not: the source admin DB itself stops at 97 countries. The gap was real and came from upstream.

This external comparison was the most productive hour of the whole effort, and it was simple. It needed no model or new metric. It needed a comparison of our file against someone else's count. The main recommendation of this post is to compare your coverage against a reference you did not build, along whatever dimension matters to you. A gap you cannot see still limits your results.

A data update fills the gap without retraining​

A pure data gap needs only a data fix. It needs no GPU, model, or retrain, because GeoNames already has the places, with coordinates and per-language names, for every country we were missing. We extended our gap-fill builder to register country codes it had not seen before and ran it on the full allCountries dump, keeping only current populated places and dropping historical and abandoned ones.

The result is 4.6 million town-level localities across 248 countries, up from 97. We also checked a handful of major cities in the previously empty countries because each result must place the city correctly. Kabul resolves to Kabul within a few hundred meters. Hong Kong, Tirana, Tripoli, Sarajevo, and Yerevan also resolve correctly. All nine checked cities landed within thirty kilometers, and most were exact. A spot-check of small towns placed them inside the correct country's bounding box two hundred times out of two hundred.

The cost​

The change has a cost. Full town-level depth grows the gazetteer from 811 MB to 1.26 GB. That size costs little on a server but matters in the browser, where the demo streams the database a few kilobytes at a time over byte-range requests, and a file about 50% larger means about 50% more data to range over. The fix is therefore two builds. The browser gets the major cities. That build is light and covers most of what people type. The CLI and server get the full town depth, where storage is cheap. One builder produces both depths, and you choose which artifact gets which.

A place resolving is also different from a full address parsing. With Kandahar in the gazetteer, the resolver now places the word "Kandahar" in Afghanistan instead of returning no result. Whether our parser reliably extracts "Kandahar" from a raw Afghan address string is a separate question for a separate post. Coverage was a prerequisite, and this work completed it.

Compare your data against a reference you did not build, count the disagreements, and look closely at the zeros. Our gap passed every test for weeks before this audit found it. Your data probably has one too.

So the questions for the day: why does looking up one name cost 243 round trips? What goes wrong when you search a database you can only read a byte range at a time? And how do you get a global gazetteer — every country, region, county, and city we resolve against — down to about a dozen reads without putting a server back in the loop?

Range requests expose database lookup costs​

The constraint explains the design. The admin database is 2.6 GB. No user will download 2.6 GB to type "Chicago" into a search box. What makes this work at all is sql.js-httpvfs: a SQLite build whose file-reading layer, instead of reading from disk, issues HTTP Range requests against the file on the CDN. SQLite asks for "bytes 4,210,688 through 4,276,224," the VFS turns that into a range GET, the CDN hands back that one 64 KB chunk, and the query engine continues as if the file were local. You're reading a multi-gigabyte database through a straw, and most of the time you never touch 99.99% of it.

For a B-tree lookup on a primary key, this is close to magic. The engine walks root → interior → leaf, and each hop is one range request. Three or four reads and you're holding your row. The straw is wide enough.

Why search scatters​

The trouble started because we weren't doing a primary-key lookup. We were doing a name search, and to make name search fast inside SQLite you reach for FTS5 — full-text search, an inverted index. For each token it stores a posting list: every row that contains it. Search becomes "go read the posting list for this token," which is exactly the kind of thing a real database does well when the disk is right there.

The disk is not right there. It's at the end of a straw. And an FTS posting list for a common name is not one tidy run of bytes — it's a chain of pages scattered through the file, because the index grew as the data loaded and the postings landed wherever there was room. A client reads one posting list by following that chain, and every hop in the chain is its own range request. A common place name dragged in a long chain. Then the candidate rows themselves lived somewhere else again. The reads piled up: 243 of them, about 16.5 MB, to answer one query.

The maddening part came when we tried to tune it. The VFS lets you widen the straw — ask for bigger chunks per request, fetch 64 KB or 256 KB at a swing. We widened it, expecting fewer round trips. We got more bytes and barely fewer trips, because the postings were scattered, so a bigger chunk mostly fetched neighbors we didn't want. We'd been getting away with it on a "slim" database — a stripped 53 MB build with three countries in it — and the slim build was cheap for the dullest possible reason: it was small, so even a scattered search couldn't scatter very far. That's not an optimization. That's a database too little to misbehave.

Stop searching. Start addressing.​

We talked it over with DeepSeek, and the reframe we kept from that discussion was this: a database you read through a range request isn't a database you should search. It's a database you should address. Don't make the browser hunt for the answer across an index — precompute the answer and put it where one short read can reach it.

Concretely: a single table, one row per name a place can be found by. Each row stores everything the resolver needs already on it — the canonical name, the centroid, the bounding box, the country, and the population rank baked in as a sortable column — so once you've found the row, you're done, with no second trip to go fetch the details. The table is WITHOUT ROWID, which in SQLite means the rows are the B-tree, sorted by the key, and the key is the normalized name. We load it pre-sorted and VACUUM it so a name and its neighbors sit on adjacent pages. Now a lookup is the thing the straw was always good at: walk the tree to the name, read the leaf, and the matches for "springfield" are sitting there in a row, ranked, ready.

One discipline holds the whole thing up: the key has to be normalized the same way when we build the table and when we query it, or the walk lands one page off and finds no entry. So both sides call the same function — normalizeLocalityForKey — by construction rather than by convention. It folds case, strips diacritics so Saint-Étienne keys as saint-etienne, and leaves scripts that aren't Latin unchanged so 北京 stays 北京. Build-side and query-side can't drift, because there's only one of it.

The cost we almost shipped​

Now the catch. The candidate table holds US postcodes and not yet the rest of the world's, the same coverage the old slim build had. So when we first wired it up and typed a Berlin address — 10115 — the demo dropped a pin in Manhattan. 10115 is a real Berlin postcode and a real New York ZIP, the table only knew the New York one, and the cascade had reached for the postcode first and taken the only match it found.

The fix wasn't more postcodes. It was order. Resolve the locality first — "Berlin," which ranks to Germany by population — then let that resolved place check the postcode lookup to its own country. No German 10115 in the table? Fine: fall through to Berlin's own centroid. That centroid is exactly where the address wanted to be. An ambiguous postcode can't drag a German address across the Atlantic anymore, because it never gets to vote before the city does.

What it bought​

A name now resolves in about 12 range requests and under a megabyte, down from 243 and 16.5 MB. The whole table is 490 MB with global coverage — every place we know rather than three countries — in a single file the browser reads a dozen pages of. US locality resolution holds at 96.8%; the European cities we'd added through Overture come back at parity. And the slim build — the per-model-version, three-country database we'd been maintaining — is gone, along with the chore of deciding which countries to ship.

The result is still supported because the constraint was never going away. When the only way to touch your data is one short read at a time, searching is the expensive verb. So we stopped searching. We built the answer ahead of time and gave it an address, and now the browser goes and reads it.

So, the questions for the morning. How does a database that passes every check still land the answer 600 kilometers wrong? What does it mean to "validate" a thing? And why is Toronto in Ohio?

The build that felt right​

The task was simple enough: add Canada to the map. The obvious change is postcodes — they're precise, they're public, and Overture ships them. I'd done exactly this for a dozen European countries the night before, same tool, same shape: read the country's address points, group by postcode, take a trimmed centroid, write one row per code. Canada came back with 843,000 of them, alphanumeric A1A 1A1 codes that survive the build intact, each pinned to its own block. I spot-checked Toronto's financial district and it sat exactly where it should. I diffed the new gazetteer against the old — US postcodes untouched, every European set intact, 843,000 new Canadian rows. Every number I could check, I checked, and every one was right.

That is the trap. I had checked everything I'd built.

The check that grades the pin​

There's one test in the stack that doesn't care what I built. It loads the actual demo in a real browser, types an address into the box, and reads the coordinate off the map. The pin the user would see. The whole point of it is to grade the assembled answer rather than any single piece of the implementation. Before promoting the new gazetteer I added a Canadian case to it: 100 Queen Street West, Toronto, ON M5H 2N2, and assert the pin lands somewhere in Toronto.

Four cases passed — Chicago, a Berlin postcode, the White House, all green. The fifth came back 40.458, −80.6. I had to look it up. That's Toronto, Ohio: population five thousand, on the bank of the Ohio River, about as far from the address I typed as Madrid is from London.

Why the perfect postcode never got a vote​

The gazetteer has two layers. There's the postcode layer I'd rebuilt, and underneath it the admin layer — the one that records a place's country and population. So I went and counted the admin layer: 1.7 million places, twenty-six countries. The number of Canadian ones is zero. Toronto, Montréal and Vancouver are all missing — Canada isn't in it at all. We'd built the gazetteer out of the European set, the US, and the CJK countries, and no build step had ever folded Canada in.

Now watch what that does. The parser reads "Toronto" off the address and hands it to the resolver. The only Toronto the resolver has ever heard of is the one in Ohio, so that's the city it picks. And then the cleverest piece of the cascade turns around and finishes the job. We added that piece a couple of weeks ago to keep Berlin's 10115 — which is also a perfectly good New York ZIP — from dragging German addresses to Manhattan: resolve the city first, then let that city restrict the postcode to its own country. It's a good rule. Here it's a guillotine. The city resolved to the United States, so the correct Canadian postcode is now "from the wrong country," and out it goes. All 843,000 of my perfect coordinates never get a vote, because the city votes first and the city is in Ohio.

I validated the layer I made rather than the one that ships​

The lesson is an old one. I validated the postcodes. The postcodes were never the problem. I checked the thing I'd built and not the thing the user receives — the coordinate, the pin, the assembled output of every layer working together. Each green check was accurate and beside the point, because not one of them graded the only number that matters.

This keeps happening to us, in different forms. A model's per-tag F1 climbs while the coordinate it produces goes nowhere. A reconcile pass improves a label and breaks the geocode that depended on it. The discipline that survives every version of it is the same: grade the assembled coordinate against the truth rather than the component you happened to touch. The e2e check isn't ceremony. It's the one thing in the stack that reads the pin the user sees, and it's the only thing that caught this.

What it costs, and what it doesn't​

The accounting: the night wasn't wasted. The 843,000 centroids are correct and sitting ready, and the tooling that made them is the same tooling that'll make the next country. But the premise was half a country short. A country, it turns out, is its admin skeleton first — the cities and regions that anchor a name to a place — and a postcode only means something once that skeleton is there to hang it on. Fold Canada's divisions into the gazetteer the way we folded Europe's, and the postcodes I built light up at no added cost. Ship them on their own and all you've done is hand the resolver a stack of right answers to a question it can't ask.

So we didn't promote it. The map still doesn't have Canada — but it knows it doesn't now, and so do we. Build the test that grades the pin your user sees, and run it before you trust the green checkmarks upstream of it. They'll tell you the data is right. The pin tells you whether you are.

That is unusual, because each of the three kinds of product on the market today requires you to give something up.

The three options, and what each one costs you​

The first option is a cloud API: Google, Mapbox, HERE Technologies, AWS, or Azure. You send the address over the network and get coordinates back. The results are fast and globally accurate, and depending on the vendor, they come with usage restrictions. Google's terms let you cache a coordinate for thirty days, after which you must delete it, so if you geocode the same ten thousand delivery addresses every morning, you pay for them every morning. AWS forbids caching autocomplete results and bills storage at a higher tier when you do cache. The address always leaves your infrastructure. That rules these services out when the address itself is sensitive, such as a shelter, a clinic, or a residence whose address is confidential.

The second option is to self-host: Nominatim, Pelias, and Photon are all mature, free software that you control. The license is fine. The cost is the infrastructure. Nominatim needs a Postgres/PostGIS install with more than a terabyte of disk and a planet import that takes days. Pelias is Elasticsearch plus five supporting services and a 450 GB index. Photon is OpenSearch with a 64 GB heap. You own your data, but you have to operate a search cluster to use it, and none of them can run in a browser.

The third option is a standalone parser. libpostal is the open address parser most projects embed, and it works well. It is also a 1.8 GB trained model (2.2 GB for the Senzing variant), written in C without a supported path to the browser, and it only parses. It tells you which token is the street, but it does not tell you where the street is.

The choices are to rent a service and send it the query, host a service and run a cluster, or parse the address and stop before geocoding. Mailwoman is a fourth option.

What if it ran on the device​

The whole pipeline, parsing and then resolving to a coordinate, runs client-side. The parser is a 29.8 MB int8 ONNX model executed by onnxruntime-web. The gazetteer is a single global SQLite file of around 530 MB covering every country we resolve against, and the page never downloads the whole file. It reads the file over HTTP range requests, pulling roughly a dozen small ranges to answer a query. The resolver needs no Elasticsearch, PostGIS, or native module. It uses plain node:sqlite, and the same code runs in Node and in a browser tab.

The resolver used to need 243 round trips. An index change reduced that count to twelve while the fetching stayed the same. A separate post covers that. The result is a global geocoder that answers queries with a few small reads and no server.

A neural parser the size of a photo​

The 29.8 MB model is roughly sixty times smaller than libpostal's default model and about seventy times smaller than the Senzing one. The smaller model is also the better parser on search-box input: typos, half-finished lines, and partial prefixes. A transformer trained on degraded queries reads 123 Penslyvania Av NW the way you meant it. A CRF over hand-built features can miss those inputs. The model is small enough to ship to a browser and performs better on the input browsers produce.

Your queries never leave, and the results are yours forever​

This matters most to anyone holding addresses they cannot send to a third party. Without a server, there is no storage clause, zero data retained, zero deletion after thirty days, and zero leakage. You geocode once and keep the coordinate as long as you like. Healthcare and other workflows with confidential addresses can require on-device processing as a procurement condition.

This argument has limits. geocode.earth's hosted Pelias and Smarty both already offer generous data-ownership terms, so the argument applies mainly to vendors that require deletion: Google, AWS, and HERE Technologies. Against those vendors, local geocoding removes the deletion requirement.

The bill is zero​

Google's geocoder costs five dollars per thousand requests after the free tier. That is five thousand dollars a month at a million addresses, and six figures a year past ten million. On-device, a query has no marginal cost, because it runs on hardware the user already owns. A self-hosted geocoder also has no per-query cost, so this comparison applies to the cloud APIs, where on-device removes both the per-call charge and the server you would otherwise run.

Where it stops​

I would rather you hear the limits from me than discover them in production.

Rooftop precision (the actual building rather than the neighborhood) currently works only where we've shipped per-state address-point data: California, New York, Michigan, and DC. Everywhere else, the resolver returns an administrative or postcode centroid. That centroid gives the right town and ZIP but not the building. The large cloud vendors have much better global rooftop coverage. If you need a rooftop in Lagos today, use Google.

Mapbox does ship on-device geocoding, but only in their native mobile SDK, and it performs reverse geocoding (coordinate to address) rather than forward geocoding. The precise claim is that Mailwoman is the only geocoder that runs the full forward pipeline in a browser without a server. That is a narrow claim, and no other product we know of meets it. There is not yet a batch endpoint or an SLA. Mailwoman is infrastructure you embed, and it is not a hosted service with an on-call team.

In the network tab, you typed an address, got a building, and no other party saw the query.

npm install mailwoman @mailwoman/neural

Or open the demo and check the network tab yourself.

Green is the National Address Database. Orange is OpenAddresses. For New York, OpenAddresses republishes the city's own NYC Open Data. You can explore the interactive version and click any dot to see its publisher.

The split is geography you can see​

The federal release blankets the state. New York City sits in its own color because the city publishes its own address file, and OpenAddresses republishes it. No guess was made about which is which. Every point kept its source, from ingest all the way to this map. The method is a source column that no pipeline stage dropped.

Credit where the data's due​

None of this is our data. The National Address Database is a federal release. OpenAddresses is a decade of county and municipal publishers doing the unglamorous work of putting their address files online. The street network underneath is Census TIGER. They're free, they're open, and they're good. The lookup was never the hard part of geocoding. Each pipeline must assemble these sources, keep them fresh, and preserve their provenance. That record lets a user trace a wrong coordinate to the dataset and release that produced it.

The layers under the dots​

This map shows the rooftop-point sources. Where there's no rooftop point, the geocoder falls back to TIGER street interpolation, a position estimated along the block, and below that to an administrative centroid from Who's On First. Each tier is a different open dataset, and every coordinate records which rung it landed on, so a rooftop in Manhattan and a block-interpolated guess in a thinly-covered county never get mistaken for each other.

See it, build it​

Explore the interactive map — serve it over localhost, since the basemap tiles are CORS-scoped to the house domains. The provenance is a plain column on the address-point table:

SELECT lat, lon, source FROM address_point WHERE locality_norm = 'brooklyn' LIMIT 1;
-- 40.6782 | -73.9442 | overture:OpenAddresses/NY/NYC Open Data

Point the same map at any state:

node \
  scripts/record-matcher/viz/source-provenance-map.ts --state ny

So you do what everyone does: you start a spreadsheet, you sort by name, and you give up around row 400.

This map shows the result of resolving them anyway, scoped to Texas:

Every dot is an entity that resolved across at least two of those datasets. Their identifiers didn't match, because there are none to match. They resolved because the records geocoded to the same place and their names agreed once you normalized them. The resolved location is the key the three publishers never agreed on.

How you join files that share no key​

The trick is to stop matching strings and start matching places. Each record's address goes through the neural parser, resolves to a coordinate and a place in the hierarchy, and gets blocked by where it is. Only then do the names come in, canonicalized — legal-form noise stripped, abbreviations expanded — and scored with label-free Fellegi-Sunter weights (the geocode-first record-matching post walks through this). "Baylor Coll. of Medicine" and "Baylor College of Medicine" stop being different strings; "123 Main St" and "123 Main Street, Ste 400" stop being different places. The geocode does the blocking, the normalized name does the deciding, and neither file had to store the other's ID.

The numbers, and exactly what they are​

219 entities resolved across two or more of these sources. Twenty-eight of them span different agencies — CMS's provider registry, the FCC's funding program, Texas's licensing list — and those are the hard ones, because inside a single agency you can sometimes lean on an internal ID, while across agencies you have only the place and the name. (The other 191 links are FCC-internal: Rural Health Care filings matched back to their own commitment records.) You can see the cross-agency subset on its own in the interactive map. The full set is here.

Then the coverage reconciliation. Of the 2,771 entities carrying an eligibility record — an NPPES organization or a TX HHSC facility — 22 also resolve to a funding record. That's 0.8%, and it's a floor rather than a coverage rate. Imperfect resolution and a capped sample only ever miss links. They never invent them. The set is the deliverable, and the percentage only reports its size.

Read this part before you read anything into it​

This is a set-membership reconciliation rather than a determination. The 2,749 eligibility entities that didn't resolve to a funding record are a candidate set for review, and no more. A missing funding link can mean the entity didn't apply, applied under a name the resolver didn't catch, isn't eligible, or only that its funding record fell outside this capped 2,000-row sample — a sampling artifact rather than a finding. At full scale the set tightens, and it's still only a set of candidates.

What we produce is the reconciled join and the candidate set. What a gap means, and whether it means anything at all, is the data consumer's call rather than the tool's. None of this is an allegation, and the map makes no claim about any provider on it beyond "these records appear to describe the same place."

Run it on your own files​

The map above is the same toMapHTML the record matcher ships, on the house MapLibre and Protomaps stack. The whole thing is one command over a config that tags each file with a role:

mailwoman registry --sources sources.json --reconcile --map-out map.html

Point it at your own pile of registries. The resolved place is the key they forgot to share.

The market offers two kinds of tools for this. Each solves part of the problem, and neither solves all of it.

The two tools, and the halves they skip​

The first tool is a string matcher, such as Splink, Zingg, dedupe, or the fuzzy-match component inside Tamr or Tilores. You give it name and address columns, and it scores how similar the text is. Used as the whole answer, it has a basic flaw: text similarity does not measure place similarity. An address two doors down the same street is one character away. The same building written two different ways can score far apart. Threshold tuning then takes weeks, and a threshold that is too loose in cities is too tight in suburbs. (We covered this in detail in Match where it is rather than how it's spelled. In short, Jyllandsgade 15 and Jyllandsgade 75 score 0.96 similar and sit 650 meters apart, and no threshold handles that.)

Strings are still useful. Names are some of the best evidence available, and later sections argue for relying on them heavily. The limitation is narrower: a string describes how a record is spelled and makes no statement about where it is, so it should not be the field you block on.

The second tool is a join key, such as Placekey, SmartyKey, or our own @mailwoman/address-id. It normalizes the address, geocodes it, and returns a deterministic identifier. Records with the same key are at the same place. This is better than string matching, because it keys on where instead of on spelling. Placekey will tell you that two records are in the same building.

The key does not go further than that, and matching the building solves only the easy eighty percent.

A key gets you to the doormat​

Key vendors say little about the remaining cases. At 1504 Taub Loop in Houston, four separate provider registrations resolve to the same rooftop, and all four are Baylor College of Medicine, one organization registered four times. A deterministic key groups them together, and that grouping is correct. At the next address, a single medical-office building holds a dozen distinct practices with different owners and specialties that share only the street address. The key groups them together too, and that grouping is wrong. The key cannot distinguish the two cases, because it encodes only location.

Placekey's own FAQ states that it "operates as a universal join key rather than a matching service… it doesn't perform record linkage itself." That is an accurate description of what a key does. The key finds records in the same building. The remaining question is whether records in the same building, with different company names, are one entity or two.

A probability answers that question.

The missing half: a number you can read, and the humility to abstain​

Mailwoman is built around this second step. It first resolves every record to a coordinate and a place hierarchy and blocks on where. It then runs a probabilistic scorer over the candidates that share a place. The scorer is Fellegi-Sunter. Every field agreement or disagreement contributes a weight in bits, and the sum gives a calibrated probability that two records are the same entity, with the evidence shown so you can see why. A shared rare street counts for more than a shared common one. A shared rooftop counts for more than a shared interpolated point, because the geocoder reports its confidence to the matcher. The score also includes the string comparison: Jaro-Winkler over the names, where agreement on a rare organization-name token counts heavily and a shared LLC counts for very little. Names still matter, but they now decide who and no longer decide where. That role suits them better. When the probability falls in the uncertain middle, the matcher abstains and routes the pair to a human reviewer.

In summary: Placekey tells you two records are in the same building. Mailwoman tells you, with a calibrated probability and a readable reason, whether they're the same company, and flags the pair for review when it should not guess. That covers the linking step the key vendors omit and the geography the string-only matchers omit, and it keeps name matching in the model.

When the string is the answer, normalize it first​

Sometimes the name decides the match. A comparison of the federal provider registry with the FCC's funding records shows one such case. The same health system appears in one file as BAYLOR COLLEGE OF MEDICINE and in the other as Baylor Coll. of Medicine, as a d/b/a, or with a , LLC that the other publisher dropped. Geography places the two records in the same area, so they are plausibly the same place. The name decides whether they are the same entity, provided it is normalized before comparison.

Normalization therefore runs first. canonicalizeOrganizationName strips legal-form terms (LLC, Inc, a leading The), splits the d/b/a, normalizes punctuation, and uses the ISO 20275 entity-form list so that Coll and College compare as the same word. Then the names are compared, and agreement on the rare token Baylor counts for much more than a shared Medical Center. The same mechanism handles both directions. The building with a dozen distinct practices stays a dozen entities, because they share a coordinate but have different normalized names. One facility registered twice under different operational names merges into one. The coordinate does the blocking, and the normalized name comparison decides the match. Neither step by itself reconciles NPPES and the FCC data. Together they do.

The matcher does not need labels​

Most ML-based matchers also assume you have training data. Zingg asks you to label thirty to fifty pairs to start, dedupe runs an active-learning loop built on labeling, and Tamr expects ongoing human curation. The input may consist of CSV files and a deadline, without labeled pairs.

Mailwoman's scorer estimates its parameters from the data without labels, using expectation-maximization. Splink uses the same label-free Fellegi-Sunter method, and it runs at a scale we have not matched. The difference is that Splink expects you to have already geocoded the addresses, while Mailwoman covers the whole stack from raw string to resolved place to calibrated link. When you do have labels, an optional gradient-boosted scorer captures non-linear interactions that the hand-built weights miss. In our held-out tests it added about six points of pairwise F1 and five of clustering F1, across every random seed we tried.

It travels​

A learned scorer may memorize one dataset's quirks. To test that, we trained the boosted scorer on Texas providers and evaluated it on held-out states it had never seen. It beat the unsupervised baseline by 20.5 points of F1 on California and 21.9 on New York. The pattern it learns, what over-merging looks like, transfers beyond Texas.

Show your ruler​

A result such as "F1 of 68%" is meaningless until you state what it was measured against. The same clusters score from 53.6% to 68.1% against four different definitions of truth. The clusters did not change. Only the ground truth changed, as it defined one entity more precisely. We hand-adjudicated 120 of the hardest Texas pairs (co-located, with similar names and distinct IDs). All 120 were the same real-world organization under different registrations, and none were unrelated companies merged by mistake. We report the ground truth next to the number. Most vendors do not.

Own it, and what it doesn't do​

It runs on your machine. It is open source, runs on your infrastructure, and has no per-record charge. The alternatives include AWS at $0.25 per thousand records (billed on non-matches as well), Senzing starting around $58k a year, and Tamr, Quantexa, and Tilores. Those three all require a quote. This can meet data-control requirements in compliance and healthcare workflows.

The claims above depend on stating these limits. Placekey does implement place-identity matching. Angle four is our response, but the overlap is real. Melissa ships patented coordinate-proximity matching, so it would be false to say no one else uses coordinates. Our narrower claim is that we make the calibrated coordinate the primary evidence in a probabilistic model. Splink offers the label-free baseline at much larger scale. The enterprise incumbents lead on raw volume, multi-domain mastering, support, and decades of name-matching expertise. None of them combines an open, label-free, geocode-first approach with the geocoder and the linker in one auditable stack. That combination is what distinguishes Mailwoman.

Every match the matcher surfaces is a candidate for review, and the final decision is yours. You decide what a correlation means and whether it means anything.

npm install @mailwoman/registry

import { resolveEntities, toMapHTML } from "@mailwoman/registry"

const entities = await resolveEntities(records, {/* sensible defaults */})
// → clustered entities, each with a coordinate, a cohesion score, and the records it merged

resolveEntities runs the whole block → score → cluster pipeline; toMapHTML drops the result on a map so you can see, dataset by dataset, what linked to what.

Our evals did, three times this week. One of them nearly talked us out of a model we should ship. One invented a coverage problem we don't have. And one had us writing "3.3 km" into a model card for a geocoder that puts most addresses within a hundred meters. Each time the fix was the same, and embarrassingly cheap: stop reading the summary row and pull the actual records the summary is averaging over.

One bad row out of twenty-seven​

We were grading a candidate model against a country-homograph set — addresses where the country name is also a US town, like "Lima, Peru" sitting next to "Lima, Ohio." The candidate scored 80.9 where the shipped model scored 83.3. A 2.4-point country regression, right there in the table. That's the kind of number that holds up a promote.

So we dumped the rows and diffed them, model against model. The entire 2.4 points was one address: Avenida Arequipa, Lima 15046, Peru. The candidate kept the street, the locality, and the postcode identical — it dropped the trailing "Peru" entirely. Every other country in the same sentence shape resolved fine. Swap "Peru" for Chile, Bolivia, France, Jordan and the candidate gets them all. The set only has twenty-seven rows with a gold country, so a single flip moves the score by 3.7 points.

A metric that one record can swing by three and a half points isn't measuring a property of the model. It's measuring that record. The "regression" was a rumor, and we'd have let it sway a decision that mattered if we hadn't read the row.

The towns the scorer threw away​

The same week, the same eval told us our rural US locality resolution was a disaster: South Dakota matching 62%, Vermont 31%. That fit a story we already believed — the gazetteer is thin in rural states, the model can't resolve what isn't there — so it was easy to nod and file it under "known coverage gap."

Then we pulled the misses. The resolver was landing the right place on nearly all of them. The problem was the scorer: it only counted a match if the resolved place was tagged locality, and New England-style civil towns are localadmin in our gazetteer rather than locality. "Barre Town," "Saint Albans Town," every Vermont township — the resolver found them, and the metric threw them on the floor for having the wrong placetype label. Credit the group the resolver treats as a locality and Vermont goes from 31% to 93%, South Dakota from 62% to 98%. The model was fine. The data was fine. The ruler was bent.

Kilometers on paper, meters in production​

This is the one that stings. Our flagship accuracy number — the one in the docs, the one in the model card, the one behind every "the US bottleneck is rural coverage" conversation we'd had for weeks — was coordinate error p50 3.3 km, p90 10 km. We read that as the ceiling. A city centroid is legitimately a few kilometers from an edge address, so we'd long since made our peace with it and gone looking for the next gazetteer to ingest.

The eval was resolving to the admin centroid. That's it. It took the parsed address, found the city, and returned the city's center point. But that is not what the geocoder ships. Production runs a cascade — it reads the parsed street, pulls that state's rooftop and interpolation data, and resolves the actual point, only falling back to the centroid when no point data exists. The eval never wired that layer in, so it had been reporting the blunt fallback as if it were the product.

We pointed the eval at the real cascade and ran the same ten thousand addresses. p50 went from 3.3 km to zero. p90 from 10 km to 1 km. 85.9% of US addresses land within a hundred meters, and 90% within a kilometer — 80% on an exact rooftop point, another 8% on a street interpolation, and only the remaining 12% falling back to the centroid we'd been quoting as the headline. We had been underselling our own geocoder by three orders of magnitude, in public, because the eval graded a path the product doesn't take. And the "rural coordinate bottleneck" we'd been planning campaigns around? Same bent ruler. The rural states land on rooftops like everywhere else.

Read the rows​

Three different failures, one fix. A country regression that was a single record. A coverage gap that was a placetype-tag mismatch. A flagship accuracy number that was grading the wrong object entirely. None of them survived contact with the actual rows, and all of them looked authoritative as a number in a table.

There's a tidy version of this lesson — grade the thing your user receives — and it's true, and we keep it on the wall. But the operational habit underneath it is blunter than that. Before a number changes a decision, dump the records it's summarizing and read them. Ask whether there's a pattern or whether it's one row on a small denominator. Ask whether the thing you scored is the thing you ship. It costs ten minutes and a --errors-json flag, and it would have caught all three of these before they reached a card or a roadmap.

The averages lie in both directions. Yesterday it flattered us — a macro-F1 climbed while the model got worse at the most common address there is. This week it sandbagged us, three times, and we almost believed it because a number that reports you're losing feels like honesty. It isn't honesty or dishonesty. It's an abstraction, and abstractions drop exactly the detail you needed. The rows still have it. Go read the rows.

It wasn't, and it took three probes to prove it. The same model that scored higher on the average had gotten worse at the most basic address there is — a town and a state. This post describes how the average lied to us, and how we caught it.

What we were trying to fix​

Mailwoman's biggest measured weakness is boundary instability: the model wobbling on where one part of an address ends and the next begins. A street suffix swallowed into the street name. A French house number that trails the street instead of leading it. The kind of thing that's invisible on a clean "123 Main St, Springfield, IL 62701" and falls apart the moment the punctuation goes missing.

So we built a synthetic training extract that puts the gold boundary on exactly those hard cases, mixed it in at a conservative weight, and retrained. It worked — on the boundaries. French street-prefix recognition went from a coin flip (55%) to essentially perfect (99.7%). Every boundary shape moved up. The macro-F1 rose. We had the release notes half-written.

The catch​

We don't ship on the macro. We ship on a battery of per-locale floors ("don't regress US locality below 72.9," that sort of thing), graded against the assembled output: the actual parse a user would get rather than the per-token score the trainer optimizes.

One floor came back red. US locality: 66.2, floor 72.9. The retrain that aced the boundaries had dropped six and a half points on locality, on real held-out US addresses, while the aggregate went up. The gains were concentrated where we'd added training data. The loss was spread across everything else, and the average averaged it away.

This is the trap we've been burned by before, and it has a name on this project: grade the assembled coordinate, never the label-F1. We wrote it on the wall. We still almost walked past it.

Three probes, one culprit​

A failing floor tells you that something regressed rather than what. So we wrote probes.

The first asked a simple question of every boundary the model got wrong: is it confidently wrong, or is it uncertain? If the model spreads its probability mass and shrugs, that's a capacity ceiling — you need a bigger model, and more data is wasted GPU. If it commits hard to the wrong answer, that's a signal problem — more of the right data can flip it. Every failure came back confident. Good: this is fixable with data.

The second probe went after the locality drop directly. It took the model before the retrain and the model after, ran both over 1,616 real US addresses, and isolated every row where locality went from right to wrong. Then it asked where the locality text went.

We expected the street to be eating it. It wasn't. That was 3% of the damage. 84% of the regression was the model dropping the locality entirely on short, bare rows: "LaMoure, ND." "Frannie, Wyoming." "Buckhorn, Wyoming." Each row held only a town and a state, and the model now returned no locality tag for the town.

The mechanism, once you see it, is almost funny. Every example in our hard-case extract had a street before the city — that's what made the boundary hard. So the model learned, a little too well, that a city is the thing that comes after a street. Take the street away, hand it a bare "LaMoure, ND," and it no longer recognized LaMoure as a place at all. We didn't teach it a new skill so much as we taught it a superstition.

The lesson we keep re-learning​

The boundary extract's labels were sound. The trouble was that it only ever showed them in one shape. Every row was a clean, full, structured address, so the model over-fit that shape and fell off the moment the input stopped matching it. The fix is to surround the hard cases with the ordinary ones they need to be told apart from. Teach the bare "City, STATE" next to the full address. Teach the house number before the street as well as after, so position stops being a shortcut the model can lean on.

But the deeper lesson is the one about the number we started with. The macro-F1 went up. The model got worse at its most common real-world input. Both of those things were true at the same time, and only one of them mattered. If you're tuning a model on an aggregate score, the aggregate is the first thing that will lie to you — and it will do it while pointing at a bigger number and smiling. Grade the thing your user receives. We had to be told twice.

Two separate things went wrong with that number, and fixing one wouldn't have caught the other. The first is the averaging. A gain on rare shapes and a small loss on the common one net out positive, so the loss disappears into the mean. The fix is to stop reading the mean and start reading each stratum on its own, US locality by itself rather than folded into a global score. The second is the one that nearly shipped the model. We were grading the wrong object: the trainer scores a label on every word, but the user receives an assembled parse, and the two can move in opposite directions. You can label more words correctly and still hand back a worse address, because one wrong word in the wrong place reshuffles the structure around it. Stratify the word-labels as finely as you like and you're still grading the wrong product. So we grade the finished parse against what a real user would hand us, and we read it stratum by stratum. Both halves, every time.

Coda: did the fix work?​

Yes, on the labels — and in a way that drove the lesson home twice. We built exactly what the probe pointed at: teach the model bare "City, STATE" rows, so it stops needing a street to recognize a town. Held-out locality F1 climbed from 66 back up to 80, past where it started before any of this. The macro-F1? It barely twitched.

That's the whole point, restated from the other direction. The aggregate that almost fooled us into shipping the broken model was the same aggregate that wouldn't have told us the fix landed, either. Both times, the truth was in the held-out core (the town, the street, the house number a real user hands us), and both times the averageshrugged. So we read the core.

And then the lesson came back for a third helping. Before promoting the new model we graded the one thing we hadn't: the assembled coordinate, the actual point on the map a user receives, measured against the model already in production. It was flat. Locality F1 had climbed, but locality-match on real addresses and the median coordinate error did not move. the result never reached the map. We had done the very thing this post opens by warning against, one level down: graded the assembled labels and called it a win without grading the assembled coordinate. So we left the production model where it was.

The flatness pointed somewhere we hadn't been looking. The addresses still wrong on the map are small rural towns the gazetteer itself resolves poorly (a third of Vermont's, in one sample), places no amount of model training reaches. For now the model has caught up to its own database, and the result is in the atlas. That's a different post, and we'll grade it on the coordinate.

One footnote, because "add more data" and "train a bigger model" aren't the only two endings a stuck case has. When a change refuses to move, there are three reasons, and they ask for different things. Maybe the model picked up a bad habit from the data. Better data breaks that habit, as the sections above show. Maybe it's a thirty-million-parameter model out of room. Then only a bigger one helps. Or maybe the answer was never in the text: "Paris," on its own, is Paris, France or Paris, Texas, and no amount of data or parameters settles that, because the string doesn't contain the answer. The cheap way to tell the second reason from the third is to ask whether a person could do it from the very same input. Our four stubborn boundary cases are the second kind. A person segments a comma-less "City STATE" without trouble, so that's capacity rather than confusion. We banked the result and left the boundaries for a bigger model. The third kind was never the model's job in the first place. It tags "Paris" as a place, and the gazetteer decides which Paris, leaning on the rest of the address and, when there's no other evidence to go on, on which one more people mean. A team can choose the right fix only after it distinguishes these three causes. That's a different post.

We call the first one v0. It is our TypeScript port of the Pelias parser — a rules engine: tokenize, classify each token against dictionaries and patterns, solve for the most plausible arrangement under a pile of hand-written constraints. It is fast, deterministic, and the product of years of accumulated postal wisdom. It is also the thing we set out to beat.

The second is the neural classifier — a sequence labeler trained on a BIO-tagged corpus, the one this blog has spent most of its life arguing with.

So: a year in, did the neural net beat the rules parser? The answer is mostly a tie, until the address gets weird — and the weird is where it gets interesting.

The scoreboard​

We don't grade the two parsers against each other's labels — thatmeasures which one looks more like the other. We grade both through the same resolver, against real address points: take 2,000 OpenAddresses US rows the model never trained on, parse each one, resolve it to a coordinate, and measure how frequently we land in the right city, the right state, and how far the resolved point is from the truth.

On clean US addresses it is close. Neural is a couple of points ahead on locality, a hair ahead on region, dead even at the median coordinate — and the difference you can feel is in the tail: neural's p99 error is 239 km versus 277. Both parsers nail the easy ones. The neural net throws fewer wild misses.

(Pelias publishes no accuracy numbers of its own — only architecture docs — which is exactly why v0 is our baseline. We can't compare to a number the Pelias project never printed, so we ported the thing and graded it ourselves.)

A couple of points on clean addresses is not the headline. The headline is what happens off the beaten path.

Where the neural net pulls ahead​

Here is the same handful of addresses through both parsers. Watch the rules engine on anything that isn't <number> <street>, <city>, <state> <zip>.

A PO box — the classic. PO Box 1207, Anchorage, AK 99510:

v0    : street="PO Box"  house_number=1207  locality=Anchorage  region=AK  postcode=99510
neural: po_box="PO Box 1207"  locality=Anchorage  region=AK  postcode=99510

The rules parser reads "PO Box 1207" as a street called "PO Box" with house number 1207. It's not wrong by its own logic — "Box" looks like a street type, a number follows it — it has no concept that this is a post office box, a different kind of place entirely. The neural net learned the po_box tag and uses it.

An intersection — 5th & Main, Springfield, IL:

v0    : street="5th"  locality=Springfield  region=IL
neural: intersection_a="5th"  intersection_b="Main"  locality=Springfield  region=IL

v0 keeps the first street and drops "Main" on the floor. The neural net knows an intersection has two sides and labels both.

A unit — 1600 Pennsylvania Ave NW Apt 4B:

v0    : house_number=1600  street="Pennsylvania Ave NW"  unit="4B"  ...
neural: house_number=1600  street="Pennsylvania Ave NW"  unit="Apt 4B"  ...

Both find the unit. The neural net keeps the designator ("Apt") attached, where v0 strips it to the bare "4B".

None of these are flukes. They're the payoff of a discipline we've written about before — negative space: you teach the model the tags the rules engine never had, by training on examples where they appear. The receipts, from our parity evals: unit coverage went from 0 → 92%, street_prefix 0 → 78%, street_suffix 0 → 67% — components the Pelias port only doesn't emit. And on multi-locale evals the same recipe is why native-order German resolves locality at ~90% where a US-order assumption falls to the 40s (a story we've told before — the right name in the wrong state).

The pattern: on the addresses that fit the template, the parsers tie. On PO boxes, intersections, units, sub-street components, and non-US word order, the neural net is doing something the rules engine architecturally cannot.

Where we still struggle (and what we haven't touched)​

Honesty is the house style, so:

• Clean addresses are a tie rather than a win. The two parsers above agree on 350 5th Ave down to the components — neural splits street="5th" + street_suffix="Ave" where v0 keeps "5th Ave" whole, and both are correct. For a model this good on US structure, the rules engine is not a thing to beat there. It's a thing to match.
• Reordered house numbers. French addresses that lead with the postcode still trip the model — fr.house_number plateaus around 87% on the hardest reordered stratum, where the model assigns the leading number to house_number. We pushed it from 54% to 87% and then hit a wall that a training-weight tweak won't move. The next change is real reordered data rather than more synthetic.
• Whole locales we haven't tackled. Japanese, Korean, and Taiwanese addressing — block-and-lot systems that don't map onto a Western street grid — are scoped but unbuilt. So is the last stubborn 0% class in the postal arena: Commonwealth and military PO-box formats (BFPO, APO/FPO).
• Coordinate precision. Today's US coordinate is the admin-centroid tier — we land you in the right city, and "right city" is a centroid that can sit tens of kilometers from an edge address. The finer tiers (postcode centroid, then street-level interpolation) are wired but not yet the default everywhere.

We even tried to have both​

If the neural net wins on weird addresses and ties on clean ones, the obvious move is to run both parsers and keep the better answer per component. We built exactly that — a per-component arbitration layer — and the result showed why label-match is the wrong grade.

On our label-match arena, arbitration looked like a triumph: it captured 122 parses that the neural net by itself "missed." Then we graded the assembled coordinate output instead of the labels, and it was a catastrophe — locality match fell from 83% to 57%, the median coordinate error blew from 3 km to over a thousand, and half the addresses lost their street-and-number entirely.

The diagnosis is the interesting part. The arena's "neural missed it" column conflates two very different things: addresses the neural net gets wrong, and addresses it gets differently right. When v0 writes street="Seminary Dr" and neural writes street="Seminary" + street_suffix="Dr", arbitrating toward v0 "wins" on label-match and loses on the actual parse — and flattening the two parsers' outputs together destroyed the tree structure the resolver needs to tell one "Mill Valley" from another. We rebuilt the layer to preserve that structure. The regression vanished, and so did the result. The safe version is a net wash.

The lesson we're keeping: for a model this strong on the addresses we serve, blending it with the rules parser left coordinate accuracy where the neural model put it without the blend, once the blend kept the tree intact — neural isn't wrong where it differs, it's differently right, and "differently right" is not something to overwrite. We grade the coordinate now, never the label, and we let the arbitration implementation sit behind a default-off flag for the day a weaker model or a new locale makes it pay.

What's on deck​

The parser is in a good place: at or ahead of a mature rules engine on clean US, and ahead on the hard structured tags. We've now mapped exactly where it isn't. The next moves are about feeding it and placing it more precisely:

• Real data over synthetic. Overture Maps address ingestion — to widen the corpus with real reordered, multi-locale, and long-tail formats, the thing the fr-house-number wall told us we need.
• Finer coordinate tiers. Make postcode-centroid and street-level interpolation the default where the data supports it, so "right city" becomes "right block."
• Geocode-first record matching. The other half of the box: once every record resolves to a coordinate, you can match records by where they are rather than how they're spelled — the entity-resolution layer we've been building on top of all of this, and the subject of its own recent post.

A tie on Main Street was never the goal. The parser matches a mature rules engine on clean US addresses, leads on difficult structured tags, and has a measured plan for its next kilometer of accuracy.

The weakness with several faces​

Boundary instability means that where two address components meet, the model puts the boundary in the wrong place. For Country Club Rd, the street name absorbs its own suffix. For North Sydney NSW 2060 with the commas stripped, the city and the state merge. For a French street like Neuve-des-Capucines 5, the house number is absorbed into the name.

For a while we filed several of these cases under "within-token punctuation" and blamed the apostrophes and hyphens. When we measured that class, the punctuation was almost never the cause. O'Connell parses fine, and so does Coeur d'Alene. What breaks is the boundary next to the punctuation. The same boundary error appears in several forms.

The plan​

We planned a synthetic extract that places the gold boundary exactly where the model errs, spread across enough real-looking addresses that a retrain learns the boundary from context instead of memorizing the strings. The steps were to generate it, baseline the current model to measure the size of the gap, write the recipe, and add it to the corpus.

Check one: the extract that graded on a curve​

The first version drew from a thin vocabulary of sixteen street names and a handful of cities. Twenty thousand rows from sixteen streets means the model sees Country Club Rd a thousand times, so it learns the word and not the boundary. We expected that. We did not expect the baseline result: the current model scored 48% on the suffix boundary and 70% on the French prefix against that thin data. Those scores looked reasonable.

We then tripled the vocabulary to a hundred streets and real cities, with every row unique, and the same model scored 41 and 48. The thin data had two effects. It taught the word instead of the boundary, and it inflated the baseline score. The runbook already warns that thin diversity teaches memorization instead of the boundary. The warning now has a measurement next to it.

Check two: the postcode that was already a house number​

To teach the Australian slash convention, where 4/2A means unit 4, number 2A, the extract used Australian addresses. The base-consistency lint blocked it. In our training corpus the token 3000 appears thousands of times, always as a US house number, and the extract labeled it as an Australian postcode. A coverage extract cannot outweigh the base corpus it is added to. It can only conflict with the base corpus and degrade results slowly over one to three releases. That has happened to us before. The lint caught the conflict before we spent any GPU-hours on it. We removed Australia, the extract now covers base locales only, and the slash convention became a separate task that must also add base coverage for its locale.

Check three: the city that was typically a street​

With the extract limited to the locales we train on, the lint found another problem. In our corpus, tokens like Paris, Springfield, and Burlington are mostly streets (Paris Avenue, Springfield Street) and only rarely cities. Our extract labels them as localities. Either label can be correct depending on context, and the model may be context-aware enough to handle it. The lint does not assume that. It flagged the minority sense, because the last time we let a similar case through, a theater on 5th Avenue caused an extract to label 5th Avenue as a venue. The retrain will track this number, and a cleaner extract will draw its cities from names that the corpus already labels as cities.

Why we didn't retrain​

At the end of the night, the extract was built and the baseline measured. The recipe was written and approved, the corpus assembled, and the manifest tested against the real base. Every checklist item was complete, and a training run needed one more command. We deliberately did not run it.

The three checks were the main work of the night. The taxonomy identified the weakness. The checks blocked the fix three times, once for the data and twice for the corpus. Each block cost a few minutes and prevented a retrain whose headline numbers would have looked fine while its labels were wrong. The checks give us the fastest way we know to fix a serious weakness correctly: they block a fix until it passes.