ScrapeField
A growing share of “which API should I use” is decided inside a coding agent, by a model that reads the documentation, picks a provider and writes the call in the same minute. That reader does not want our navigation bar. It wants the parameters.
The embarrassing part
We published an llms.txt from day one — a table of contents for models, linking every endpoint’s documentation as a .md file. Every one of those twenty-two links returned 404. Our flagship agent-facing file pointed at nothing, and every test we had was happy, because no test followed the links. One curl found it. The same file also printed a literal ${brand.keyPrefix} where a key prefix should have been.
Two sources
The documentation pages — every endpoint, the errors, pricing, the four surfaces — are written for a model and generated from the same table the gateway runs on, so they cannot drift from what the API does. Every other page is the rendered page converted, stripped to its content. That second path is the one that makes the promise “every page” rather than “the pages somebody remembered”: a page written next month is readable as Markdown the day it ships.
curl https://scrapefield.com/docs/google-maps/places.md
curl https://scrapefield.com/pricing.md
curl https://scrapefield.com/skill.mdThe tests are the curl
Two tests now do what the curl did: nothing we generate may contain an un-interpolated placeholder, and every .md link in llms.txt must be a document something actually produces. A green suite is not a working system; the fix is to make the suite do what the person checking it would have done.