Docusaurus and developer tools company
Indexing a versioned Docusaurus site without answering from the wrong release
The version dropdown is a promise: everything under this path describes one release. A crawl that walks the sitemap does not know that promise exists, and it will happily read four releases of the same page and treat them as four sources for the same question. On a documentation site for developers, that is not an inconvenience. It is the failure that costs a reader an afternoon and costs you the credibility you built the docs for.
Why this pairing is its own job
The Docusaurus install guide covers the tag, and the developer tools company guide covers what the assistant has to know. What follows is the part that belongs to neither.
- Versioning is a first class feature here, so a Docusaurus site that has ever cut a version has several complete copies of most pages, differing by one path segment and almost nothing else.
- The unreleased tree is built from whatever is on the working branch, which means it documents behaviour nobody can install yet. It is published, it is in the sitemap, and it is the single most dangerous thing in the corpus.
- The dropdown changes the URL without a page load, so a reader can move between releases inside one conversation. Their sense of which version they are in comes from the page. The assistant's comes only from what was indexed, and those two drift apart silently.
- Archived versions are snapshots taken at release. They keep whatever was wrong at the time, including the error strings and method names you fixed afterwards, which are exactly what people paste into a chat box.
What changes about the install here
<script src="https://cdn.askably.xyz/w.js" data-key="pk_live_YOUR_KEY" defer></script>The tag is the same one on the Docusaurus guide. Everything below is about where it goes on a developer tools company site specifically.
One configuration entry covers every version path
The scripts array is site level, so the tag lands on the current tree, the unreleased tree and every archived tree from one entry. There is no per version install and nothing to repeat when you cut the next release.
That is convenient and it is also the trap, because it makes the install look finished at the point where the real decision has not been made. Where the widget appears and what it is allowed to read are two separate settings, and only one of them was just configured.
Decide the crawl scope before the tag goes live, not after
The generated sitemap lists every version path the site builds. Pointing a crawl at it without a path filter takes all of them, and once a page is in the corpus it stays there until you re-index, so a scope mistake is not something the next deploy fixes.
Work out first which releases you support, then restrict the crawl to those paths, then install. Doing it in that order means the first answer anybody sees is one you would stand behind.
If your team is still arguing about which versions to support, that argument is the blocker rather than the install. Ship the tag with a corpus of the current release only. Adding a path later is cheap and unpicking a wrong answer someone screenshotted is not.
Where the answerable material lives
| The current release tree | The path with no version segment in it, which resolves to whatever you last cut. This is the corpus for almost every question, and on most sites it is the only tree that should be indexed at all. |
|---|---|
| The unreleased tree | Built from the working branch and published alongside everything else. It describes signatures that have never shipped, and an answer taken from it is wrong for every reader on a released version, which is all of them. Exclude the path. |
| Archived version trees | Frozen copies kept for people who cannot upgrade yet. They are genuinely useful to those readers and actively harmful to everyone else, so if you index them at all, index them alone under an assistant of their own rather than mixing them with current material. |
| Migration guides, deprecation notices and the changelog | These usually sit outside the versioned tree, in the blog section or on a single unversioned page. That makes them the one part of the corpus that is safe to index whole, and they are what answers the question underneath most version questions, which is what replaced this and when does the old one stop working. |
The first thing to get right
Exclude the unreleased path from the crawl before you index anything else, then ask the assistant about a method that only exists on that branch and confirm it refuses.
Every other version problem produces an answer that is merely old. This one produces an answer that has never been true for anybody, phrased with the same confidence and carrying a citation to a real page on your own site. It is the only exclusion worth doing before launch rather than after the first complaint.
The failure that belongs to this combination
A citation that names the page but not the release
Two versions of the same page share a title, a heading structure and most of their sentences. When an answer cites one of them, the reader sees a familiar page name and a link, reads the answer as confirmed, and never looks at the path segment that says which release it came from.
The way this surfaces is a bug report saying the assistant told me this method takes three arguments, from a reader whose installed version takes two. Both statements are true of some release. Nothing in the exchange told either party which one was in play.
The fix is structural rather than conversational. Restrict the corpus to releases you support, keep archived material under a separate assistant if you want it answerable at all, and raise the caution setting so a weak match returns your refusal instead of reaching into the nearest page with matching words.
Before you go live
- The origin the page is served from has to be on the allowlist for that assistant, or nothing renders and the browser console says which origin was refused. An apex domain and its www are two different origins to a browser, so list both, along with any staging or preview host you want it to work on.
- Open the site as a visitor would, on the pages a developer tools company visitor actually lands on, and ask it something only your own material could answer. A widget that renders is not the same as a widget that has read anything.
Questions
- Should archived versions be in the index at all?
- Usually not in the same index. A question rarely arrives with a version attached, so retrieval has no way to prefer the release the reader is on, and the older page often matches better because it is the one with the old method name in it. If a supported long term release genuinely needs answering, give it its own assistant with its own key on that path.
- Can it tell which version of the docs the reader is on?
- No. It answers from indexed material, not from the page around it, so it has no view of the URL a reader is standing on and does not infer one. That is why the corpus has to be scoped: the version discipline has to live in what you indexed, because it cannot live in the conversation.
- We removed a method two releases ago and it keeps coming back. Why?
- Because it is still published somewhere the crawl reached: an archived tree, an old tutorial that was never versioned, or a blog post announcing the feature when it launched. Search your own built site for the method name and you will usually find the page in under a minute. Removing it from the corpus needs a re-index, because the assistant answers from what it last read.
Keep reading
- Installing on DocusaurusAdd an object to the scripts array in docusaurus.config.js carrying the source, defer and the data key, then rebuild the site.
- Everything for a developer tools companyOn a docs site an assistant competes with search, not a phone line. Version skew, deprecations and error strings decide whether it earns its place.
- One origin, two build systems, and a tag from eachDocs proxied under a path on the marketing domain can end up carrying two tags, and the loader installs once, so one team's key quietly loses.
- Documentation you cannot put a launcher on, and two public copies of itThe tag cannot go on a Notion page, so the reader who failed your search box is the one person who cannot reach the assistant.
- The forum is not your documentation and must not be read as itThe threads are written by your users, not by you. What to index when the documentation and the forum sit on two origins.
Try it on your own material
Upload a document or point it at your site, paste one line of HTML, then ask it something only your business could answer.