Documentation with versions, and the answer that was true last year

A visitor asks how a setting behaves. They get back a clear, well written, internally consistent explanation with a link to the page it came from. The explanation is accurate. The page is real. The visitor follows it, finds the setting is not where it was described, and concludes that either the documentation or the assistant is lying to them.

Neither is. The paragraph was true of a release that shipped eighteen months ago and is still published, still linked, still indexed, and still the best textual match for the words the visitor typed. Nothing in the system knew that the visitor was on a different release, because nothing in the system was told, and nothing in the paragraph said which release it belonged to.

This is why versioned material deserves its own treatment rather than being folded into general advice about keeping content fresh. Stale content is wrong and can be corrected. Versioned content is right and wrong at the same time, depending on a fact about the reader that the text does not contain.

From inside the site, a version selector is a dropdown. From the point of view of anything that reads the text, it is a decision to publish several near identical copies of every page, differing in a handful of sentences, at several addresses, all of them live.

That is a hostile shape for retrieval. Near duplicates compete with each other, and when passages are almost identical the difference between the top scoring one and the third is close to noise. Which copy wins can turn on a stray heading, an extra sentence, or which one happened to be crawled more completely. You are not choosing a version at that point. You are getting one.

It is worth being blunt about the consequence. If every published version is in the same body of material with nothing distinguishing them, the version the visitor is on has no influence at all on the answer they get. The selector on the page is doing work for human readers that it does not do for anything reading the text.

The instinct is to treat the current release as the material and older releases as a degraded copy of it. That framing produces the wrong handling, because it suggests the old pages can be left in place and quietly outranked.

The better framing is that each supported release is its own body of material with its own set of true statements. The archived release is not a worse version of the truth. It is the truth about a different thing, one that some of your users are still running and will be running for a while yet. It has its own defaults, its own limits, its own names for features that were renamed later, and a question answered from it is not a mistake if the person asking is on it.

That reframing decides the architecture. Two corpora means two boundaries, and a boundary is something an assistant can respect. One corpus containing both means no boundary, and no amount of instruction will reliably reconstruct one, because the retrieval step has already thrown away the distinction before anything gets to decide what to say.

Releases nobody is running are the easy exclusion, and the one most often skipped because taking pages down feels destructive. It is not destructive if they are archived somewhere a person can still reach them. It is destructive to leave them in a body of material that answers questions, because they will answer some.

Pre release material is the exclusion people forget. Beta notes, release candidates and anything documenting behaviour that changed before it shipped are all confidently written descriptions of something that never existed in public. They read exactly like documentation because they are documentation, and they describe a product nobody has.

Migration guides need a decision rather than a default. A guide describing the path from a release you no longer support is a rich source of statements about how things used to work, phrased in the present tense. Either exclude it or rewrite its framing so that every claim about old behaviour is explicitly about old behaviour. Changelogs deserve the same scrutiny for the opposite reason: they are a log of differences, not an account of how anything currently works, and an answer assembled from changelog entries tends to describe a sequence of changes to somebody who only wanted to know the current state.

The single highest value edit in versioned documentation is putting the version into the body text rather than only into the address, the breadcrumb, the sidebar or the selector. Everything except the body text is chrome, and chrome is exactly what gets stripped before text is indexed.

So write the applicability into the sentences. A short line at the top of each substantial section naming which release it describes. Where behaviour changed, a sentence in the paragraph itself saying what it was before and from which release it has been the way it is now. This is duplication and it will look slightly clumsy to a human reader skimming the page. It is the difference between an answer that carries its own qualification and one that does not.

The test is simple and worth running on your own pages. Copy a single paragraph out of the middle of a versioned page and paste it somewhere with no surrounding context. Read it. If nothing in it tells you which release it is about, then nothing that quotes it will tell anybody either, and quoting a paragraph out of context is precisely what a retrieval system does.

Given separate corpora, there are two workable designs and they trade off differently. The first is one assistant per supported release, each embedded on the pages for that release and answering only from that corpus. It is the simpler design, it makes the boundary structural rather than conversational, and its weakness is that a visitor who landed on the wrong version of the docs gets confidently answered from the wrong material with no prompt to check.

The second is to establish the version before answering. Ask, once, and hold the answer for the conversation. This handles the misdirected visitor and it costs you a question at the start of every conversation, including the ones where the answer is the same across all releases. It also depends on the visitor knowing which version they are on, which for a hosted product they usually do not and for an installed one they usually do.

There is no third option that quietly does the right thing without either a boundary or a question. Anything that tries to infer the version from the phrasing of the question is guessing, and it will guess wrong on exactly the questions where the versions differ, because those are the questions where the vocabulary changed.

Upgrade questions are the genuinely hard case, because they are about the relationship between two corpora rather than about either one. What breaks if I move up. Is this feature in the release I am on. Why does the thing I read about not exist for me.

These are worth handling as their own small body of material rather than trying to make either version answer them. A short, deliberately maintained upgrade document that names the releases explicitly and states the differences as differences is answerable. The same information scattered across two corpora as two competing present tense descriptions is not, and will produce an answer that describes one release while sounding like it describes both.

It is also the place where a handover earns its keep. Upgrade questions frequently depend on what the person has actually deployed, which is not in any document you own. An assistant that recognises the shape of the question and offers a person is behaving better than one that answers it from the general case.

Add the release cut to whatever checklist already exists, because this is a maintenance problem that presents as a content problem months later. The steps are short. Reindex the new current corpus. Move the previous release into its own boundary rather than leaving it mixed in. Retire anything now out of support so it stops answering. Update the applicability lines in any section whose behaviour changed.

Then check the default. Every setup has a default corpus, whether or not anybody chose it deliberately, and after a release cut the default is frequently still pointing at the version that was current when the thing was first configured. This is the single most common way a versioned setup goes quietly wrong: nothing broke, nothing errored, and the answers are a release behind.

Finally, ask a handful of questions whose answers changed in this release and check what comes back. Not a general test of whether the assistant works, which will pass. A targeted test of the specific sentences that are now different, which is where the failure lives.