Source material
Release notes describe changes, and customers ask about the present
A changelog looks like a perfect source: dated, factual, written down as it happened, already public. It is one of the most reliably harmful documents you can index, because every entry in it is a true statement about a moment and almost every question is about now. Nothing in the document distinguishes the two, and nothing about being a year old makes an entry look less relevant to a matcher.
Why this one is harder than it looks
Retrieval has no sense of chronology. Entries are matched by meaning, and an entry from three years ago describing a feature by name will match a question about that feature just as strongly as this month's entry does. The document is ordered by date and the ordering is presentational; it exists in the arrangement of the page and not in the meaning of any passage. Newest and oldest arrive looking identical.
Which leads directly to the reverted change. Software teams add a thing, discover it was wrong, and remove it again, and both events are recorded honestly in the changelog. The addition is a confident, well-written passage describing a capability. The removal is a shorter line somewhere further up. A question about that capability retrieves the addition, and the assistant tells a customer the product does something it stopped doing.
Entries also assume the reader was here last time. Release notes are written for people following along, so they say things like this now respects the setting introduced in the previous release, or the behaviour described below is now the default. Read in sequence that is perfectly clear. Read as a single passage it is a sentence about an unnamed setting and an unspecified behaviour, and it can still be retrieved and quoted, because it mentions the right nouns.
Deprecations are the highest-stakes case and are almost always underwritten. A note saying the old export format is deprecated is useless on its own: it does not say when it stops working, it does not say what replaces it, and it does not say what the customer should do. Split across a passage boundary from whatever paragraph carried those details, it becomes an answer that raises an alarm and provides no way to act on it.
What it has to contain
Structure rather than wording. A passage pulled out of this document has to stand on its own, because that is the only form in which it will ever be read.
| A current behaviour page for anything customers ask about | A page describing how a feature works today, with no history in it. This is the page that gets indexed. The changelog links to it rather than the other way round. |
|---|---|
| Every entry self-dated in its own text | The date written into the sentence, not only in the heading above a group of entries. A passage lifted out of a dated section takes none of the section heading with it. |
| No entry that depends on the previous entry | Every entry names the feature, the setting or the behaviour in full, rather than referring to what was described last time or above. |
| Reversals written onto the original entry | When a change is undone or superseded, the original entry gains a line saying so and naming the date. Correcting the record where the record is, rather than only further up the page, is what stops the old entry being quoted as current. |
| A deprecation stated in one sentence | What is being retired, the date it stops working, and what to use instead, all in the same sentence. Three facts in three paragraphs is three passages, and the customer only gets one of them. |
The rewrite, in order
Build the current behaviour pages and index those instead
For each feature customers actually ask about, write one page describing how it works today. Present tense, no history, no mention of when it changed. This is the source of truth for the assistant, and it is a genuinely useful page for humans too, because the question how does this work today is not answerable from a changelog by anybody, machine or otherwise.
Keep the changelog exactly as it is for the audience it serves, developers and customers tracking what moved. Just take it out of the assistant's material, or keep only the most recent portion of it. The changelog and the assistant want opposite things from the same facts: one wants the record of every state, the other wants only the current one.
Link the changelog entries to the current behaviour pages. That way somebody arriving at an old entry through search finds their way to what is true now, and you have one canonical description to maintain rather than a truth scattered across every release that touched it.
Make every entry carry its own date and its own nouns
If you are going to index release notes at all, go through and put the date inside each entry rather than only in the heading above a group of them. In this release becomes in the release of that date. From now on becomes from that date onwards. The passage then says when it is talking about even when the heading has not come with it.
Remove every backward reference. As mentioned in the previous release, the setting described above, this builds on last month's change. Replace each with the actual name of the thing. The entry gets longer and reads slightly more stiffly to somebody going through the whole list, and it becomes capable of standing on its own, which it currently is not.
Be strict about the word now. Now is the most dangerous word in a changelog, because it means at the time of writing and reads as at the time of asking. Replace it with the date.
Correct the record where the record is
Walk the changelog for changes that were later undone, replaced or quietly superseded. For each one, edit the original entry to say so: this was reverted on that date, this was replaced by the following behaviour on that date. An addition and a removal recorded as two separate entries is two passages, and only one of them will be retrieved.
Do the same for anything that has drifted. A feature described accurately at launch and altered three times since has an entry that is now subtly wrong in a way nobody has noticed, because nobody rereads old release notes. Those are the entries that produce the most confident wrong answers, since they are detailed, well written and completely obsolete.
If editing history feels wrong, and for some teams it genuinely is, that settles the question of whether to index the changelog: do not. Keep the historical record immutable for the people who need a historical record, and let the current behaviour pages be what the assistant reads.
Write deprecations as one sentence with three facts
Every deprecation notice needs the thing being retired, the date it stops working, and the replacement, in a single sentence. The old bulk export stops working on that date, and the replacement is this other export, which takes the same filters. One sentence, one passage, one complete answer.
The detail can follow: what changes in the output, what has to be adjusted, who is affected. But the sentence above it has to be complete on its own, because it is the one that will be retrieved and it may be the only one that is.
A deprecation without a date is not a deprecation, it is an opinion, and it produces the most frustrating answer an assistant can give: an alarming statement with no action attached. If the date is not decided yet, say that explicitly in the same sentence rather than leaving it out, because a customer who is told the date is not fixed is better informed than one who is told to worry.
What happens if you skip it
A confident description of something that no longer exists
A customer asks whether the product can do a particular thing. It could, briefly, eighteen months ago. The entry announcing it is detailed and enthusiastic, because launch entries always are, and it matches the question beautifully. The line removing it three releases later is four words long and matches nothing. The assistant quotes the launch entry and cites it accurately.
The customer is now told the product has a capability it does not have. They may buy on that basis. They will certainly try to use it, fail, and come back with a support conversation that now starts from a position of being misled, which is a far worse conversation than one that starts from not knowing.
The near miss is just as expensive: an answer describing behaviour as it was two versions ago, correct in every detail and wrong in the only way that matters. Because a changelog entry reads as a factual record, and because the assistant cites it, the answer carries more apparent authority than a vaguer but current one would. The precision is what makes it dangerous.
Check it against this
Before you index it
- A current behaviour page exists for every feature customers ask about
- The assistant indexes those pages, not the release history
- Every indexed entry contains its own date in its own text
- The word now has been replaced with a date throughout
- No entry refers to the previous release without naming what it means
- Reverted and superseded changes are marked on the original entry
- Every deprecation states what, when and the replacement in one sentence
Questions
- Can I index just the last few months of release notes?
- It is much safer than indexing all of them, and it still has the reverted change problem within that window. If you do it, mark reversals on the original entries and keep the window short enough that you can actually reread it. Recent release notes are most useful for answering what changed recently, which is a real question and one the current behaviour pages do not answer.
- Customers ask when something changed. Does removing the changelog break that?
- It means the assistant will decline that question rather than guess at it, which for a question about dates is the right trade. A wrong date sounds exactly like a right one. If those questions are common, add a short summary page of significant changes with their dates and index that instead of the full history.
- What about a page that lists known issues?
- Same problem, sharper. A known issues page is a list of temporary truths, and a resolved issue left on it will be quoted as current. Either keep it rigorously pruned and dated, or leave it out of the material. An assistant telling a customer about a bug you fixed last quarter is worse than one that does not mention it.
Keep reading
- How to turn an onboarding guide into a source an assistant can answer fromOnboarding guides are sequences. Retrieval hands back a middle. Here is how to rewrite the steps, the prerequisites and the screenshots.
- How to write a troubleshooting article that gets found by the error messagePeople paste the error text. If your article never quotes it verbatim, it is invisible. How to restructure symptoms, causes and decision trees.
- How to make an API reference answerable and not just completeGenerated reference pages are mostly boilerplate, which makes every passage look alike. Fix the tables, the examples and the version paths.
- Every kind of source materialWhat to feed an assistant, and the surgery each document needs first.
Try it on your own material
Upload a document or point it at your site, paste one line of HTML, then ask it something only your business could answer.