Why the trial never finishes, and how to make it finish

The pattern is consistent enough to be worth describing precisely. It goes live on a section of the site nobody visits, or behind a login, or on a staging address shared in a message thread. Four or five people from the team try it and report back with impressions. Somebody collects the impressions.

Then the decision gets attached to something else. After the redesign. After the busy period. After the new hire starts. Once we have moved the help centre. Each deferral is individually reasonable and the sequence never runs out, because there is always a next thing.

Six months later the state is unchanged. It is still switched on. Nobody has read a conversation since the second week. The material behind it has not been touched since it was indexed. Nobody would describe the project as failed, and nobody would describe it as done, and there is no meeting at which that gap would come up.

The mechanical reason the stall persists is that nothing is accumulating. A trial run on a page with no visitors, using questions the team invented, is a demonstration. It looks like evidence and it is not, because the inputs came from the wrong population.

Team members ask questions they already know the answer to. They use internal vocabulary, because it is the vocabulary they have. They test the interesting cases rather than the boring ones, and boring is where the volume is. Then they form an impression from a handful of exchanges and that impression becomes the organisation's view, immovable, based on nothing.

Strangers with real questions produce completely different material within days. They ask things nobody thought of. They phrase things nobody would phrase that way. They ask the same three things repeatedly, which is itself the finding. So the first fix for a stalled pilot is almost never more evaluation. It is exposure. Put it where real visitors are, on a page that matters, and the evidence problem solves itself.

Underneath the deferrals there is usually one unresolved question, and it is not whether the thing works. It is who maintains the material, permanently.

Everyone can imagine approving a tool. Approving a tool is a bounded act with a cost you can put in a budget. What is actually being proposed is a small ongoing obligation with no end date: somebody reads the failed questions, somebody writes the missing answers, somebody notices when a price changes and updates the page it lives on. That is not a project. It is a permanent addition to a job, and nobody wants to be the person who volunteers for it in a meeting.

So the discussion keeps getting reframed as a technical question, because technical questions can be sent away for more testing and ongoing obligations cannot. If your pilot has stalled and you cannot say why, ask the room who owns the content in six months. The silence after that question is the actual blocker, and naming it usually moves things more than any amount of further evaluation.

There is a second reason these projects avoid a decision, and it is that they are usually pure addition. Nothing is being replaced. The phone line stays. The contact form stays. The inbox stays. A new channel is being added alongside all of them, at low cost, and a thing that adds without removing never forces anybody to decide anything.

So ask what changes on the day it is approved. Something concrete and visible: a contact form removed from a page, a phone line's advertised hours reduced, a set of pages rewritten and consolidated, an inbox rule changed, an out of hours message replaced. If the honest answer is that nothing changes, then approval has no content and the pilot can run indefinitely without anybody noticing, because it makes no difference either way.

This is also the test for whether anybody actually believes it works. A team that is confident will name something they are willing to stop doing. A team that names nothing is telling you something useful about their confidence, whatever the impressions in the feedback thread say.

The stall feels cheap, which is why it lasts. The subscription is small, nobody is spending time on it, and no decision means no wrong decision. Each of those is partly true and the combination is misleading.

The material rots while nobody owns it. Prices change, the team changes, a service is discontinued, and the indexed version keeps saying what it said. A neglected assistant is not neutral, because it is still answering, and it is answering from a snapshot that gets older every week.

Meanwhile the organisation's opinion hardens around the demonstration rather than around evidence, and it hardens in whichever direction the loudest early tester leaned. And the decision eventually gets made by attrition. The person who championed it leaves, or the person who disliked it gets a bigger job, and the outcome is settled by staffing. Every one of those is worse than a considered no in month two.

The intervention is a paragraph, written before anything is switched on, and it is short enough that there is no excuse for skipping it. It contains four things: the date the decision is made, the name of the person making it, the two or three things that will be looked at on that date, and what happens on both a yes and a no.

The both is what does the work. Most pre commitments describe only the success case, which means a no has no defined consequence and can therefore be avoided indefinitely. Write the no out. On a no, it comes off the site by this date, the pages we wrote stay, and we stop. A no with a plan is a completely respectable outcome and a project that cannot produce one is not being evaluated.

Put the paragraph where the team will see it rather than in a document nobody opens. It only has to survive until the date, and its job is to make the arrival of that date impossible to miss.

Three things, all obtainable, none of them a composite score. First, the list of questions it could not answer. That list is a content plan and it is worth having whatever you decide, which makes it the safest thing to build the evaluation around.

Second, a read of a fixed number of real conversations from real visitors, decided in advance so that the sample is not chosen after the fact by whoever wants a particular answer. Third, whether anybody complained, internally or externally, and what about.

Expect the honest answer to be partial. It very often works for one section of the site and not another, usually because the material for one section is in decent shape and the material for the other is in somebody's head. A partial yes is a real result and it is better than either of the clean answers, because it tells you exactly where the next piece of work is.

If the date arrives and there has been no exposure, no owner, and no work on the material, the evaluation has not happened and cannot be rescued by extending it. Extending it produces another six months of the same nothing.

The honest reading of that situation is that the organisation has decided against it without saying so. That is a legitimate decision. There are only so many small ongoing obligations a business can carry and this one lost. Say it in those words, because the alternative framing, that it did not work, is both untrue and unhelpful to whoever tries again in two years.

Then take it off cleanly and keep the content work, which was the valuable part all along. Every page written during the pilot to fill a gap is still a page that answers a real question, and it goes on working whether anything is switched on or not.