A pilot needs an exit decision, not just enthusiastic users

Enthusiastic pilot users are encouraging. They are also a poor substitute for an exit decision. People may like a tool while relying on a project team to repair its outputs, grant access manually and explain every unusual case.

A pilot should reduce uncertainty about a specific commitment. Before inviting participants, I would write down what decision the pilot will support and what evidence could change that decision. Otherwise a temporary trial can become a permanent service by accident.

Choose the uncertainty worth testing

Different pilots answer different questions. A technical trial might test whether an integration works with the required identity controls. An operational pilot might test whether staff can complete a task during a busy shift. Neither automatically establishes the case for organisation-wide use.

For a fictional room-booking service, the uncertainty might be whether reception staff can resolve clashes without keeping a parallel spreadsheet. That is more useful than "validate the solution" because it directs attention to a particular behaviour and its consequences.

State what the pilot excludes. A small office trial may not test remote connectivity, seasonal demand or the needs of people using assistive technology. Those exclusions should follow the pilot into the decision paper. Limited evidence can justify a limited next step; it should not quietly become evidence for a much larger rollout.

Put entry conditions ahead of invitations

Participants should not become unpaid testers of avoidable setup problems. Before the pilot begins, confirm the permitted data, access arrangements, support route and fallback. The service does not need every future feature, but its limitations must be compatible with the work participants will do.

Nominate someone who can pause the trial. Describe the events that require a pause, such as unexpected disclosure, lost records or a failure of the agreed fallback. The exact triggers depend on the task. They should be concrete enough that a participant does not need executive judgement while dealing with a live problem.

There is a temptation to recruit only confident volunteers. They make a trial easier to run, but may hide the effort other users will need. Include the roles, locations and working conditions relevant to the decision, within the limits of a small trial. Record who could not participate and why.

Managers also need to release time. A pilot that only works because someone stays late has discovered an operating cost, even if the interface is popular. That cost belongs in the findings.

Observe the ordinary work, including assistance

Watch task completion and the help required along the way. If a project team member fixes a booking behind the scenes, count the intervention. Do not report the user journey as successful without mentioning the assistance.

Use a modest evidence plan. Capture the starting task, whether it completed, material errors, workarounds and the participant's explanation. Compare with the existing process where a credible baseline is available. Where it is not, say that the pilot established an initial baseline rather than claiming an improvement.

Feedback should explain the task rather than rate the product in isolation. "I could not move the recurring booking without deleting later sessions" gives the team something to investigate. "Easy to use" may be sincere, but it does not identify the conditions behind the judgement.

The practical detail of collecting feedback from a small internal pilot matters here. Keep collection light enough that people can participate without turning their working day into a research exercise.

Hold an exit meeting with real options

Set the exit date before starting. Bring the evidence, unresolved risks and operating proposal to the meeting. The decision should be one of several genuine choices:

• Proceed to a defined next group because the required conditions were met.
• Extend for a named unanswered question, with a time and effort limit.
• Change the proposed solution and test the changed assumption.
• Stop and return participants to an agreed alternative.

An extension should buy information. "Users would like more time" may be reasonable, but it needs a question that more time can answer. Repeated extensions without a changed evidence plan often mean the organisation has avoided choosing whether to fund a service.

Separate user satisfaction from readiness to operate. A support owner, maintenance capacity and an affordable licensing arrangement may still be missing. The pilot team should not accept those obligations on behalf of another team.

Keep the handover proportional to the result

If the decision is to expand, preserve the limitations that remain. A staged rollout might retain manual review for a difficult class of booking or exclude a location until connectivity is tested. State who owns those restrictions and what would justify removing them.

If the decision is to stop, close access and manage trial data according to the agreed retention arrangements. Tell participants what happened to their feedback. Useful findings can inform a later purchase even when the trialled product is abandoned.

I would regard a well-evidenced stop as a legitimate pilot result. The money and attention committed to a small trial should buy a better decision, not create an obligation to defend the original idea. A recommendation that clearly states its limits is more useful than a glowing report whose claims extend beyond the people and tasks actually observed.