Run a Software Pilot With Clear Acceptance Criteria
A pilot succeeds when you define, before it starts, which workflows must work, who will use them, what data they will touch, and what happens if the system fails. Write those acceptance criteria down, test them with real records, and keep a rollback path ready.
Pick workflows that carry real weight
Start with the two or three workflows that cause the most friction today. A quote-to-invoice flow, a stock reorder, or a customer onboarding sequence are good candidates because they cross roles and systems. Avoid piloting a workflow nobody depends on; it will pass and teach you nothing.
For each workflow, write the steps as they happen now, including who hands off to whom and where delays or errors appear. This becomes your test script. If a step is unclear, that is a finding before the pilot even begins.
Suppose a distributor wants to pilot a custom inventory and invoicing system. The team picks two workflows: receiving stock from a supplier and issuing a customer invoice with a partial payment. They write down each step, the person responsible, and the record created at the end.
Name the roles and their permissions
A pilot without named users is a demo. List every role that touches the workflow, from data entry to approval to reporting. For each role, state what they must be able to see, create, edit, and approve, and what they must not.
Test permissions with the actual people who will use the system. A warehouse clerk and a finance manager should not see the same screens. If a role cannot complete its part of the workflow without help, the pilot has not met its criteria.
One practical test: ask each named user to complete their step without assistance. If they need a developer to adjust something mid-task, note it as a failure for that role and fix it before expanding the pilot.
Use sample records that look like real ones
Load a small set of records that mirror your actual data: a customer with multiple addresses, an item with two units of measure, an invoice with a discount and a tax line. Synthetic data that is too clean hides integration and validation problems.
Include at least one record that is incomplete or unusual, such as a supplier without a tax number or an order with a backordered item. The system should either handle it gracefully or flag it clearly, not silently drop it.
Suppose a clinic pilots a scheduling and billing module. They load a patient with two insurance plans and a cancelled appointment. The test is whether the front desk can reschedule and bill correctly without calling the vendor.
Test the exceptions, not just the happy path
Most pilot failures appear at the edges: a duplicate customer, a partial shipment, a refund after a price change. Write a short list of exception cases and run them during the pilot. If the system cannot handle an exception, decide whether to change the process or the software.
For each exception, define the expected behavior. Should the system block the action, create a task, or allow it with a warning? Ambiguity here becomes a support ticket later.
A concrete pass or fail test: take one exception case, such as an invoice that must be voided after payment. Run it end to end. If the void leaves the payment unmatched and no one can explain why, the pilot fails that criterion until the behavior is defined and corrected.
Plan the rollback before you go live
A pilot is not a commitment to full rollout. Decide in advance what triggers a rollback: a workflow that cannot complete, data that cannot be reconciled, or a role that cannot work without constant intervention. Write the trigger down and name who can call it.
Rollback does not always mean turning the system off. It can mean reverting to the previous process for one workflow while keeping another in the pilot. The point is to have a known, tested way back.
Suppose a retailer pilots a new point-of-sale and inventory link. They agree that if daily stock counts cannot be reconciled within a set time for three consecutive days, they will revert to the old process for that store while the vendor investigates. The decision is made before the pilot starts, not in the middle of a bad week.
Close the pilot with evidence, not opinion
At the end, review each acceptance criterion against what actually happened. Use the test scripts, the exception cases, and the user feedback. A criterion is met only if the named role completed the workflow with real records and the exception behavior matched what you defined.
Document what worked, what did not, and what changed during the pilot. This record becomes the basis for a rollout decision, a change request, or a decision to stop. It also protects both sides from vague expectations.
If the pilot met its criteria, expand in stages, adding one workflow or one team at a time. If it did not, fix the specific failures and re-test before widening the scope. The goal is a decision you can defend, not a feeling that it went well.
Continue reading
A practical next step
Explore the related Devign work, then discuss your requirements before committing to a project scope.