Running a demonstration that tells you something
A standard vendor demonstration is designed to be impressive, and it succeeds. Rehearsed data, a clean happy path, an experienced presenter who steers away from anything untidy. Watching four of those in a fortnight leaves a committee with an impression rather than information. Take control of the format instead. Send scenarios in advance, drawn from your own transactions and your own data, and require them to be run live, in the order you set, by a consultant rather than a marketing lead.
Exceptions are where the useful information hides. Ask for the partial delivery, the credit note against a prior period, the customer paying three invoices with one short transfer, the item purchased in cartons and sold in pieces, the price agreed in a foreign currency and settled two months later. Ask to be shown rather than told, then ask a second question after every answer: was that standard behaviour, configuration, or development. The answers separate what the product does from what the vendor is willing to build.
Scoring should happen individually, immediately, before the room discusses anything. Give every attendee the same sheet and collect it before the conversation starts, otherwise the most confident voice in the room becomes the group's memory of what happened. Include the people who will use the system daily, not only the managers who will sign for it. One boundary is worth naming. A demonstration proves the software can be made to do something. It does not prove that your team will operate it that way on a Tuesday in March.
- Scenario scripts issued in advance and built from your own transactions and data
- Every answer classified as standard behaviour, configuration or development work
- Individual scoring collected before any group discussion of what was seen
- Daily users present in the room alongside the managers who will approve the purchase
- Exception cases scripted deliberately, since happy path demonstrations separate nothing