- Market
- Saudi Arabia
- Written for
- CIO, Head of Digital, Procurement
- Decision
- Interrogate an AI vendor on data, accuracy, failure, exit and cost before committing
- Languages
- EN · AR
The decision
Five answer sets that decide whether an AI contract is signable
The demo answers none of these. The questions do not require technical depth to ask — only the discipline to refuse vague answers.
Five inputs — where the data goes, accuracy on your work, behaviour when wrong, what leaves with you, the costs that grow — converge into a signing decision with three outcomes: sign with the answers in the contract, pilot with measures agreed first, or walk away while it is cheap.
Where the data goes
Storage, processing, sub-processors, and whether your data trains their models.
Accuracy on your work
Measured on a sample of your data, with a metric you chose.
Behaviour when wrong
Fallbacks, human handover, and who reviews what.
What leaves with you
Prompts, configurations, evaluations and data, exportable at exit.
The costs that grow
Usage pricing, integration upkeep and the human review bill.
What we do
The signing decision
Weighed together, in writing — a strong demo compensates for none of them.
Sign, with the answers in the contract
Each commitment survives the salesperson who made it.
Pilot, with measures agreed first
A pilot without a pass bar is a longer demo.
Walk away while it is cheap
Refusing to answer is itself an answer.
Accuracy
Why a benchmark score says little about accuracy on your own work
Vendor accuracy figures are produced on the vendor's evaluation set — data chosen, cleaned and sometimes tuned against, under conditions that flatter the product. Your invoices, your dialect mix on the phone lines, your abbreviations and your edge cases were not in it. The only accuracy number that predicts your experience is one measured on a sample of your own data, with a metric you defined, scored by your people. A capable vendor will run that evaluation before contract, because their product survives it; reluctance to be measured on your data is information about the product.
The second half of the accuracy question is what happens to the errors, because there will be errors, delivered fluently and with confidence. Ask the vendor to show a failure: what the system does when it is unsure, how a person is brought in, what the customer or employee sees during the handover, and what record the failure leaves for review. A vendor who can demonstrate a graceful failure has engineered for reality; one who steers back to the happy path has engineered for demos.
The questions
The questions to put to an AI vendor, phrased so the answers commit
Each question names the written commitment to walk away with. A verbal reassurance is the absence of one.
1 · Where is our data stored and processed, and does it train your models?
Regions for storage and processing, every sub-processor in the chain, retention periods, and an explicit statement on whether your data — including prompts and outputs — is used for model training or improvement, with the opt-out mechanics if it is. Data-processing terms naming locations, sub-processors and the training-use position.
2 · Will you be measured on our data before we sign?
A pre-contract evaluation on a representative sample of your data, with the metric, the sample and the pass bar agreed in advance and your people doing the scoring. An evaluation protocol in writing, run before signature.
3 · Show us the system failing, and the human path out
A live demonstration of low-confidence behaviour, the handover to a person, and the audit trail an error leaves. Agree who on your side reviews errors and at what cadence. A documented fallback design and a review cadence with named owners.
4 · If we leave in two years, what leaves with us?
Your data, your prompts and configurations, your evaluation sets and the interaction history — exportable in open formats, within a defined window, at a price fixed now. What cannot leave defines your real lock-in, whatever the sales deck says. Exit and portability terms enumerated in the contract.
5 · Which of our costs grow with use, and by what rule?
The usage dimensions that drive the bill, the price protections at renewal, who pays when an integration breaks on either side's change, and the internal cost the vendor never quotes: the people who review the system's output. A cost model you drew yourselves, checked against the contract.
Our role
How we answer this page when we are the AI vendor in the room
- 01
Measured before signed
We evaluate on your data with your metric before contract, and we publish the protocol up front — because a system that cannot survive your sample should not survive your procurement.
- 02
Failure engineered, not hidden
Fallbacks, human handover and error review are part of what we build and demonstrate, not an appendix — the wrong answer a system gives is an operational event, and it needs a path.
- 03
Exit terms we accept ourselves
Your prompts, configurations, evaluations and data are yours to export in open formats, and we put that in the contract — a relationship held together by switching costs is not one we want to be in.
Whether any AI product earns a place in your operation is your call; these questions exist so the call is made on evidence, and we hold ourselves answerable to every one of them.
Common questions
What buyers ask
How do we find out whether an AI vendor trains its models on our data?+
Ask the question in writing and require the answer in the data-processing terms, not in an email from sales. The precise wording matters: ask whether customer content — inputs, prompts, files and outputs — is used for training, fine-tuning or "service improvement", because the last phrase is where training use often hides. Ask whether the position differs between the product's tiers, since consumer and enterprise tiers of the same product frequently differ. Ask how the commitment binds the vendor's own sub-processors, including any underlying model provider. And ask what changes on contract renewal. If the final contract does not state the position explicitly, the working assumption should be that your data is usable, whatever was said in the meeting.
What should an AI pilot measure before the contract is signed?+
Three things, all agreed before the pilot starts: quality, effort and behaviour at the edges. Quality means the accuracy metric you chose, measured on your data, against a pass bar written down in advance — a pilot whose success criteria are decided after the results exist will always succeed. Effort means what it actually took: the integration hours, the data preparation, the tuning cycles, because that effort recurs at every change. Behaviour at the edges means deliberately feeding the pilot the hard cases — the dialect calls, the malformed documents, the ambiguous requests — and watching the fallback path run, since the easy cases were never in question. A pilot that only exercises the demo path is a longer demo, at your expense.
Which AI running costs tend to grow after signature?+
Four, predictably. Usage-based pricing grows with adoption — the better the system works, the more it is used, and the bill scales with the success; ask what volume protections and renewal caps exist. Integration upkeep grows because both sides change: your systems evolve, the vendor ships new model versions with different behaviour, and someone pays for re-testing each time. Human review grows quietly, because the people checking the system's output are a real cost that appears in nobody's quote. And data preparation recurs, since the estate the system was tuned to keeps changing underneath it. None of these is a reason not to buy; each is a reason to model the cost over the contract's life rather than at its first month.
In this cluster
Pressure-test a vendor with us
A session applying these questions to a shortlist you are actually weighing — what the answers mean, and which gaps belong in the contract.
Start the conversation
