Techbanq

Buyer's checklist · AI procurement

The AI vendor answers that matter more than the demo

A demo is the product on its best day, on the vendor's data, with the failures edited out. The contract you are considering will run on your data, on ordinary days, failures included. Five families of questions close that gap, and every one of them can be asked before a riyal moves.

Serving Saudi Arabia from our Riyadh office and our engineering hubs in Lahore and Karachi

Market
Saudi Arabia
Written for
CIO, Head of Digital, Procurement
Decision
Interrogate an AI vendor on data, accuracy, failure, exit and cost before committing
Languages
EN · AR

Enterprise delivery credentials

  • ISO

    ISO/IEC 27001

    ISO/IEC 27001 certified

Certifications & registrations

The decision

Five answer sets that decide whether an AI contract is signable

The demo answers none of these. The questions do not require technical depth to ask — only the discipline to refuse vague answers.

Five inputs — where the data goes, accuracy on your work, behaviour when wrong, what leaves with you, the costs that grow — converge into a signing decision with three outcomes: sign with the answers in the contract, pilot with measures agreed first, or walk away while it is cheap.

  • Where the data goes

    Storage, processing, sub-processors, and whether your data trains their models.

  • Accuracy on your work

    Measured on a sample of your data, with a metric you chose.

  • Behaviour when wrong

    Fallbacks, human handover, and who reviews what.

  • What leaves with you

    Prompts, configurations, evaluations and data, exportable at exit.

  • The costs that grow

    Usage pricing, integration upkeep and the human review bill.

What we do

The signing decision

Weighed together, in writing — a strong demo compensates for none of them.

  • Sign, with the answers in the contract

    Each commitment survives the salesperson who made it.

  • Pilot, with measures agreed first

    A pilot without a pass bar is a longer demo.

  • Walk away while it is cheap

    Refusing to answer is itself an answer.

Accuracy

Why a benchmark score says little about accuracy on your own work

Vendor accuracy figures are produced on the vendor's evaluation set — data chosen, cleaned and sometimes tuned against, under conditions that flatter the product. Your invoices, your dialect mix on the phone lines, your abbreviations and your edge cases were not in it. The only accuracy number that predicts your experience is one measured on a sample of your own data, with a metric you defined, scored by your people. A capable vendor will run that evaluation before contract, because their product survives it; reluctance to be measured on your data is information about the product.

The second half of the accuracy question is what happens to the errors, because there will be errors, delivered fluently and with confidence. Ask the vendor to show a failure: what the system does when it is unsure, how a person is brought in, what the customer or employee sees during the handover, and what record the failure leaves for review. A vendor who can demonstrate a graceful failure has engineered for reality; one who steers back to the happy path has engineered for demos.

The questions

The questions to put to an AI vendor, phrased so the answers commit

Each question names the written commitment to walk away with. A verbal reassurance is the absence of one.

STAGE 01

1 · Where is our data stored and processed, and does it train your models?

Regions for storage and processing, every sub-processor in the chain, retention periods, and an explicit statement on whether your data — including prompts and outputs — is used for model training or improvement, with the opt-out mechanics if it is. Data-processing terms naming locations, sub-processors and the training-use position.

STAGE 02

2 · Will you be measured on our data before we sign?

A pre-contract evaluation on a representative sample of your data, with the metric, the sample and the pass bar agreed in advance and your people doing the scoring. An evaluation protocol in writing, run before signature.

STAGE 03

3 · Show us the system failing, and the human path out

A live demonstration of low-confidence behaviour, the handover to a person, and the audit trail an error leaves. Agree who on your side reviews errors and at what cadence. A documented fallback design and a review cadence with named owners.

STAGE 04

4 · If we leave in two years, what leaves with us?

Your data, your prompts and configurations, your evaluation sets and the interaction history — exportable in open formats, within a defined window, at a price fixed now. What cannot leave defines your real lock-in, whatever the sales deck says. Exit and portability terms enumerated in the contract.

STAGE 05

5 · Which of our costs grow with use, and by what rule?

The usage dimensions that drive the bill, the price protections at renewal, who pays when an integration breaks on either side's change, and the internal cost the vendor never quotes: the people who review the system's output. A cost model you drew yourselves, checked against the contract.

Our role

How we answer this page when we are the AI vendor in the room

  1. 01

    Measured before signed

    We evaluate on your data with your metric before contract, and we publish the protocol up front — because a system that cannot survive your sample should not survive your procurement.

  2. 02

    Failure engineered, not hidden

    Fallbacks, human handover and error review are part of what we build and demonstrate, not an appendix — the wrong answer a system gives is an operational event, and it needs a path.

  3. 03

    Exit terms we accept ourselves

    Your prompts, configurations, evaluations and data are yours to export in open formats, and we put that in the contract — a relationship held together by switching costs is not one we want to be in.

Whether any AI product earns a place in your operation is your call; these questions exist so the call is made on evidence, and we hold ourselves answerable to every one of them.

Common questions

What buyers ask

How do we find out whether an AI vendor trains its models on our data?+

Ask the question in writing and require the answer in the data-processing terms, not in an email from sales. The precise wording matters: ask whether customer content — inputs, prompts, files and outputs — is used for training, fine-tuning or "service improvement", because the last phrase is where training use often hides. Ask whether the position differs between the product's tiers, since consumer and enterprise tiers of the same product frequently differ. Ask how the commitment binds the vendor's own sub-processors, including any underlying model provider. And ask what changes on contract renewal. If the final contract does not state the position explicitly, the working assumption should be that your data is usable, whatever was said in the meeting.

What should an AI pilot measure before the contract is signed?+

Three things, all agreed before the pilot starts: quality, effort and behaviour at the edges. Quality means the accuracy metric you chose, measured on your data, against a pass bar written down in advance — a pilot whose success criteria are decided after the results exist will always succeed. Effort means what it actually took: the integration hours, the data preparation, the tuning cycles, because that effort recurs at every change. Behaviour at the edges means deliberately feeding the pilot the hard cases — the dialect calls, the malformed documents, the ambiguous requests — and watching the fallback path run, since the easy cases were never in question. A pilot that only exercises the demo path is a longer demo, at your expense.

Which AI running costs tend to grow after signature?+

Four, predictably. Usage-based pricing grows with adoption — the better the system works, the more it is used, and the bill scales with the success; ask what volume protections and renewal caps exist. Integration upkeep grows because both sides change: your systems evolve, the vendor ships new model versions with different behaviour, and someone pays for re-testing each time. Human review grows quietly, because the people checking the system's output are a real cost that appears in nobody's quote. And data preparation recurs, since the estate the system was tuned to keeps changing underneath it. None of these is a reason not to buy; each is a reason to model the cost over the contract's life rather than at its first month.

Pressure-test a vendor with us

A session applying these questions to a shortlist you are actually weighing — what the answers mean, and which gaps belong in the contract.

Start the conversation