Who this questionnaire is for and how to use it
This questionnaire is for the people who sign off on AI software: CTOs, security and privacy leads, and procurement. It covers the questions a standard software security review tends to miss when a product sends your data to a language model, lets an agent act in your systems or produces answers your staff will rely on.
Traditional questionnaires ask about hosting, encryption and certifications. Those still matter, so keep asking them. AI products add new questions: which model providers see your data, whether your data is used for training, what the product logs about prompts and outputs, how quality is measured and what happens when the model gets something wrong. The download groups 48 questions into eight areas, each with a note on why it matters.
How to use it:
- 01Scope the use case first. Write down what the product will do, which data it touches and whether it takes actions or only makes suggestions. A tool that drafts internal summaries needs a lighter review than an agent that updates customer records.
- 02Send the questions early. Share them before the final commercial round, so the answers shape the shortlist instead of arriving after the decision is made.
- 03Ask for evidence. For each answer, ask for the document, setting or sample that supports it: a sub-processor list, a data processing agreement, an eval report, an extract from an audit log.
- 04Score consistently. Use the scoring in this guide so two vendors, or a vendor and your own team, are compared on the same terms.
- 05Keep the answers. They belong in the contract file and become the baseline for your next review.
What good answers look like
A good answer is specific, checkable and consistent with the contract. Vague reassurance is common when AI products are sold, so it helps to know what a strong answer sounds like before the responses come in.
| Question | Weak answer | Strong answer |
|---|---|---|
| Is our data used to train models? | We take privacy very seriously. | No. Our agreements with model providers exclude training on customer data. Here is the clause and the account setting. |
| Which model providers process our data? | We work with leading AI providers. | A named list with the processing region for each, included in the sub-processor list, with notice before changes. |
| How do you measure answer quality? | Our AI is highly accurate. | A test set per use case, the pass rates from the last release and the rules that block a release. |
| Can we see what the AI did? | Logs are available on request. | Every run is logged with inputs, sources, tool calls and outputs. You can export the logs and set retention. |
| What happens if we leave? | We can discuss that at the time. | Export in a documented format within an agreed period, then deletion with written confirmation. |
Strong answers also admit limits. A vendor that tells you which cases its product handles poorly, and how it detects them, is usually easier to work with than one that claims to handle everything. When a vendor says it cannot share something, ask for a call with its security or engineering lead under NDA.
Pay attention to who answers. If every response comes from sales and none from engineering or security, the answers describe what the vendor intends. Ask for the person who would handle an incident on your account to review the answers in the sections that matter most to you.