Feature · security

AI vendor security review: what to send

Most security questionnaires are generic. An AI vendor review has to cover the model supply chain, the data path and the change process. Here is the packet that answers the questions a reviewer will actually ask.

Short answer. Send a review pack, not a folder of logos. The minimum is an architecture diagram, a data-flow map, a subprocessor list, a current penetration-test summary, an incident SLA, a model-change notice policy and an evidence index keyed to each control. Make every answer point to a dated artifact.

The 120-question packet in eight sections

A 120-question packet is long enough to cover the AI-specific risks and short enough to finish. Split it into eight sections, assign an owner for each and require an evidence link for every “yes”. The counts below sum to 120; adjust them to your risk tier, but do not drop the model supply chain or data-flow sections.

SectionQuestionsEvidence you should receive
Security program and governance15Policies, risk assessment, ownership, training, exception process
Architecture and data flow20Diagram, trust boundaries, regions, egress, key management
Identity, access and keys15SSO, MFA, RBAC, SCIM, key rotation, break-glass
Data handling and retention20Classification, payload logging, retention, deletion, backups
Model supply chain15Provider list, model changes, evals, deprecation, rollback
Resilience and incidents15Status page, incident SLA, RCA, DR test, capacity
Privacy and legal10DPA, subprocessors, residency, BAA, data subject requests
AI-specific governance10Acceptable use, evals, red teaming, human review, model cards

Architecture, data flow and the evidence index

The diagram should show the browser or service, API gateway, authentication, router, every model provider, logging, vector storage, SIEM and key management. Draw trust boundaries and label regions. Mark where data is encrypted, decrypted, persisted and leaves your control. A diagram that stops at “AI API” is not a review.

The data-flow map is the same picture with data classes attached. For each hop, record PHI, PII, confidential or public; encryption; retention; subprocessor; country; and training use. Then build the evidence index: one row per control, the document that proves it, the date, owner and freshness. An index turns a questionnaire into an audit trail.

Control areaEvidenceFreshness
Independent assuranceSOC 2 Type II report, scope and period; ISO certificates if claimedWithin 12 months
Penetration testingSummary, scope, tester, findings, remediation, retest letterWithin 12 months
Access controlSSO configuration, RBAC matrix, latest access reviewWithin 90 days
Data protectionDPA, BAA if applicable, retention schedule, deletion testCurrent contract
ResilienceIncident SLA, status history, last DR test, RCA sampleWithin 12 months
Model changeNotice policy, deprecation policy, eval and rollback recordCurrent release

Subprocessors and the model supply chain

List every subprocessor that can access customer data, its service, data classes, location and binding contract. For an AI API, include hosting, observability, support tooling and model providers. The current Plugsky Terms name opencode.ai, NVIDIA NIM and others; the authoritative list should come from the DPA or trust package and be dated.

The model supply chain is what generic questionnaires miss. Ask how a model is selected, whether the provider can change the model behind an alias, how changes are evaluated, what notice you get, how deprecation works and whether you can pin a version. Ask for the eval set and threshold, not just a model card. If the vendor cannot answer, add your own regression tests.

Pen-test evidence, incident SLA and model-change notice

A penetration-test badge is not evidence. Ask for a summary with scope, date, tester, methodology, critical and high findings, remediation status and a retest letter. The full report can stay under NDA; the summary should show whether the test covered the API, console, identity path and data stores. Ask when the next test is scheduled.

Separate the uptime SLA from the security-incident SLA. Uptime credits do not tell you how fast a suspected breach is escalated. Ask for the notification clock, escalation contacts, evidence preservation, RCA commitment and cooperation with regulators and customers. The current Plugsky Terms publish uptime credits and a 30-day claim window, but not a security-incident clock; get that in the DPA or a security addendum.

Model-change notice deserves its own clause. Ask for the notice period for an update and for deprecation, whether the change can be opt-in, how evaluations are run and what happens if your regression tests fail. A minor update may need 30 days; a major swap may need 60 or 90 days, a migration window and rollback. Write it into the contract, not just the trust page.

How to run the review in two weeks

Day 1: scope the review, classify the data and agree the decision criteria. Day 2: send the packet and evidence-index template. Day 4: receive the pack; check every answer has an artifact. Day 6: interview the security and platform owners; walk the architecture and data flow. Day 8: test one control end to end, such as deprovisioning or deletion. Day 10: draft findings with severity and owner. Day 12: agree remediations and risk acceptance. Day 14: sign off or escalate. Keep the pack, notes and decision in one place so the next review starts from evidence.

Use the AI vendor security questionnaire to collect answers, the vendor risk assessment checklist to structure the decision, the enterprise AI security checklist for controls, and the security overview for the vendor side. For the commercial context, see Enterprise.

FAQ

How many questions should an AI vendor security review have?

A 120-question packet is a practical default: 15 program, 20 architecture and data flow, 15 identity, 20 data handling, 15 model supply chain, 15 resilience, 10 privacy and 10 AI governance. Adjust by risk tier, but keep the model supply chain and data-flow sections.

What is an evidence index?

A table that maps each control to the document that proves it, with a date, owner and freshness date. It turns a questionnaire into an audit trail and makes the next review faster.

Is a SOC 2 report enough?

No. SOC 2 covers controls relevant to security, availability, processing integrity, confidentiality or privacy, but the scope and period vary. Ask for the report, the scope, the period and the bridge letter, then test the controls that matter to your data flow.

What should a penetration-test summary include?

Scope, date, tester, methodology, critical and high findings, remediation status and a retest letter. The full report can be under NDA; a badge or a one-line claim is not enough.

How much notice should a model change require?

It depends on the change. Ask for a notice period, an opt-in or pinning option, an evaluation record, a migration window and a rollback path. Put the commitment in the contract, not only on a trust page.

Sources

Primary sources checked 20 November 2026. Provider terms, limits and prices change; verify the current version before you commit.

  • NIST Cybersecurity Framework 2.0 — organizes cybersecurity outcomes into Govern, Identify, Protect, Detect, Respond and Recover, which maps cleanly to a vendor review packet.
  • AICPA SOC suite of services — describes SOC examinations and the trust services criteria for security, availability, processing integrity, confidentiality and privacy.

Reviewed 20 November 2026. Plugsky's public security pages describe its controls and programs; request the current reports, scope and penetration-test summary under NDA. Do not rely on a badge or a logo.