How do developers implement AI audit logs on Plugsky?

Key facts

Per-call fields	Request ID, model alias, token counts, latency, status and actor
Platform events	Authentication, key lifecycle and admin changes through audit export
Export targets	SIEM platforms including Splunk, Sentinel, QRadar and Datadog
Retention	Configurable prompt retention; log metadata by default
Identity	Scoped API keys per service make actor attribution meaningful
Correlation	Request IDs join model calls to application traces and user sessions
Endpoint status	Chat completions, streaming and function calling are live
Free tier	plugsky-micro and plugsky-lite free, no card, for building the logging path

TL;DR

• One structured event per call; treat AI calls like any other dependency.
• Propagate request IDs so model calls join your traces and user sessions.
• Default to metadata; redact or hash prompt content.
• Log key lifecycle events separately from application traffic.
• Sampling hides bugs — sample success, keep all failures.

How it works, step by step

1. Define an event schema for model calls: request ID, actor, model alias, tokens, latency, status and feature name.
2. Capture the request ID returned by the API and propagate it through your tracing context.
3. Redact or hash prompts and completions before logging; store full content only when explicitly required.
4. Emit events as structured JSON to your log platform, and export key lifecycle events from the workspace to the same pipeline.
5. Set sampling deliberately: keep all errors, timeouts and tool failures; sample successful calls if volume demands it.
6. Add dashboards for latency percentiles, error rates, token usage and refusal rates per feature.
7. Wire alerts on anomalies — sudden token spikes, unusual key usage or model drift after a deployment.

Try it yourself

Open the AI API key security checklist →

A schema that answers real questions

Start from the questions you will ask at 2 a.m.: which feature called the model, with which key, at what cost, and did it fail? One event per call, emitted as structured JSON, covers that: request ID, timestamp, actor or key ID, feature name, model alias, prompt and completion token counts, latency, status code and retry count.

Join on the request identifier the API returns and propagate it through your traces. Then a user complaint, a spike in errors and a specific model call all resolve to the same correlation key instead of three disconnected records.

Redaction and retention by default

Raw prompts are the riskiest thing you can log: they contain personal data, credentials accidentally pasted into context, and customer content. Default to metadata and store a hash or an application-side reference to the prompt. Where full capture is genuinely needed for evaluation, put it behind an explicit toggle, a short retention window and access control.

• Never log secrets: strip API keys and tokens before any content capture.
• Short windows: content retention measured in days, metadata per your observability policy.
• Access control: content logs visible to fewer people than metrics.

Separating platform and application events

Two streams matter. Application events come from your code: calls, tools, retries, user context. Platform events come from the workspace: key creation, rotation, permission and retention changes. Both belong in the same pipeline so a key rotation is visible next to the traffic it affects.

Sampling deserves discipline. Successful high-volume calls can be sampled, but failures, timeouts and tool errors should never be. When a regression ships, the difference between diagnosing it in minutes and in days is whether the failing events were kept.

Honest comparison

Developer concern	Plugsky	Typical model API	Self-hosted stack
Request identifiers	Returned and correlatable with your traces	Usually available	You generate them
Platform audit events	Key, admin and authentication events exportable	Varies	Your implementation
SIEM export	Splunk, Sentinel, QRadar, Datadog	Limited	You build pipelines
Prompt retention	Configurable and minimisable	Often opaque	Fully yours
Model coverage	30+ models behind one endpoint with consistent events	Per-model variation	One model per deployment
Setup effort	Schema plus client hooks	Similar	Substantially more

Frequently asked questions

Should we log raw prompts?

Not by default. Log metadata plus a hash or application reference, and enable content capture only for evaluation with short retention and restricted access.

What is the single most useful field?

The request ID. It joins the model call to your traces, the user session and the platform-side event, which turns debugging into correlation rather than guesswork.

How do we attribute calls to a user or feature?

Issue scoped API keys per service or environment and log the feature name alongside the key ID. That distinguishes abusive features from popular ones.

Do we need a separate AI observability vendor?

Not necessarily. A structured event per call plus your existing log platform covers most needs; add tooling when evaluation and prompt versioning demand it.

What should trigger an alert?

Token spikes per key or feature, latency regressions, rising error or refusal rates, and unusual model usage after a deployment. Alert on changes you can act on.

How do we correlate tool calls with model calls?

Emit the same request ID for the parent model call and each tool invocation, and include a step index so the sequence is reconstructable.

Can we try this on the free plan?

Yes. plugsky-micro and plugsky-lite are free with no card, which is enough to build and validate the logging path before scaling.