Key facts
| Minimum schema | Request ID, actor, model alias, tokens, latency and status |
| Platform events | Authentication, key lifecycle and admin changes available from Plugsky audit export |
| Cost visibility | Token counts per feature reveal which part of the product is expensive |
| Debugging | Request IDs join model calls to application traces and user reports |
| Default capture | Metadata-first; avoid storing raw prompts until a purpose exists |
| Free tier | plugsky-micro and plugsky-lite free, no card, for building the logging path |
| SIEM export | Available when a customer asks; Splunk, Sentinel, QRadar and Datadog |
| Status | Audit export is live; assistants and responses endpoints are coming soon |
TL;DR
- Log request ID, actor, model, tokens and status — nothing more is required early.
- Use logs to see cost per feature before your first invoice surprises you.
- Do not store raw prompts without a reason and a retention plan.
- Lean on platform-side events for keys and admin changes.
- Add retention tiers and exports when your first enterprise deal asks.
How it works, step by step
- Emit one structured event per model call with the minimum schema and write it to your existing log platform.
- Propagate the request ID through your traces so debugging joins model calls to user sessions.
- Attribute calls to a feature and a user or service identity; scoped keys per environment make this accurate.
- Track token counts per feature to find expensive paths while usage is still small.
- Avoid raw prompt logging; store a hash or reference until a concrete purpose exists.
- Use platform audit export for key and administrative changes rather than building a control-plane log system.
- Review the event schema monthly; add fields when real questions go unanswered, not speculatively.
Try it yourself
Open the AI API key security checklist →
The minimum viable audit trail
Early-stage logging has one job: let you understand and debug production. One event per model call, written as structured JSON, covers it: request ID, timestamp, actor or key ID, feature name, model alias, prompt and completion tokens, latency and status. If you can answer which feature is slow, which is expensive and what failed for this user, you have enough.
Plugsky returns request identifiers you can correlate and keeps platform-side events for authentication, key lifecycle and administrative changes. That means you get the control-plane record without building one, and you can spend the engineering time on your product.
Cost and privacy from day one
Token counts per feature are the cheapest cost-control tool you will ever build. They show which customer segment or workflow is driving usage before the invoice arrives, and they make routing decisions — smaller model here, frontier model there — evidence-based rather than speculative.
- Metadata only: raw prompts carry personal data and secrets; log references instead.
- Short horizons: keep debugging windows short and metrics longer.
- Scoped keys: one per environment and service keeps attribution honest.
When customers start asking
Enterprise buyers eventually ask what you record, where it is stored and whether they can export it. By then you want three things: a documented event schema, a retention statement and an export path. Plugsky audit export supports common SIEM platforms, which covers the platform side; your application events need their own answer.
Do not build retention tiers, webhooks and tenant-visible views before someone pays for them. Add them when the first serious security questionnaire arrives — but keep the schema clean enough that adding those surfaces later is presentation work, not archaeology.
Honest comparison
| Startup decision | Log the minimum | Log everything | Log nothing yet |
|---|---|---|---|
| Engineering cost | Hours | Days to weeks | Zero now |
| Debugging value | High | High | None |
| Cost visibility | Yes, per feature | Yes | No |
| Privacy exposure | Low | High if content captured | Minimal |
| Enterprise readiness | Schema in place | Ahead, but over-built | Late scramble |
| When to expand | At first security review | Already done | Before first enterprise deal |
Frequently asked questions
What is the absolute minimum to log?
Request ID, actor, model, token counts and status. If you can trace a complaint and attribute cost, you have the essentials.
Should we log prompts?
Not by default. Prompts can contain customer data and secrets. Log a hash or a reference, and revisit only when an evaluation need justifies content capture.
How do we track AI cost per feature?
Emit token counts with a feature label, then sum by feature. It is the fastest way to spot expensive paths before they show up on an invoice.
Do we need a logging vendor?
Usually not early on. Your existing log platform plus the platform audit export covers debugging and basic compliance questions.
What about key and admin events?
Plugsky audit export already records authentication, key lifecycle and administrative changes, so you do not need to instrument the control plane yourself.
When do we need retention tiers?
When a customer contract requires a specific retention period, or when your privacy policy demands shorter windows. Build it then, not speculatively.
How do we start?
The free plan with plugsky-micro and plugsky-lite, no card, is enough to build and test the logging path. Scale when usage justifies it.