Key facts
| Logged fields | Timestamp, model, tokens, latency, status, key ID, project, region, request ID, user tag |
| Export targets | Datadog, Splunk, Grafana, New Relic, OpenTelemetry, SIEM |
| Dashboard views | Usage and analytics pages with per-model and per-key breakdowns |
| Usage API | GET /v1/plugsky/usage is documented as coming soon |
| Token counts | usage.prompt_tokens and usage.completion_tokens in every response |
| Billing model | Flat monthly plans with fair-use usage; token counts do not drive billing |
| Tracing | Request ID correlates a client call with platform logs |
| Product status | Live (dashboard and logs); usage API coming soon |
TL;DR
- Every request is logged with enough fields to attribute and debug it.
- Export logs to your existing APM or SIEM instead of building a new pane of glass.
- Track latency percentiles, error rates and model distribution — not just volume.
- Use per-key attribution to find the workload behind a spike.
- The usage API is coming soon; dashboard analytics and log export work today.
How it works, step by step
- Standardise a request ID and user tag on every client call for correlation.
- Export Plugsky logs to your APM or SIEM and confirm fields arrive intact.
- Build dashboards for p50/p95 latency, error rate and model distribution.
- Break usage down by key and project to attribute traffic to services.
- Alert on 429 and 5xx rates and on time-to-first-token regressions.
- Review token usage per response as a prompt-efficiency signal, not a billing metric.
- Revisit thresholds quarterly as workloads and models change.
Try it yourself
Open the LLM token calculator →
What Plugsky logs, and why each field matters
- Timestamp, request ID, status: the spine of any trace — join client logs to platform logs with the request ID.
- Model and region: failover and routing can change which model answered; without this field, performance shifts are unexplainable.
- Tokens and latency: prompt and completion tokens support prompt-efficiency work; latency feeds the user-experience SLO.
- Key ID, project and user tag: attribution. Per-key breakdowns turn "the API is slow" into "the batch worker on key 3 is saturating embeddings".
Logs export to Datadog, Splunk, Grafana, New Relic, OpenTelemetry and general SIEM pipelines, so observability stays in the tools your on-call already uses.
The metrics that actually predict trouble
Because self-serve plans are flat-rate, token spend is not the alarm to build. Track time-to-first-token for streaming, end-to-end latency percentiles, 429 and 5xx rates, and the distribution of models answering requests. A rising p95 with a stable p50 usually means queuing or retry storms; a model-distribution shift after a config change usually explains a quality regression. Pair platform metrics with an application-level quality signal — thumbs, eval scores or task success — because everything can look healthy while answers get worse.
Per-key attribution and governance
Scoped keys are the unit of attribution. Give each service, environment and agent fleet its own key, then build a dashboard per key: requests, error rate, latency, models used. This makes capacity conversations concrete — you can see whether chat, embeddings or an agent loop is driving load — and it bounds incidents, because revoking one key is a contained action. For enterprise deployments, audit logs capture key creation, rotation, scope changes and deletion with actor, timestamp, IP and request-body hash, exportable to your SIEM.
Closing the loop
Observability only pays off when it changes behaviour. Wire alerts to on-call with clear runbooks: 429s mean back off and consider a tier change; sustained 5xx means check the status page before code; latency regressions trigger a model or prompt review. Keep a monthly review of per-key usage, retired services and stale keys. The GET /v1/plugsky/usage endpoint is coming soon for programmatic reporting; until then, the dashboard plus log export covers production needs. See /docs for the current field list and export setup.
Honest comparison
| Capability | Plugsky | Logging only in your app | Third-party LLM gateway |
|---|---|---|---|
| Request logging | Platform-side with model, tokens, latency, region | Only what you remember to log | Varies by vendor |
| Attribution | Key ID, project and user tag | Custom | Usually key-based |
| Exports | Datadog, Splunk, Grafana, New Relic, OTel, SIEM | Your pipeline | Vendor integrations |
| Dashboard | Usage and analytics views | Build your own | Vendor dashboards |
| Failover visibility | Model field shows who answered | Guesswork | Varies |
| Usage API | Coming soon | Not applicable | Varies |
Frequently asked questions
What does Plugsky log per request?
Timestamp, model, token counts, latency, status, key ID, project ID, region, request ID and an optional user tag — enough to attribute and debug traffic without application-side instrumentation.
Can I export logs to my own tools?
Yes. Logs export to Datadog, Splunk, Grafana, New Relic, OpenTelemetry and SIEM pipelines, so you can keep one observability stack.
Are token counts used for billing?
Not on self-serve plans — those are flat monthly with fair-use usage. Token counts are returned for observability and prompt-efficiency work.
Is there a usage API?
GET /v1/plugsky/usage is documented as coming soon and will return per-key, per-model, per-day usage. The dashboard and log exports cover reporting today.
How do I attribute traffic to a service?
Use a separate scoped key per service and environment, then group by key ID in exports or the dashboard. Per-key views make spikes easy to trace.
What should alert on-call?
429 and 5xx rates, latency percentiles (especially time-to-first-token for streaming), and abrupt model-distribution changes after config updates.
Does export include request and response bodies?
The documented log fields cover metadata and usage rather than full payloads. If you need prompt-level tracing, log it on your side and correlate by request ID.
How long are logs retained?
Retention details are documented in the platform docs and can be extended by enterprise arrangements. Confirm current terms for your compliance requirements.