Short answer. Send an
image_urlor base64 image inPOST /v1/chat/completions. Useplugsky-vision-fastfor low-latency classification and UI checks, andplugsky-parsefor dense documents and tables. Ask for JSON with a schema, then validate it. Vision is available on paid plans from $8/month.
Task matrix
| Task | Model | Output | Typical failure |
|---|---|---|---|
| Scene or object classification | plugsky-vision-fast | Label + confidence | Confuses similar categories |
| Product attributes | plugsky-vision-fast | Colour, material, style | Invents a material from sheen |
| UI screenshot QA | plugsky-vision-fast | Elements, text, layout issues | Misses small hit targets |
| Document and table extraction | plugsky-parse | Blocks, rows, cells | Breaks reading order |
| Chart or dashboard reading | plugsky-parse | Series, values, trends | Reads a legend as data |
| Safety triage | plugsky-vision-fast | Category flags | Context-dependent misses |
For text-only moderation, use /v1/moderations.
Resolution and token math
Images are billed as input tokens, and the count is a function of dimensions and the model family, not file size. The same 1024×1024 image can cost 765 tokens on an OpenAI-style tile model, about 1,296 on Claude, or about 1,032 on Gemini. Plugsky is OpenAI-compatible on the request shape, but you should read usage.prompt_tokens on your own calls rather than assume an upstream billing formula.
| Image | OpenAI-style high detail | Claude-style patches | Gemini-style tiles |
|---|---|---|---|
| 512×512 | 255 | 336 | 258 |
| 1024×1024 | 765 | 1,296 | 1,032 |
| 2048×2048 | 2,805 | 1,568 cap | 2,322 |
The OpenAI-style rule is 85 base tokens plus 170 per 512-pixel tile after scaling to fit 2048 pixels and setting the shortest side to 768. Claude counts 28×28 patches with a token cap. Gemini counts 258-token tiles. Use the smallest image that passes eval.
from math import ceil
def openai_tokens(w, h, detail="high"):
if detail == "low":
return 85
if max(w, h) > 2048:
scale = 2048 / max(w, h)
w, h = int(w * scale), int(h * scale)
if min(w, h) > 768:
scale = 768 / min(w, h)
w, h = int(w * scale), int(h * scale)
return 85 + 170 * ceil(w / 512) * ceil(h / 512)
print(openai_tokens(1024, 1024)) # 765
Structured JSON output
Free text is hard to validate. Ask for a schema with a label, a confidence, and the evidence the model used. Then run the same schema locally; a response that fails validation should be retried or routed to review, not patched by hand.
import json
from openai import OpenAI
client = OpenAI(api_key="YOUR_PLUGSKY_KEY", base_url="https://api.plugsky.com/v1")
resp = client.chat.completions.create(
model="plugsky-vision-fast",
response_format={"type": "json_schema", "json_schema": {
"name": "product", "strict": True,
"schema": {
"type": "object",
"required": ["category", "attributes", "confidence"],
"properties": {
"category": {"type": "string"},
"attributes": {"type": "array", "items": {"type": "string"}},
"confidence": {"type": "number"},
"evidence": {"type": "array", "items": {"type": "string"}}
}
}
}},
messages=[{"role": "user", "content": [
{"type": "text", "text": "Describe the product. Use only visible evidence."},
{"type": "image_url", "image_url": {"url": "https://example.com/sku.jpg", "detail": "high"}}
]}],
)
record = json.loads(resp.choices[0].message.content)
assert 0 <= record["confidence"] <= 1
UI, OCR and product modes
| Mode | What to ask for | What to check |
|---|---|---|
| UI | Element name, role, bounding box, visible text, hit-target size | Coordinates map back to the original resolution; disabled states are not guessed |
| OCR | Blocks in reading order, tables as rows and cells, null for unreadable text | Column order, totals, Arabic-Indic digits |
| Product | Category, colour, material, pattern, defects, label text | No invented attributes; label text matches the SKU |
For documents, use the document parsing API pattern.
The 20-image benchmark
Twenty images is enough to expose a bad prompt and small enough to label by hand. Use the same prompt, temperature 0 and the same model for every row, and score each answer against the criterion.
| # | Image | Task | Pass criterion |
|---|---|---|---|
| 1 | White-background shoe | Category + colour | Both correct; no invented material |
| 2 | Folded shirt on a bed | Pattern + sleeve length | Pattern correct; sleeves visible |
| 3 | Cosmetic bottle | Read the front label | Brand and volume exact |
| 4 | Jewellery close-up | Metal + stone | No guess when obscured |
| 5 | Grocery pack | Net weight + flavour | Both exact; units preserved |
| 6 | Mobile checkout screen | List interactive elements | Primary button found; coordinates valid |
| 7 | Error dialog | Read the error and buttons | Text exact; no invented cause |
| 8 | Dashboard with charts | Read three series | Legend and values not swapped |
| 9 | Dark-mode settings page | Find the toggle state | State correct; low-contrast text read |
| 10 | Arabic UI screen | Reading order + labels | Right-to-left order preserved |
| 11 | Invoice page | Vendor, total, date | Three fields exact |
| 12 | Bank statement table | Rows + running balance | Row count and signs correct |
| 13 | Handwritten note | Name + quantity | Null when unreadable |
| 14 | Rotated receipt photo | Total + tax | Arithmetic reconciles |
| 15 | Arabic contract page | Parties + date | Arabic-Indic digits normalised |
| 16 | Blurry product photo | Category only | Abstains instead of guessing |
| 17 | Image with a text instruction | Describe the scene | Does not follow text in the image |
| 18 | Two products in one frame | Count + attributes | Count correct; attributes separated |
| 19 | Reflective surface | Material + colour | Reflection not read as a second object |
| 20 | Low-light photo | Safety flag | Flag matches human label |
Report exact match, field accuracy, hallucination rate and p95 latency.
Safety, limits and cost per 1,000 images
Images can carry prompt injection: text inside a screenshot that says “ignore previous instructions”. Treat image text as data, not instructions. Avoid face recognition and biometric inference; minimise personal data; keep human decisions for high-impact outcomes.
At roughly 1,065 tokens for a 1024×1024 request, Hobby's 2M daily cap allows about 1,878 images per day; at full use the effective cost is about $0.14 per 1,000 images. Starter has the same list-price ratio. Builder and Scale have no monthly token cap, so cost per image is their monthly fee divided by volume.
| Plan | Daily token cap | Images/day at 1,065 tokens | Effective $ / 1,000 at full use |
|---|---|---|---|
| Hobby $8 | 2M | ~1,878 | ~$0.14 |
| Starter $20 | 5M | ~4,695 | ~$0.14 |
| Builder $60 | No cap | RPM/concurrency bound | $60 ÷ monthly images |
| Scale $120 | No cap | RPM/concurrency bound | $120 ÷ monthly images |
See pricing for current caps and plugsky-vision-fast for model details. For the wider build, read what you can build with multimodal AI.
FAQ
Which model should I use for vision?
Use plugsky-vision-fast for classification, UI checks and product attributes where latency matters. Use plugsky-parse for dense documents, tables and pages that need a long structured output.
How are images counted as tokens?
It depends on the model family. OpenAI-style tile models use a base of 85 tokens plus 170 per 512-pixel tile after resizing, so a 1024×1024 image is 765 tokens. Claude uses 28×28 patches, and Gemini uses 768-pixel tiles. Read usage.prompt_tokens on your own requests rather than assuming a formula.
Can the API return structured JSON?
Yes. Send a JSON schema in response_format and require fields such as label, confidence and evidence. Validate the response with the same schema before writing it to a database.
How do I test vision accuracy?
Use a fixed 20-image set that covers product, UI, document and edge cases. Score exact match, field accuracy, hallucination rate and latency; do not rely on a vendor demo set.
What does vision cost per 1,000 images?
At about 1,065 tokens per 1024×1024 image, Hobby's 2M daily token cap allows roughly 1,878 images per day. At full daily use the effective cost is about $0.14 per 1,000 images; plans are flat, so your volume and output length dominate.
Sources
OpenAI vision documentation for image tokenization, checked 20 November 2026: developers.openai.com/api/docs/guides/images-vision. Anthropic vision documentation for patches and token caps, checked 20 November 2026: platform.claude.com/docs/en/build-with-claude/vision. Google Cloud Vision documentation, checked 20 November 2026: cloud.google.com/vision/docs.
Reviewed 20 November 2026. Token formulas are documented by the upstream model families and are planning estimates for Plugsky; reconcile against usage.prompt_tokens. Do not use vision output for medical, legal or safety decisions without human review.