TopolearnScreenshots, photos, receipts and charts are common inputs, and generated images are a common output.
Current OpenAI models can look at images. You put the image in a user message as a content part next to your text:
{"type": "input_text", "text": "..."}for the question.{"type": "input_image", "image_url": ...}for the picture. The value can be a public URL, a data URL (the image bytes encoded as Base64 text, startingdata:image/png;base64,...), or you can pass afile_idfrom an uploaded file.
The detail setting trades accuracy for cost: low is cheap and coarse, high sees more, original keeps full resolution for reading small text or for computer use, and auto (default) lets the model decide. Images are billed as input tokens based on their size and detail.
Supported formats are PNG, JPEG, WEBP and non-animated GIF. Very large images can be rejected rather than shrunk, so resize them yourself.
For generating images there are two routes:
- Add the
image_generationtool to a normal Responses call; the model decides when to draw and returns the image as Base64 data. - Call the separate Images API with a GPT Image model when you only need images.
Image model names change quickly (several were retired in 2026), so check the docs for the current one. Vision has limits too: small text, rotated images and exact counts can be misread, so verify numbers that matter.
Example
import base64
from openai import OpenAI
client = OpenAI()
with open("receipt.jpg", "rb") as f:
b64 = base64.b64encode(f.read()).decode()
r = client.responses.create(
model="gpt-6-astra",
input=[{
"role": "user",
"content": [
{"type": "input_text", "text": "What is the total on this receipt?"},
{"type": "input_image",
"image_url": f"data:image/jpeg;base64,{b64}",
"detail": "high"},
],
}],
)
print(r.output_text)Remember
- Images go in as input_image parts: URL, data URL or file_id
detail(low/high/original/auto) trades cost for accuracy- Generate images with the image_generation tool or the Images API
Watch out for
- Sending huge images at high detail for a task that low detail handles, multiplying cost.