Batch

Batch processing runs many LLM requests asynchronously, with up to a 24-hour completion window. The workflow uses the Files API for JSONL input, the Batch API to create a job, and another file for output. It is intended for offline work such as classification, extraction, evaluation, and document processing; use the synchronous endpoints when a user is waiting for an answer.

WARNING

Batch creation requires at least $50 of available balance. This protects a job that may consume a large amount of credit before its final cost is known.

Workflow

Create a JSONL file with one request per line. Each line contains custom_id, method, url, and body; use /v1/chat/completions or /v1/responses as the URL.
Upload it with purpose=batch.
Create a batch with input_file_id and the same endpoint.
Poll the batch until it is completed, then download output_file_id content.

The batch file is newline-delimited JSON, not one large JSON array. Every line must use POST, and all requests in one batch must use the same model and endpoint.

{"custom_id":"classify-001","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Classify: The package arrived today."}],"max_tokens":20}}
{"custom_id":"classify-002","method":"POST","url":"/v1/chat/completions","body":{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Classify: The invoice is overdue."}],"max_tokens":20}}

curl -X POST "https://api.llmapi.ai/v1/files" \
  -H "Authorization: Bearer $LLM_API_KEY" \
  -F "purpose=batch" \
  -F "file=@requests.jsonl"

curl -X POST "https://api.llmapi.ai/v1/batches" \
  -H "Authorization: Bearer $LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

curl "https://api.llmapi.ai/v1/batches/BATCH_ID" \
  -H "Authorization: Bearer $LLM_API_KEY"

curl "https://api.llmapi.ai/v1/files/OUTPUT_FILE_ID/content" \
  -H "Authorization: Bearer $LLM_API_KEY"

When the batch is complete, each output line contains the original custom_id and either a successful endpoint response or an error:

{
  "id": "batch_req_1",
  "custom_id": "classify-001",
  "response": {
    "status_code": 200,
    "body": { "choices": [{ "message": { "content": "delivery" } }] }
  },
  "error": null
}

Requests can finish in a different order than they were submitted. Always use custom_id to join results back to your source records. If individual lines fail, the batch can still complete; inspect error_file_id as well as output_file_id.

Batch and file operations

POST /v1/files uploads a JSONL input or media file and returns a file_id.
GET /v1/files lists files available to the project.
GET /v1/files/{file_id} retrieves file metadata.
GET /v1/files/{file_id}/content downloads file bytes or JSONL content.
DELETE /v1/files/{file_id} removes a file that is no longer needed.
POST /v1/batches creates a job; GET /v1/batches lists jobs and GET /v1/batches/{batch_id} returns status.
POST /v1/batches/{batch_id}/cancel requests cancellation. Cancellation is best-effort and may not stop requests already submitted upstream.

Notes

Batch supports OpenAI and Google Vertex AI providers. All lines in one batch must use the same model and endpoint.
The supported target endpoints are /v1/chat/completions and /v1/responses. See Responses for their body formats.
The completion_window is currently 24h; statuses include validating, in_progress, finalizing, completed, failed, expired, and cancellation states.
Upload purpose=audio_batch for audio files or purpose=video_input for video files referenced by multimodal JSONL requests. For the full flow, see Batch Processing.
Batch input, output, error, and media files are project-scoped. Delete files you no longer need with DELETE /v1/files/{file_id}.
Batch jobs do not create videos or images and do not target the dedicated audio endpoints; use the dedicated APIs for those operations.