Batch work without a second integration model

Submit independent chat turns to an Interlocute node, let the runtime use supported provider batch capabilities, and follow every item from submission through its final response.

Try Interlocute Read the Batch API

One runtime contract

Applications submit an array of independent chat turns to one node endpoint. The runtime owns provider-specific batch file formats, provider IDs, polling, result ingestion, usage reconciliation, and lifecycle normalization, so consumers do not need a separate OpenAI or Anthropic integration.

Built for real operational work

A batch has a stable ID, item counts, provider lifecycle state, cancellation state, billing reconciliation, and a paged result feed. Every item retains its caller-supplied custom ID and receives its own invocation record, so downstream systems can correlate success, failure, and usage precisely.

Same responses as chat

Completed batch items return the canonical invocation result used by the runtime's chat polling surface. Answer text, structured JSON, citations, provider metadata, errors, output pointers, and token usage retain one shape across synchronous, deferred, and batch execution.

Visible lifecycle, not a black box

The Interlocute console shows running batches with live indicators, progress by item outcome, provider status, polling heartbeat, timelines, and paged results. Successful items materialize their conversation thread only after a provider response is finalized; unsuccessful items remain observable without creating empty threads.

Frequently Asked Questions

Batch Processing

Is a batch one invocation?

No. A batch is the parent operation. Each submitted item receives its own invocation ID and terminal outcome, which keeps result correlation, token usage, and audit history per item.

Do I upload JSONL to Interlocute?

No. Runtime API callers submit ordinary JSON. JSONL is an internal provider-adapter format used only when a provider's native batch API requires it.

How do I retrieve results?

Poll the batch lifecycle endpoint while work is active, then read the paged results endpoint once results are ready. Each result is keyed by your custom ID and contains the same canonical invocation payload available through chat polling.

What happens when only some items fail?

A batch can finish with a mix of successful, failed, and canceled items. Item-level outcomes and error details remain visible, rather than collapsing the entire operation into one all-or-nothing result.

Can internal applications use batches too?

Yes. The same application service and runtime contracts are designed for both public API consumers and applications that extend Interlocute.

When is a thread created?

The runtime allocates stable invocation and message identifiers at submission time. It materializes a conversation thread only for a successful terminal item, so failed or canceled work does not leave empty conversation records.

Documentation

Batch API Reference Chat API Reference Invocation Polling

Related Features

Observability & Logging

Built-in logging, tracing, and token usage visibility for every node call. Understand exactly what your AI is doing and what it costs.

Commercial Control

Govern usage, enforce quotas, and maintain cost visibility across all your AI nodes. Built for teams that need accountability.

Protocol-First Runtime

A contract-first AI runtime that respects semantic boundaries. Interlocute separates protocol concerns from application logic.

Ready to build with Batch Processing?

Deploy your node in seconds and start using Batch Processing today.

Try Interlocute Read the Batch API