Live Voice Sessions

GET /v1/live/sessions opens a full-duplex voice session: you stream microphone audio up and receive model audio back on the same socket, with the model handling turn taking, interruption, and transcription. It is the right endpoint for phone agents and voice assistants, where the streaming STT and TTS endpoints only cover one direction at a time.

INFO

Gemini Live models speak Google's own protocol on a separate endpoint — see Gemini Live.

The gateway is a byte-faithful relay. You speak the provider's native Live event protocol; the gateway authenticates the connection, resolves the upstream credential, rewrites only the first frame, and passes every other frame through untouched in both directions — so provider features the gateway does not model still reach you.

Connect

wss://api.llmapi.ai/v1/live/sessions

Authenticate with the same API key as the HTTP endpoints:

Authorization: Bearer $LLM_API_KEY

Browsers cannot set headers on a WebSocket, so the key may also be passed as a token query parameter:

const ws = new WebSocket(
  `wss://api.llmapi.ai/v1/live/sessions?token=${encodeURIComponent(API_KEY)}`
);

WARNING

A key in a query string is visible to anything that logs URLs. Use it only from a browser, and prefer a short-lived key minted for the session.

Session lifecycle

1. Client opens the WebSocket
2. Client sends: {"type":"session.start","session":{"model":"gpt-live-1","delegation":{"type":"client"}}}
3. Server sends: {"type":"session.started", ...}
4. Client sends: {"type":"session.input_audio.append","audio":"<base64 PCM16>"}   ← repeatedly
5. Server sends: {"type":"session.output_audio.delta","audio":"<base64 PCM16>"}   ← repeatedly
   Server sends: transcript, commentary, and thinking deltas as they are produced
   Server sends: {"type":"session.usage.updated","usage":{"seconds":42.5}}        ← periodic
6. Client sends: {"type":"session.close"}
7. Server sends: {"type":"session.closed","reason":"close_requested","usage":{"seconds":61.2}}

The first frame must be session.start, and it must arrive within 60 seconds of the upgrade — the gateway needs the model id before it can authorize the session or pick a credential.

`session.start`

{
  "type": "session.start",
  "event_id": "e1",
  "session": {
    "model": "gpt-live-1",
    "delegation": { "type": "client" },
    "instructions": "You are a concise support agent.",
    "audio": { "output": { "voice": "marin" } }
  }
}

Field	Required	Description
`type`	Yes	Must be `session.start`.
`session.model`	Yes	Live model id, such as `gpt-live-1`.
`session.delegation`	No	Must be `{"type":"client"}` when present; omitting it selects client delegation.
`event_id`	No	Echoed back on any error about this frame, for correlation.
everything else	No	Voice, instructions, turn detection, and other provider options pass through.

The gateway rewrites exactly two things in this frame before forwarding it: the model id becomes the provider's wire id (a gpt-live-1 session may run on either OpenAI or Azure, and the routing is invisible to you), and store is forced to false so no transcript is retained on the provider account. Every other field you send survives verbatim.

INFO

Only client delegation is supported — your application performs any backend reasoning and feeds the results into the session. Provider-hosted Responses delegation, WebRTC, sideband connections, and SIP are not served.

Audio format

Audio travels as base64 in JSON frames, not as binary WebSocket frames:

Encoding: PCM16LE (signed 16-bit little-endian)
Channels: mono
Sample rate: 24 kHz
Frame size: 20–100 ms per session.input_audio.append

Client frames

Frame	Behaviour
`session.start`	First frame only; a second one is rejected
`session.update`	Relayed, unless it switches delegation
`session.input_audio.append`	Relayed — base64 PCM16LE mono 24 kHz
`session.input_audio.mute` / `session.input_audio.unmute`	Relayed
`session.instructions.append` / `session.thinking.append` / `session.commentary.append`	Relayed
`session.close`	Relayed; starts the graceful close
`response.create` / `response.item.create`	Rejected — Responses delegation is out of scope
anything else	Relayed, so new provider events keep working

A rejected frame does not end the session: the gateway answers with an error frame carrying code unsupported_event and keeps relaying.

Server frames

Server frames are relayed to you exactly as the provider sent them. The gateway reads three of them for its own bookkeeping:

Frame	Why the gateway reads it
`session.started`	Starts the billing clock
`session.usage.updated`	Records `usage.seconds` — a cumulative total, not a delta
`session.closed`	Records the final usage and `reason`

Everything else — session.output_audio.delta, transcript deltas, session.delegation.created, error, and any event added upstream later — passes straight through.

Closing a session

Send {"type":"session.close"} and keep reading until session.closed arrives. That final frame carries the confirmed usage, and it is what stops the meter on the provider side.

If your socket simply drops, the gateway still runs the close handshake for you: the upstream session is alive and metering until it is told otherwise. session.closed reports why the session ended:

`reason`	Meaning
`close_requested`	A `session.close` was sent
`expired`	The session hit its maximum duration
`content`	Ended by a content policy
`remote_hangup`	The far end hung up
`connection_lost`	The transport failed

Limits

Limit	Value
Idle timeout	60 s without a client frame
Maximum duration	2 hours, or the model's own maximum if it is shorter
Keepalive	The gateway pings every 30 s; respond to pings to stay connected
Concurrent sessions	Capped per organization when your plan sets a cap
Credit guard	The session is terminated at 90% of the remaining credit balance

When the credit guard fires you receive an error frame with code budget_exhausted before the socket closes. Sessions on your own provider key (BYOK) are not gated on gateway credit.

Errors

Errors arrive as frames in the provider's envelope, so one parser covers both gateway and provider failures:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "code": "model_not_found",
    "message": "unknown live model: gpt-live-9",
    "client_event_id": "e1"
  }
}

Code	Cause
`invalid_session_start`	The first frame was not a valid `session.start`
`missing_model`	`session.model` was absent
`unsupported_delegation`	A delegation other than `client` was requested
`unsupported_event`	A frame this gateway does not serve (for example `response.create`)
`unauthorized`	Authentication failed
`organization_suspended`	The organization is not active
`forbidden`	The key is not allowed to use this model
`model_not_found`	Unknown Live model id
`concurrent_session_limit`	The organization is at its concurrent session cap
`credential_unavailable`	No usable provider credential
`provider_blocked`	The provider is blocked for this organization
`provider_unavailable` / `upstream_unavailable`	The provider could not be reached
`budget_exhausted`	The credit guard ended the session
`server_draining`	The gateway is shutting down; reconnect
`internal_error`	Unexpected gateway failure

Billing

Live sessions bill per second of voice time, priced from the model's per-minute rate. The provider's cumulative usage.seconds is authoritative; when a session ends without a session.closed — a hard network drop, for instance — the gateway bills its own measured wall clock instead and flags the record as unconfirmed.

Each session appears in your logs and analytics as a single streamed request against /v1/live/sessions, with its duration, close reason, and cost.

Example

import WebSocket from "ws";

const ws = new WebSocket("wss://api.llmapi.ai/v1/live/sessions", {
  headers: { Authorization: `Bearer ${process.env.LLM_API_KEY}` },
});

ws.on("open", () => {
  ws.send(
    JSON.stringify({
      type: "session.start",
      event_id: "e1",
      session: {
        model: "gpt-live-1",
        delegation: { type: "client" },
        instructions: "You are a concise support agent.",
      },
    })
  );
});

ws.on("message", (raw) => {
  const event = JSON.parse(raw.toString());

  switch (event.type) {
    case "session.started":
      // 24 kHz mono PCM16, base64-encoded, 20-100 ms per frame
      sendMicrophoneAudio((pcm16) =>
        ws.send(
          JSON.stringify({
            type: "session.input_audio.append",
            audio: pcm16.toString("base64"),
          })
        )
      );
      break;
    case "session.output_audio.delta":
      playback(Buffer.from(event.audio, "base64"));
      break;
    case "session.usage.updated":
      console.log("billed so far:", event.usage.seconds, "s");
      break;
    case "session.closed":
      console.log("closed:", event.reason, event.usage?.seconds, "s");
      break;
    case "error":
      console.error(event.error.code, event.error.message);
      break;
  }
});

// Graceful shutdown: request the close, then wait for session.closed
function hangUp() {
  ws.send(JSON.stringify({ type: "session.close" }));
}