WS /v1/tts/stream WS

WSSwss://api.vartalaap.si/v1/tts/stream

One WebSocket, many utterances, served in order. The client sends JSON text messages; the server answers with JSON events (text frames) and audio (binary frames).

events.txt

wss://api.vartalaap.si/v1/tts/stream

Auth on the handshake: x-api-key: <key> or Authorization: Bearer <key> header, or ?api_key=<key> on the URL (browsers). Missing/invalid key → the upgrade is refused with 401.

Client → server

A JSON object per utterance, same fields as POST /v1/tts:

```js title="tts.mjs"on {"text": "Callback +44 00000 19024 पर करें।", "voice": "saanvi", "format": "pcm16", "request_id": "turn-12-s0"}

- `format`: `pcm16` or `opus`. `wav` is downgraded to `pcm16` (no container on a message stream); the `start` event says which is in effect.
- `raw`, `digits`, `en_digits`, `temperature`, `seed`, `speed` (alias `pace`), `postfx`, `profile` behave as in REST. `stream` is ignored (always streaming).
- Send the next utterance any time; it queues behind the current one on this connection. Binary messages from the client are rejected
  with an `error` event (`expected a JSON text message`).

## Server → client

Text frames are JSON events, each with the utterance's `request_id`:

| `type` | Fields | When |
|---|---|---|
| `start` | `voice`, `format`, `chunks`, `normalized`, `speed`, `postfx` | request accepted and a worker is assigned; `normalized` is exactly what will be spoken; `postfx` is the output-chain profile in effect (`default`, `telephony`) or `null` |
| `chunk` | `chunk` (index), `text`, `tokens` | a chunk begins synthesis |
| `first_audio_ms` | `chunk`, `ms` | sent right after the first binary audio frame of the utterance; `ms` is measured from the request message |
| `chunk_end` | `chunk`, `frames`, `max_frames`, `audio_s`, `prefill_ms`, `prefill_cached` (bool), `eos` (bool), `stop`, `eos_step` | a chunk finished. `stop` is why: `eos` (ordinary), `eos_tail_cut` (end-of-speech fired near the frame budget: audio complete, tail short), `cancelled`, `unknown`. A chunk whose budget ran out with no end-of-speech is not served: you get an `error` message instead (`frame cap … reached without end-of-speech`); retry with another `seed` or shorter text. `eos_step` = the frame end-of-speech fired on |
| `end` | `audio_s`, `frames`, `chunks`, `ttfa_ms` (engine-side), `wall_ms`, `rtf` | utterance complete; the last binary frame (e.g. final Ogg page) was sent before it |
| `error` | `error`, optional `status`, `retry_after` | validation (`400`/`413`), capacity (`status: 503`, `retry_after`), out of credits (`status: 402`), `timeout`, engine error. The connection stays open. |

Binary frames are audio for the utterance currently in progress: raw `pcm16` samples (s16le, 24 kHz, mono; the first frame is 80 ms =
3 840 bytes, later frames ~320 ms) or Ogg/Opus pages. For Opus the first binary frame is the Ogg header pages.

## Timeline of one utterance

```text title="events.txt"
→ {"text":"…","voice":"saanvi","format":"pcm16","request_id":"t1"}
← start          chunks=2  normalized="…"
← chunk 0        tokens=31
← <audio 3840 B>
← first_audio_ms ms=44.2
← <audio …> ×N
← chunk_end 0    frames=39 max_frames=142 audio_s=3.12 prefill_cached=true eos=true stop=eos eos_step=36
← chunk 1 … <audio …> … chunk_end 1
← end            audio_s=4.9 ttfa_ms=17.8 wall_ms=690.3 rtf=0.141

Capacity and errors

If every worker is busy and the queue is full, or the queued request waited more than 2 s, the utterance gets {"type":"error","request_id":"t1","status":503,"retry_after":1,"error":"busy"} and nothing is billed; resend it after retry_after seconds. The connection is unaffected. Per-utterance wall limit: 120 s → {"type":"error","error":"timeout"}.

Cancelling

Close the socket. Synthesis of the current utterance stops within one frame and queued utterances are dropped. The utterance in flight is billed for its full text (metered when its stream ends). Open a new socket for the next turn (see voice agents & barge-in).

Example

tts.py

import asyncio, json, os, websockets

async def speak(lines):
    headers = {"x-api-key": os.environ["VARTALAAP_API_KEY"]}
    async with websockets.connect("wss://api.vartalaap.si/v1/tts/stream", additional_headers=headers) as ws:
        for i, line in enumerate(lines):
            await ws.send(json.dumps({"text": line, "voice": "saanvi", "format": "pcm16", "request_id": f"s{i}"}))
        pending = len(lines)
        async for msg in ws:
            if isinstance(msg, bytes): play(msg); continue
            ev = json.loads(msg)
            if ev["type"] == "first_audio_ms": print(ev["request_id"], "first audio", ev["ms"], "ms")
            elif ev["type"] == "end": print(ev["request_id"], ev["audio_s"], "s rtf", ev["rtf"]); pending -= 1
            elif ev["type"] == "error":
                if ev.get("status") == 503: await asyncio.sleep(ev.get("retry_after", 1)); await ws.send(json.dumps({...}))
                else: print("error", ev); pending -= 1
            if pending == 0: break