Skip to content

How the API executes a prompt

Updated
Reading time
4 min
Level
beginner

Your request is written to a durable queue, dispatched to an idle browser agent on one of your own paired machines, typed into a provider's web interface, and read back when the answer finishes generating. Every design decision below follows from one fact: the execution environment is a real browser on a computer you own, which is slower and less reliable than a data centre, and is sometimes asleep.

Understanding this is what stops you designing an integration that fights the system.

The path

your application
  → POST /v1/completions
  → your queue                    durable; survives restarts and offline devices
  → is one of your devices online?
  → Revoye Desk on your machine   authenticated socket, signed with your device key
  → the Revoye extension          via a local browser bridge
  → an idle agent                 one browser tab, one provider
  → the provider's web UI         the prompt is typed in
  → the agent reads the finished answer
  → back the same way to your waiting request, or to your callback_url

1. Accepted means durable

Revoye writes the job down before it does anything else. From that moment:

  • A dropped connection does not lose it. Fetch it at GET /v1/completions/{id}.
  • A Revoye deploy does not lose it.
  • Your machine being offline does not lose it.

This is the single most useful property to design around. Your integration's job is to record the id, not to hold a connection.

2. The device check comes before the dispatch

Nothing is dispatched until the router knows one of your devices is connected. If none is:

Your request saidWhat happens
wait: trueThe request waits. The job queues, and runs when a device returns
wait: false with a callback_urlAccepted and queued. The webhook fires when it runs
wait: false, no callback_url503 NO_DEVICE_ONLINE immediately — there would be nobody to tell

For anything unattended, the second row is the one you want.

3. Agent selection is a filter, then a strategy

A candidate agent must clear all of:

  • enabled in your dashboard, and enabled in your extension
  • currently idle
  • its provider is enabled
  • its device is connected
  • it has not already failed this job
  • under its own hourly limit, its provider's hourly limit, and your global hourly limit
  • permitted by your time-window policy, if you use one
  • matching the provider you asked for, if you asked for one

"Idle" includes work you started yourself. The same agents serve the extension's multi-agent chat, so an agent mid-way through a chat prompt is not idle and the router skips it rather than interrupting it. If your API traffic and your own chatting compete, the fix is more agents — or switching individual ones off for one use or the other.

Whatever survives, your rotation strategy chooses from: round robin, least recently used, priority, or time window. You set the strategy in the dashboard; rate limits and capacity covers what happens when the filter empties.

4. The prompt is typed, and generation is waited for

The agent opens the provider's page, enters the prompt, and waits for generation to actually finish — not for a fixed timer. Then it reads the answer back out of the page.

This is why run_ms in the response is usually tens of seconds, and why it varies with prompt length, provider load and the model the provider decided to use. It is also why there is no streaming: there is no partial output to forward, only a page that is still changing.

5. Timeouts and retries protect the job, not the connection

Two separate clocks:

FieldDefaultRangeCovers
Per attempttimeout_ms180 0005 000–600 000One dispatch to one agent
Whole jobdeadline_ms900 00010 000–3 600 000Every attempt together

When an attempt times out, the job is taken off that agent and offered to the next eligible one — never back to one that already failed it. When the deadline passes, the job stops.

wait: true holds your HTTP connection for at most min(deadline_ms, 600000). If that expires you get 504 JOB_TIMEOUT with the job id, and the job keeps running. The timeout is your patience, not the job's.

6. The result comes back with its history

Every completed job tells you what happened, not just what it produced: provider, agent_id, attempts, queue_ms (how long it waited before the first dispatch, even after retries) and run_ms. When something is slow, those four fields say whether it was your queue, your fleet, or the provider.

What this means for your integration

  1. Store the job id. It is the only thing you cannot recover.
  2. Set a long client timeout, or don't wait at all. A 30-second default will abandon healthy work.
  3. Use Idempotency-Key. Retrying safely.
  4. Treat NO_DEVICE_ONLINE and NO_AGENT_AVAILABLE as capacity, not failure. Back off and retry.
  5. Run more agents for more throughput. One agent is one prompt at a time.