RolloutClient and the Awaitable Rollout Handle

RolloutClient submits rollout requests to a rollout server and polls for results until the rollout finishes. osmosis eval run uses it internally. Construct it yourself when your own code drives a rollout server, such as a self-hosted harness or a custom evaluation loop.

RolloutClient ships in the base osmosis-ai package starting with 0.3.3. Import it from osmosis_ai.rollout.client. The server side uses create_rollout_server() and requires the server extra. Upgrade callers and servers together; see Upgrading to 0.3.3.

Run a Rollout to Completion

The client takes server connection settings and an optional admission deadline. The chat completions URL, llm_api_key, grading choice, metadata, and phase timeouts belong to each rollout request. This example assumes a rollout server with a configured grader and a reachable chat endpoint; set ROLLOUT_LLM_API_KEY when that endpoint requires authentication:

import asyncio
import os
from uuid import uuid4

import httpx

from osmosis_ai.rollout.client import RolloutClient


async def main():
    async with httpx.AsyncClient() as http_client:
        client = RolloutClient(
            url="http://127.0.0.1:8000",
            http_client=http_client,
            admission_timeout_sec=60,
        )
        result = await client.run_rollout(
            initial_messages=[{"role": "user", "content": "What is 6 * 7?"}],
            chat_completions_url="http://127.0.0.1:9000/v1",
            rollout_id=str(uuid4()),
            llm_api_key=os.environ.get("ROLLOUT_LLM_API_KEY"),
            label="42",
            grade=True,
        )
        print(result.status, result.sample.reward if result.sample else None)


asyncio.run(main())

run_rollout() waits asynchronously until the rollout terminates and returns the terminal RolloutResultResponse. Inspect status, sample, err_message, and err_category to distinguish success from execution failure. Pass grade=False to skip grading for that request; a graded LocalBackend request needs both a configured grader and a label or metadata. Use a fresh rollout_id for each new attempt. llm_api_key authenticates to the chat endpoint, not to the rollout server. To authenticate to the rollout server, see Authenticate to the Rollout Server. Keep the HTTP client open until all handles finish. If you let RolloutClient create its own HTTP client, close it with await client.aclose() afterward.

Track Progress with a Rollout Handle

await client.run_rollout_async() returns an awaitable RolloutHandle after admission and starts polling in the background. Call this helper with an open client and await it before closing that client:

from uuid import uuid4

from osmosis_ai.rollout.client import RolloutClient
from osmosis_ai.rollout.types import RolloutResultResponse


async def track_rollout(
    client: RolloutClient,
    chat_completions_url: str,
    llm_api_key: str | None = None,
) -> RolloutResultResponse:
    rollout = await client.run_rollout_async(
        initial_messages=[{"role": "user", "content": "What is 6 * 7?"}],
        chat_completions_url=chat_completions_url,
        rollout_id=str(uuid4()),
        llm_api_key=llm_api_key,
        label="42",
        grade=True,
    )
    phase = await rollout.wait_for_grading()
    print(phase)
    return await rollout

Awaiting the handle returns the same terminal RolloutResultResponse that run_rollout() returns. The handle also exposes the rollout lifecycle:

Member	Behavior
`await handle`	Returns the terminal `RolloutResultResponse`
`wait_for_running()` / `wait_for_grading()`	Return the latest status once that milestone is reached or passed, or the rollout terminates
`wait_for_completion()`	Equivalent to awaiting the handle
`status`	Latest observed `RolloutStatus`
`latest_result`	Latest result response after the first poll, or `None` before it
`done()`	Whether the polling task has finished, including an exception or cancellation
`cancel()`	Requests cancellation of the client-side polling task without contacting the server

Lifecycle statuses are queued, running, and grading; terminal statuses are success, failure, and cancelled. A rollout may skip phases, and polling may miss intermediate statuses. In particular, wait_for_grading() can return a terminal status when grading was skipped or execution failed. Result responses omit the persistence-only trajectory_messages field.

Admission, Leases, and Timeouts

The server creates a polling lease at admission and chooses the long-poll wait and lease timeout. The client sends the returned token as X-Osmosis-Rollout-Lease on every result request; each valid request renews the lease. You do not supply a lease token or wait duration.

Admission retries automatically on HTTP 429. Duplicate active or retained rollout_id values are rejected with HTTP 409.
When set on the client, admission_timeout_sec must be finite and bounds admission, including HTTP requests and retry delays. It does not limit execution after admission.
An admission deadline that expires during an HTTP request does not prove the server rejected the rollout. RolloutAdmissionTimeoutError reports that admission may have succeeded. The client does not cancel automatically by ID because a lost duplicate-ID rejection could otherwise cancel another active rollout.
If polling stops and the lease expires, the server publishes a failure with error category lease_expired and requests cancellation. This lease failure can be visible before execution cleanup finishes.

In 0.3.7, each result GET has two independent retry schedules:

Failure	Retry delays
`httpx.RemoteProtocolError`, `httpx.NetworkError`, or `httpx.ReadTimeout`	At most two retries, after 0.1 and 0.5 seconds
HTTP 502, 503, or 504	At most six retries, after 0.25, 0.5, 1, 2, 4, and 8 seconds

Retries reuse the rollout ID and polling lease. A retry starts only if its backoff and a full request fit before the last confirmed lease deadline; failed requests never extend that deadline. Persistent gateway errors raise RolloutProtocolError with the last status code. Connect, write, and pool timeouts, other HTTP errors (including 500 and every 4xx), and invalid result responses fail immediately. Cancellation still propagates, and result retries never replay admission or cancellation requests. Remove custom result-GET retry wrappers to avoid nested retries. Upgrade the caller environment to receive this recovery behavior.

Cancel a Rollout

await client.cancel_rollout(rollout_id) asks the server to cancel the rollout, with a five-second wall-clock bound on the request. Cancellation is idempotent for the built-in backends, including LocalBackend. The request acknowledges cancellation; continue polling to observe the terminal result after cleanup:

from osmosis_ai.rollout.client import RolloutClient, RolloutHandle
from osmosis_ai.rollout.types import RolloutResultResponse


async def cancel_and_wait(
    client: RolloutClient, rollout: RolloutHandle
) -> RolloutResultResponse:
    await client.cancel_rollout(rollout.rollout_id)
    return await rollout

handle.cancel() cancels an active polling task locally. It does not send the server cancellation request; awaiting a cancelled handle raises asyncio.CancelledError. Use client.cancel_rollout() while leaving the handle polling when you need to wait for server cleanup.

Authenticate to the Rollout Server

Pass api_key to create_rollout_server() to require a bearer token on every route, including /health, admission, result polling, and cancellation. Requests without a matching Authorization: Bearer <api_key> header receive HTTP 401. Pass the same value to RolloutClient(api_key=...) so the client sends that header on each request:

import os

from osmosis_ai.rollout.client import RolloutClient
from osmosis_ai.rollout.server import create_rollout_server

app = create_rollout_server(
    backend=backend,
    api_key=os.environ["ROLLOUT_SERVER_API_KEY"],
)

client = RolloutClient(
    url="http://127.0.0.1:8000",
    api_key=os.environ["ROLLOUT_SERVER_API_KEY"],
)

Authentication is optional. Servers created without api_key accept unauthenticated requests as before. Both the server and the client raise ValueError for an empty or whitespace-only api_key. The client does not follow redirects, so it never forwards the key to another URL.

Check Health and Wait for Idle Work

Use these lifecycle methods to observe a rollout server before a restart, redeployment, or scale-down. wait_idle() observes completion but does not prevent new admissions or confirm that artifacts have been uploaded to external storage.

Method	Behavior
`await client.health()`	Returns the `GET /health` payload: backend health fields plus `instance_id`, `process_id`, and `lifecycle`
`await client.wait_idle(timeout_sec=120, poll_interval_sec=1, process_id=None)`	Polls `/health` until `lifecycle.active_rollouts` is `0` and returns that health payload. It does not stop new admissions

The lifecycle object reports accepting_rollouts and active_rollouts. A rollout counts as active until the server finishes execution, artifact archiving, trajectory persistence, and result finalization. process_id is unique to each server process, so a changed value means the server restarted. wait_idle() retries transport errors and HTTP 408, 429, 500, 502, 503, and 504 responses until timeout_sec expires, then raises TimeoutError. Both timeout_sec and poll_interval_sec must be positive and finite. It raises RolloutProtocolError when the server reports no lifecycle metadata, such as an older server. It also raises when process_id changes while polling, or differs from the process_id you pass in. To confirm that a specific server process has finished its active work, pin the wait to the process_id from a health check:

from osmosis_ai.rollout.client import RolloutClient


async def wait_until_idle(client: RolloutClient) -> None:
    health = await client.health()
    await client.wait_idle(timeout_sec=120, process_id=health["process_id"])

To stop a Uvicorn rollout server, send it SIGTERM. Uvicorn first stops accepting connections and waits for in-flight HTTP requests, including result long polls, to finish. The app then closes admissions, waits up to 10 seconds for active work, requests cancellation and allows up to 120 seconds for cleanup, then cancels remaining tasks and waits another 10 seconds before calling backend.shutdown(). Callers cannot start new result polls after Uvicorn begins shutting down, so stop submitting rollouts and call wait_idle() before sending SIGTERM when you need every result. Allow enough host termination grace for these stages and any additional backend cleanup. No HTTP drain request is needed first.

Execution Backends

Choose the backend that runs the workflow behind the rollout server.

LocalBackend

Configure in-process execution, grading, concurrency, artifacts, and errors.

Get started

Core API

Integrations

Execution

Agent tools

RolloutClient and the Awaitable Rollout Handle

Run a Rollout to Completion

Track Progress with a Rollout Handle

Admission, Leases, and Timeouts

Cancel a Rollout

Authenticate to the Rollout Server

Check Health and Wait for Idle Work

Execution Backends

LocalBackend

​Run a Rollout to Completion

​Track Progress with a Rollout Handle

​Admission, Leases, and Timeouts

​Cancel a Rollout

​Authenticate to the Rollout Server

​Check Health and Wait for Idle Work

​Related Pages

Execution Backends

LocalBackend

Run a Rollout to Completion

Track Progress with a Rollout Handle

Admission, Leases, and Timeouts

Cancel a Rollout

Authenticate to the Rollout Server

Check Health and Wait for Idle Work

Related Pages