RolloutClient and the Awaitable Rollout Handle

RolloutClient submits rollout requests to a rollout server and polls for results until the rollout finishes. osmosis eval run uses it internally. Construct it yourself when your own code drives a rollout server, such as a self-hosted harness or a custom evaluation loop.

RolloutClient ships in the base osmosis-ai package starting with 0.3.3. Import it from osmosis_ai.rollout.client. The server side uses create_rollout_server() and requires the server extra. Upgrade callers and servers together; see Upgrading to 0.3.3.

Run a Rollout to Completion

The client takes server connection settings and an optional admission deadline. The chat completions URL, llm_api_key, grading choice, metadata, and phase timeouts belong to each rollout request. This example assumes a rollout server with a configured grader and a reachable chat endpoint; set ROLLOUT_LLM_API_KEY when that endpoint requires authentication:

import asyncio
import os
from uuid import uuid4

import httpx

from osmosis_ai.rollout.client import RolloutClient


async def main():
    async with httpx.AsyncClient() as http_client:
        client = RolloutClient(
            url="http://127.0.0.1:8000",
            http_client=http_client,
            admission_timeout_sec=60,
        )
        result = await client.run_rollout(
            initial_messages=[{"role": "user", "content": "What is 6 * 7?"}],
            chat_completions_url="http://127.0.0.1:9000/v1",
            rollout_id=str(uuid4()),
            llm_api_key=os.environ.get("ROLLOUT_LLM_API_KEY"),
            label="42",
            grade=True,
        )
        print(result.status, result.sample.reward if result.sample else None)


asyncio.run(main())

run_rollout() waits asynchronously until the rollout terminates and returns the terminal RolloutResultResponse. Inspect status, sample, err_message, and err_category to distinguish success from execution failure. Pass grade=False to skip grading for that request; a graded LocalBackend request needs both a configured grader and a label or metadata. Use a fresh rollout_id for each new attempt. llm_api_key authenticates to the chat endpoint, not to the rollout server. Keep the HTTP client open until all handles finish. If you let RolloutClient create its own HTTP client, close it with await client.aclose() afterward.

Track Progress with a Rollout Handle

await client.run_rollout_async() returns an awaitable RolloutHandle after admission and starts polling in the background. Call this helper with an open client and await it before closing that client:

from uuid import uuid4

from osmosis_ai.rollout.client import RolloutClient
from osmosis_ai.rollout.types import RolloutResultResponse


async def track_rollout(
    client: RolloutClient,
    chat_completions_url: str,
    llm_api_key: str | None = None,
) -> RolloutResultResponse:
    rollout = await client.run_rollout_async(
        initial_messages=[{"role": "user", "content": "What is 6 * 7?"}],
        chat_completions_url=chat_completions_url,
        rollout_id=str(uuid4()),
        llm_api_key=llm_api_key,
        label="42",
        grade=True,
    )
    phase = await rollout.wait_for_grading()
    print(phase)
    return await rollout

Awaiting the handle returns the same terminal RolloutResultResponse that run_rollout() returns. The handle also exposes the rollout lifecycle:

Member	Behavior
`await handle`	Returns the terminal `RolloutResultResponse`
`wait_for_running()` / `wait_for_grading()`	Return the latest status once that milestone is reached or passed, or the rollout terminates
`wait_for_completion()`	Equivalent to awaiting the handle
`status`	Latest observed `RolloutStatus`
`latest_result`	Latest result response after the first poll, or `None` before it
`done()`	Whether the polling task has finished, including an exception or cancellation
`cancel()`	Requests cancellation of the client-side polling task without contacting the server

Lifecycle statuses are queued, running, and grading; terminal statuses are success, failure, and cancelled. A rollout may skip phases, and polling may miss intermediate statuses. In particular, wait_for_grading() can return a terminal status when grading was skipped or execution failed. Result responses omit the persistence-only trajectory_messages field.

Admission, Leases, and Timeouts

The server creates a polling lease at admission and chooses the long-poll wait and lease timeout. The client sends the returned token as X-Osmosis-Rollout-Lease on every result request; each valid request renews the lease. You do not supply a lease token or wait duration.

Admission retries automatically on HTTP 429. Duplicate active or retained rollout_id values are rejected with HTTP 409.
When set on the client, admission_timeout_sec must be finite and bounds admission, including HTTP requests and retry delays. It does not limit execution after admission.
An admission deadline that expires during an HTTP request does not prove the server rejected the rollout. RolloutAdmissionTimeoutError reports that admission may have succeeded. The client does not cancel automatically by ID because a lost duplicate-ID rejection could otherwise cancel another active rollout.
If polling stops and the lease expires, the server publishes a failure with error category lease_expired and requests cancellation. This lease failure can be visible before execution cleanup finishes.

Cancel a Rollout

await client.cancel_rollout(rollout_id) asks the server to cancel the rollout, with a five-second wall-clock bound on the request. Cancellation is idempotent for the built-in backends, including LocalBackend. The request acknowledges cancellation; continue polling to observe the terminal result after cleanup:

from osmosis_ai.rollout.client import RolloutClient, RolloutHandle
from osmosis_ai.rollout.types import RolloutResultResponse


async def cancel_and_wait(
    client: RolloutClient, rollout: RolloutHandle
) -> RolloutResultResponse:
    await client.cancel_rollout(rollout.rollout_id)
    return await rollout

handle.cancel() cancels an active polling task locally. It does not send the server cancellation request; awaiting a cancelled handle raises asyncio.CancelledError. Use client.cancel_rollout() while leaving the handle polling when you need to wait for server cleanup.

Execution Backends

Choose the backend that runs the workflow behind the rollout server.

LocalBackend

Configure in-process execution, grading, concurrency, artifacts, and errors.

Get started

Core API

Integrations

Execution

Agent tools

RolloutClient and the Awaitable Rollout Handle

Run a Rollout to Completion

Track Progress with a Rollout Handle

Admission, Leases, and Timeouts

Cancel a Rollout

Execution Backends

LocalBackend

​Run a Rollout to Completion

​Track Progress with a Rollout Handle

​Admission, Leases, and Timeouts

​Cancel a Rollout

​Related Pages

Execution Backends

LocalBackend

Run a Rollout to Completion

Track Progress with a Rollout Handle

Admission, Leases, and Timeouts

Cancel a Rollout

Related Pages