Changelog

​

Breaking Changes

• Upgrade rollout callers and servers together. Replace HttpRolloutDriver and callback models with RolloutClient; each request needs a unique rollout_id and an explicit llm_api_key when its chat endpoint requires authentication.
• Custom backends return ExecutionOutcome from execute(request) and publish progress with await RolloutContext.set_status(). Move local LLM bridge imports from osmosis_ai.rollout.controller to osmosis_ai.eval.local.
• Local evaluation’s protocol fingerprint is now 0.4. Runs recorded under the previous protocol need a new run name, or the previous SDK to resume.
• Managed SkyPilot placement is removed. Select EnvironmentType.DAYTONA with Daytona credentials; HARBOR_SKYPILOT_CONTEXT is no longer read. Omitting environment_config still selects Docker.

​

Added

• RolloutClient handles lease renewal, HTTP 429 admission retries, cancellation, and optional grade=False. await run_rollout_async() returns an awaitable RolloutHandle with live status and milestone waits.
• osmosis eval upload and osmosis eval run --upload include logs.txt in the platform Logs tab. Local logging and upload redact known ambient provider and platform credentials of at least eight characters available in the current process environment; review logs before sharing because redaction does not detect every possible secret.

​

Changed

• Built-in Daytona environments with delete=True default to stopping after 60 minutes of Daytona-observed inactivity and immediate deletion. Increase auto_stop_interval_mins for longer periods without provider-visible activity, or set it to 0 to disable auto-stop. See Daytona lifecycle defaults.
• Supported dependency ranges expand to Harbor >=0.20.0,<0.23 and OpenAI Agents >=0.18.1,<0.21. When upgrading Harbor, remove top-level trajectory.json inputs from SDK workflow task directories and check Docker host nftables support before restricted-network trials. Generated rollout trajectory artifacts remain supported.

​

Fixed

• Polling preserves completed rewards and failure details, bounds admission and cancellation cleanup, and leaves local evaluation work without a recorded terminal result pending for resume.
• Login falls back to an owner-only credentials file when the system keyring is unavailable. OSMOSIS_TOKEN_STORE=keyring requires the keyring; file selects file storage explicitly. See credential storage.
• osmosis train info lists checkpoints while a training run is still running.

​

Upgrade

​

What changed

• RolloutClient.run_rollout_async() now returns an awaitable RolloutHandle instead of asyncio.Task. The handle exposes status, latest_result, done(), cancel(), and the milestone waits wait_for_running(), wait_for_grading(), and wait_for_completion(). A milestone wait also finishes when the phase has already passed or the rollout terminates.
• Result long polling returns on status changes, including grading milestones from the Local and Harbor backends, without waiting for the full polling timeout.
• Custom backends publish intermediate progress by awaiting the active RolloutContext.set_status() method. Result polling no longer reads progress from ExecutionBackend.rollout_status().
• osmosis eval upload and osmosis eval run --upload include the combined logs.txt, so local evaluation logs appear in the platform’s run Logs tab.
• Local evaluation redacts known provider and platform credentials of at least eight characters found in the process environment from logs, and repeats the redaction before upload hashing.

​

Who needs to act

• Update callers that treated the return value of run_rollout_async() as an asyncio.Task. Keep awaiting the handle for the terminal result, and use its done(), cancel(), and milestone methods instead of Task-only APIs.
• Update custom backends that reported intermediate progress through ExecutionBackend.rollout_status() to await RolloutContext.set_status() instead.

​

What changed

• Rollout completion now uses leased long polling instead of callbacks. The new RolloutClient submits a rollout, renews the server-issued polling lease automatically, retries admission on HTTP 429, polls for the terminal result, and supports explicit cancellation. A request can skip grading with grade=False.
• HttpRolloutDriver and the callback models are removed. Each request supplies a unique rollout_id, plus an explicit llm_api_key when the chat endpoint requires authentication.
• Custom backends return an ExecutionOutcome from execute(request) instead of invoking result callbacks. Local LLM bridge imports move from the removed osmosis_ai.rollout.controller package to osmosis_ai.eval.local.
• Supported Harbor versions expand to >=0.20.0,<0.23, and OpenAI Agents to >=0.18.1,<0.21.
• Polled results preserve completed rewards and failure details through cancellation and serialization errors, and lease expiry or server shutdown allows bounded workflow and sandbox cleanup.
• Admission deadlines cover HTTP requests and retry delays. A lost admission response leaves unobserved work to lease expiry instead of risking cancellation of another rollout with the same ID.
• Local evaluation bounds result polling, waits for cancellation cleanup, and leaves work without a recorded terminal result pending for resume, including on polling 403 and 404 errors.

​

Who needs to act

• Upgrade callers and rollout servers together. The leased long-poll protocol does not interoperate with the 0.3.2 callback protocol, so replace HttpRolloutDriver and callback models with RolloutClient.
• Local evaluation’s protocol fingerprint is now 0.4. A run recorded with the previous protocol cannot resume under this release; use a new run name, or finish the run with the previous SDK.
• When upgrading Harbor, remove top-level trajectory.json inputs from SDK workflow tasks, and ensure the Docker host passes its nftables probe before running restricted-network trials.

​

Breaking Changes

• The public MessageResult, GraderInitRequest, GraderInitResponse, RolloutDriver, and resolve_workspace_directory_from_cwd() APIs are removed. Use OperationResult, HttpRolloutDriver, and resolve_workspace_directory() instead; the two unused grader-init types have no replacement.
• The SDK no longer declares requests as a base dependency or tqdm in the rubric extra. Declare either package directly if your own code imports it.

​

Added

• osmosis eval run reaches Daytona, SkyPilot, and other cloud sandboxes through a managed cloudflared tunnel that starts automatically when the resolved sandbox cannot reach host loopback, or through a tunnel you run yourself with --advertise-url.
• Root --workspace <name> selects a platform workspace by name. Benchmark, dataset, model, secret, and run-inspection commands then work without a local repository, and eval submit and train submit accept an absolute config path when its Git repository matches the selected workspace.
• osmosis eval upload <run-name> resolves a completed run under .osmosis/evals/; explicit run directories still work.
• The eval installation extra is documented, and Python 3.14 is supported.

​

Changed

• Local osmosis eval run --dataset-file ... no longer loads platform credentials unless --upload is requested.
• Newly scaffolded rollouts depend on the stable osmosis-ai[server]>=0.3.0,<0.4 release line.
• Local evaluation output, retry, resume, and upload paths are printed relative to the directory the command was invoked from.

​

Fixed

• Local evaluation validates the model and rollout server before opening a tunnel, keeps slow non-streaming tunnel responses alive, accepts a registered Cloudflare connection when the host cannot probe the tunnel URL, and fails fast on an unreachable loopback endpoint instead of hanging.
• Local evaluation warns when the CLI and the rollout environment use different osmosis-ai versions.
• The harbor extra installs Harbor’s Daytona dependencies, so Daytona environments work without separate dependency setup.
• --secrets-file values override the process environment only for the local evaluation run and are restored on every exit path, and orphan cleanup rejects symlinked run directories.

​

Upgrade

​

Breaking Changes

• RolloutDriver.run now takes a single RolloutRunRequest argument. Update custom drivers and their callers to pass the request object.
• ExecutionBackend.max_concurrency and the import-time osmosis_ai.platform.auth.PLATFORM_URL are removed. Read concurrency capacity from the rollout server’s /health response, and the active platform URL from get_platform_url().
• Unused server-owned fields are removed from the public record types. Stop reading UploadInfo.s3_key and .upload_id, DatasetFile.df_stats and .organization_id, TrainingRunMetrics.training_run_id, EvalRunMetrics.eval_run_id, RolloutInfo.last_synced_at, and TrainingRunCheckpoints.training_run_id.

​

Added

• osmosis eval run executes a crash-safe local evaluation from the same TOML a managed run uses, through the new eval installation extra. It supports dataset slicing, resumable output, uv-managed rollout environments, LocalBackend and Harbor’s Docker environment, readable generated run names, OpenAI Responses routing, bounded admission, and orphan rollout-server cleanup.
• osmosis eval upload <run-dir> and eval run --upload import a completed local run into the platform. Both are server-authoritative and idempotent, and neither launches a managed evaluation.

​

Changed

• The CLI machine contract is standardized. --json and --plain never prompt, JSON errors use stable {code, message, details} envelopes on stderr, machine-readable warnings use JSON Lines, and non-finite values can no longer produce invalid JSON.
• Logins are platform-scoped and persist in the operating-system keyring across directories and environments. They survive HTTP 401 responses, and a non-production OSMOSIS_TOKEN is validated against OSMOSIS_TOKEN_PLATFORM_URL before any network access.

​

Fixed

• Mini SWE-agent benchmark credential validation matches the Platform, while the Cursor CLI harness-key requirement is preserved.
• osmosis dev server up prints the one-time API key the Platform returns, so the provisioned server can be used immediately.

​

Upgrade

​

Breaking Changes

• Every rollout now produces exactly one RolloutSample and reward through rollout-scoped URLs. Update custom contexts, graders, integrations, callbacks, and backend adapters to the single-sample contract.
• Runtime integrations use explicit server, strands, openai-agents, harbor, rubric, and parquet extras and feature-specific imports. AgentWorkflow.run() returns one message history, and the former loader, validator, and multi-trajectory helpers are removed.
• HarborBackend is now the container-native implementation previously called HarborBackendV2; the legacy backend, its constructor arguments, OsmosisInstalledAgent, and HarborAgentWorkflowContext are removed.
• Prompt and metadata datasets each use one uniform schema across every row, and benchmark detail output replaces required_secret_names with requires_judge_api_key.

​

Added

• osmosis benchmark now supports catalog discovery, submission, run inspection, logs, cancellation, and output downloads.
• The container-native Harbor backend adds installable workflow bundles, native agents, template and dataset modes, prewarming, diagnostics, artifacts, admission control, status polling, and cancellation.
• Train, evaluation, and benchmark submissions can resolve per-run secrets from a dotenv file, standard input, process environment, or hidden terminal prompt without saving them to the Platform secret store.
• osmosis quickstart guides users through authentication, workspace repository setup, cloning, billing checks, and a ready-to-paste agent prompt; matching onboarding APIs are available on OsmosisClient.

​

Reliability and Security

• Rollout bundles use content-addressed atomic caches, support flat, src/, and PEP 420 layouts, reject unsafe symlinks and cache layouts, and install a compatible uv builder through the Harbor extra.
• Local execution enforces controller deadlines, validates rewards, sanitizes optional callback telemetry, and preserves consistent terminal callback outcomes.
• Harbor preserves samples, diagnostics, and ATIF trajectories while preventing task-source symlinks and rollout credentials from entering archived trial data; secrets-file errors no longer echo secret-bearing input.

​

Upgrade

​

What changed

• osmosis quickstart now signs in, selects a workspace, waits for its repository connection, clones or reuses the workspace directory, checks billing, and produces a ready-to-paste agent prompt for training, evaluation, or benchmarks.
• OsmosisClient adds list_workspaces(), get_quickstart_status(), and complete_quickstart() for workspace-scoped onboarding integrations.
• RolloutSample.reward now rejects NaN, infinity, and non-numeric values at construction and assignment. A grader that assigns one of these values raises pydantic.ValidationError instead of silently reaching the controller as no reward at all. NumPy-like numeric scalars are normalized to float.
• LocalBackend counts time spent waiting on its concurrency limiter against the controller’s agent_timeout_sec, and applies an independent grader_timeout_sec around grading. Workflows that swallow CancelledError or block the event loop past their deadline are now reported as timeouts instead of returning a late success.
• osmosis eval submit, osmosis train submit, and osmosis benchmark submit accept --secrets-file lines written as export NAME=value and NAME="quoted value". The CLI reports a malformed line by source and line number only, so the secret text never reaches CI logs. The CLI also rejects names that do not pass Python’s identifier check up front.
• Rollout bundle builds are now content-addressed by project contents, interpreter ABI, and platform, and each build runs in an isolated directory that must produce exactly one wheel. src/ layouts (as scaffolded by uv init --lib) and PEP 420 namespace packages are supported alongside flat layouts, and the Harbor extra installs its compatible uv builder.
• Harbor rejects symlinks before task materialization and scrubs rollout controller credentials from retained trial files before they can be archived.

​

Who needs to act

• Upgrade to osmosis-ai>=0.3.0rc4 before using osmosis quickstart or relying on these rollout hardening fixes.
• Update any grader that could assign float("nan"), float("inf"), or a non-numeric object to ctx.set_reward. Osmosis now rejects these values. Return the intended numeric reward, or leave the reward unset to mean “not graded”.
• Update --secrets-file sources that relied on the previous parser accepting quotes as part of a value or an export prefix as part of a name. The old behavior silently submitted the wrong string; the new parser resolves both forms to the intended value.

​

What changed

• Prompt datasets require user_prompt plus ground_truth or its label alias; system_prompt remains optional.
• Metadata datasets require a non-empty metadata object on every row. Validation now checks every JSONL and CSV row and every Parquet metadata value.
• osmosis train submit, osmosis eval submit, and osmosis benchmark submit accept --secrets-file; values can also come from the process environment or an interactive prompt.
• Benchmark details replace required_secret_names with requires_judge_api_key.

​

Who needs to act

​

What changed

• Install optional features with the server, strands, openai-agents, harbor, rubric, or parquet extras; full installs all of them.
• Import server, Harbor, Strands, OpenAI Agents, and rubric features from their public submodules instead of the rollout or package root.
• AgentWorkflow.run() returns one message history as AgentWorkflowOutput, a bare message list, or None for the active sample-source fallback. Output metrics must be finite.
• The former HarborBackendV2 is now HarborBackend; the legacy backend and HarborAgentWorkflowContext were removed.
• The CLI adds benchmark catalog, submission, status, logs, cancellation, and output-download workflows.

​

Who needs to act

​

What changed

• GraderContext.samples is now GraderContext.sample, and set_sample_reward(sample_id, reward) is now set_reward(reward).
• Custom integrations register one source with set_sample_source() and read it with get_sample().
• RolloutSample.id and MultiTurnMode were removed. The rollout URL now supplies execution identity.
• Model and callback requests use rollout-scoped URLs. Integrations no longer attach per-call sample or rollout routing headers.
• Backends exchange sample.json and a single-value reward.json ({"reward": <float>}).
• Strands and OpenAI Agents integrations enforce one registered agent or session per workflow execution.

​

Who needs to act