HarborBackend Task and Dataset Execution

HarborBackend runs an Osmosis AgentWorkflow or a registered native Harbor agent inside a Harbor trial. Every rollout gets a copied task directory, a task-defined environment, and Harbor-managed agent and verifier phases.

SDK v0.3 removed the pre-v0.3 HarborBackend and renamed HarborBackendV2 to HarborBackend. The current class has a different constructor and no compatibility alias. Follow the v0.3 migration guide before upgrading an existing harness.

Install and Import

Install the Harbor and rollout-server features:

pip install "osmosis-ai[server,harbor]>=0.3.0rc1,<0.4"

uv must also be on PATH: through v0.3.0rc3 the Harbor extra does not install it, and the backend runs uv build whenever it packages an AgentWorkflow or Grader bundle. Import the backend from its Harbor submodule:

from osmosis_ai.rollout.backend.harbor import HarborBackend, TaskMode

HarborBackend is not re-exported from osmosis_ai.rollout or osmosis_ai.rollout.backend. Do not install harbor[skypilot]; the managed rollout runtime supplies its compatible SkyPilot SDK.

Choose a Task Mode

task_mode determines who owns the instruction, how the default task is selected, and whether the request prompt reaches the task.

Behavior	Template mode	Dataset mode
`tasks_dir`	One reusable Harbor task directory	Root containing one directory per Harbor task
Instruction owner	The incoming rollout request	The selected Harbor task
Request prompt	Serialized into the copied task’s `instruction.md`	Discarded by the backend
Default task selection	Always the configured task	`metadata["harbor_task_id"]` names a child directory
Typical reward source	Osmosis `Grader` or a reusable task verifier	The selected task’s native verifier
Best fit	One environment applied to changing dataset rows	Existing Harbor tasks or datasets already runnable as authored

Both modes use Harbor trials and provide the same isolation. Mode selection changes task semantics, not the sandbox technology.

Template Mode

Template mode starts every rollout from one task directory. When the request contains a prompt, the backend copies the task and replaces instruction.md with the serialized message list before Harbor starts the trial.

rollouts/my-rollout/
├── main.py
├── pyproject.toml
└── task-template/
    ├── task.toml
    └── environment/
        └── Dockerfile

An instruction.md file is optional for the configured template when every rollout supplies a prompt. Add a placeholder instruction if you also want the directory to pass Harbor’s standalone task validation. A tests/test.sh is optional when the backend supplies an Osmosis Grader.

from pathlib import Path

from harbor.models.environment_type import EnvironmentType
from harbor.models.trial.config import EnvironmentConfig as HarborEnvironmentConfig
from harbor.trial.queue import TrialQueue
from osmosis_ai.rollout.backend.harbor import HarborBackend
from osmosis_ai.rollout.server import create_rollout_server

backend = HarborBackend(
    orchestrator=TrialQueue(n_concurrent=4),
    tasks_dir=Path("task-template"),
    task_mode="template",
    agent=MyWorkflow,
    workflow_config=my_workflow_config,
    grader=MyGrader,
    grader_config=my_grader_config,
    environment_config=HarborEnvironmentConfig(type=EnvironmentType.SKYPILOT),
    max_queue_depth=8,
)

app = create_rollout_server(
    backend=backend,
    lifespan=backend.prewarm_lifespan(),
)

Template mode is the Harbor path most similar to LocalBackend: one workflow handles changing dataset prompts. Choose it when those prompts need the same isolated operating system, tools, services, or filesystem setup on every rollout.

In template mode, metadata["harbor_task"] can select a different task for one request, but the request prompt still replaces that task’s instruction. The backend logs a warning when it overwrites an instruction from a dynamically selected task.

Dataset Mode

Dataset mode treats each Harbor task as the complete unit of work. The task retains its own instruction.md, environment, configuration, and verifier.

tasks/
├── task-a/
│   ├── instruction.md
│   ├── task.toml
│   ├── environment/
│   │   └── Dockerfile
│   └── tests/
│       └── test.sh
└── task-b/
    ├── instruction.md
    ├── task.toml
    ├── environment/
    │   └── Dockerfile
    └── tests/
        └── test.sh

Configure the root directory and a native Harbor agent or Osmosis workflow:

from pathlib import Path

from harbor.models.environment_type import EnvironmentType
from harbor.models.trial.config import EnvironmentConfig as HarborEnvironmentConfig
from harbor.trial.queue import TrialQueue
from osmosis_ai.rollout.backend.harbor import HarborBackend
from osmosis_ai.rollout.server import create_rollout_server

backend = HarborBackend(
    orchestrator=TrialQueue(n_concurrent=4),
    tasks_dir=Path("tasks"),
    task_mode="dataset",
    agent="mini-swe-agent",
    grader=None,
    environment_config=HarborEnvironmentConfig(type=EnvironmentType.SKYPILOT),
    max_queue_depth=8,
)

app = create_rollout_server(
    backend=backend,
    lifespan=backend.prewarm_lifespan(["task-a", "task-b"]),
)

Each Osmosis metadata-mode dataset row selects the Harbor task by directory name:

{"metadata": {"harbor_task_id": "task-a"}}
{"metadata": {"harbor_task_id": "task-b"}}

The backend deliberately removes any request prompt in dataset mode. Harbor passes the selected task’s instruction to the native agent or bundled AgentWorkflow. The task’s verifier normally computes the reward; if tests/test.sh is absent, a configured Osmosis Grader supplies the verifier instead.

Integrate an Existing Harbor Dataset

If a local dataset already works with harbor run, its task directories are the reusable boundary:

Keep the task directories

Point tasks_dir at the local dataset root. Each child must be a Harbor task with task.toml, instruction.md, an environment/, and the verifier files required by that task.

Choose a supported agent track

Use agent="terminus-2", "mini-swe-agent", or "oracle" when that registered native agent fits. Otherwise package your agent behavior as an Osmosis AgentWorkflow and pass its class or import path.

Create the selector dataset

Upload an Osmosis metadata-mode dataset whose harbor_task_id values match the task directory names. The Platform dataset selects trials; the Harbor task directories remain with the rollout code.

Run evaluation before training

Confirm that the native verifier emits the reward channel and that trainable agents produce a usable trajectory before starting training.

This path reuses task content, not the complete Harbor job definition. HarborBackend does not read Harbor JobConfig, dataset filters, attempt counts, arbitrary agent configs, or harbor run CLI flags. Osmosis supplies request scheduling, the model endpoint, callbacks, and rollout identity.

Dynamic Task Sources

In either mode, a request can set metadata["harbor_task"] instead of using the configured default selection:

Source	Metadata
Local task	`{"harbor_task": "./tasks/task-a"}`
Harbor package	`{"harbor_task": "org/name@ref"}`
Git task	`{"harbor_task": "path/in/repo", "git_url": "https://...", "git_commit_id": "<sha>"}`

Pin package references and Git commits for reproducibility. Instruction ownership still follows task_mode: template mode replaces the fetched task’s instruction, while dataset mode keeps it.

Treat these fields as trusted control-plane input, not as untrusted dataset content. The rollout server resolves, downloads, and stages the referenced task outside the trial sandbox, and a task’s Dockerfile and verifier scripts are executable content. Accept harbor_task, git_url, and git_commit_id values only from task sources you control.

Agent Tracks

agent selects one of two execution tracks:

Agent value	Behavior
`type[AgentWorkflow]` or `"module:attr"`	Builds the rollout project as a wheel, installs it in the trial, and runs the workflow inside the task environment
Registered native name	Runs Harbor’s native agent with the active Osmosis model endpoint wired into its supported configuration

An AgentWorkflow project must contain pyproject.toml and an importable Python package, and the task image must support Python. Pass code_dir when the project cannot be inferred, or pass a prebuilt bundle wheel. Keep the task Dockerfile focused on task dependencies; the backend installs the rollout bundle and can preinstall its declared dependencies into the copied image. Registered native names:

Name	Use
`"terminus-2"`	Trainable Harbor terminal agent
`"mini-swe-agent"`	Trainable lightweight software-engineering agent
`"oracle"`	Runs the task’s reference solution to validate tasks and verifiers; it does not emit a trainable model trajectory

Use workflow_config with an AgentWorkflow. Use native_agent_kwargs only with a registered native agent. model_name defaults to openai/osmosis-rollout; per-request metadata can override it with harbor_model.

A custom agent name that works in a separate Harbor installation is not automatically available through HarborBackend. The backend accepts only the registered native names above or an Osmosis AgentWorkflow.

Reward Source and Precedence

Harbor always treats a task-provided verifier as authoritative:

Configuration	Result
Task contains `tests/test.sh`	Harbor runs that script, even when `grader=MyGrader` is also set
No task `tests/test.sh`, `grader=MyGrader`	The backend generates a verifier script that installs and runs the bundled Osmosis grader
Task contains `tests/test.sh`, `grader=None`	Harbor runs the task-native verifier
No task verifier and `grader=None`	The rollout fails validation because it has no reward source

There is no custom_tests_dir in v0.3. Keep native verifier files under each task’s tests/ directory. A native verifier must write Harbor’s reward channel, normally through /logs/verifier/reward.txt or reward.json with a reward key; the backend does not guess another channel.

Constructor Reference

Parameter	Type / default	Description
`orchestrator`	`TrialQueue`	Harbor queue used to run trials
`tasks_dir`	`Path`	One task directory in template mode, or a task-root directory in dataset mode
`agent`	`type \| str \| None`	Workflow class, import path, registered native agent name, or `None` when a prebuilt bundle supplies the workflow
`native_agent_kwargs`	`dict[str, Any] \| None`	Extra configuration for a registered native agent; invalid with an `AgentWorkflow`
`task_mode`	`"template"`	`"template"` or `"dataset"`
`model_name`	`"openai/osmosis-rollout"`	Model passed to native agents; request metadata can override it with `harbor_model`
`grader`	`type \| str \| None`	Optional Osmosis grader used only when the task does not already provide `tests/test.sh`
`workflow_config`	`Any`	Optional workflow config instance or import path bundled with a workflow agent
`grader_config`	`Any`	Optional grader config instance or import path bundled with the grader
`code_dir`	`Path \| None`	Python project to package; inferred from the workflow or grader when possible
`bundle`	`Path \| None`	Prebuilt Osmosis bundle wheel used instead of building `code_dir`
`environment_config`	`EnvironmentConfig \| None`	Harbor runtime and placement configuration
`trials_dir`	`Path \| None`	Host directory for Harbor trial data
`cleanup_successful_trials`	`True`	Removes successful trial staging after artifacts are archived
`patch_dockerfile_with_sdk`	`None`	Preinstalls bundle dependencies into the copied task image; defaults on when a bundle exists
`agent_setup_timeout_sec`	`float \| None`	Optional Harbor agent setup timeout
`max_queue_depth`	`int \| None`	Maximum waiting queue depth; use an integer >= 1 or `None` for no bound

Prewarming, Capacity, and Cancellation

prewarm() builds task images and runs agent setup before the server accepts traffic. Template mode prewarms its configured task; dataset mode requires the task IDs you want to prewarm. TrialQueue(n_concurrent=<n>) controls Harbor trial concurrency. max_queue_depth limits waiting rollouts so the server can return HTTP 429 instead of growing an unbounded queue. rollout_status() reports queued, running, grading, or recently finished state; cancel_rollouts() cancels queued or running work by ID, prefix, or all. Keep TrialQueue at its default RetryConfig(max_retries=0). Queue-level attempt retries are not supported by the backend’s terminal-event contract; resubmit with a new rollout ID instead.

Managed and Self-Hosted Environments

Osmosis Platform Harbor rollouts use EnvironmentType.SKYPILOT. The Platform builds the selected task’s Dockerfile and provides the compatible SkyPilot runtime; you do not build or push an image, configure registry credentials, or select a cluster. EnvironmentType.DOCKER remains useful in a self-hosted SDK harness with a Docker daemon, but the managed rollout server does not provide that daemon.

Get started

Core API

Integrations

Execution

Agent tools

HarborBackend Task and Dataset Execution

Install and Import

Choose a Task Mode

Template Mode

Dataset Mode

Integrate an Existing Harbor Dataset

Dynamic Task Sources

Agent Tracks

Reward Source and Precedence

Constructor Reference

Prewarming, Capacity, and Cancellation

Managed and Self-Hosted Environments

Next Steps

Execution Backend Overview

v0.3 Migration

​Install and Import

​Choose a Task Mode

​Template Mode

​Dataset Mode

​Integrate an Existing Harbor Dataset

​Dynamic Task Sources

​Agent Tracks

​Reward Source and Precedence

​Constructor Reference

​Prewarming, Capacity, and Cancellation

​Managed and Self-Hosted Environments

​Next Steps

Execution Backend Overview

v0.3 Migration

Install and Import

Choose a Task Mode

Template Mode

Dataset Mode

Integrate an Existing Harbor Dataset

Dynamic Task Sources

Agent Tracks

Reward Source and Precedence

Constructor Reference

Prewarming, Capacity, and Cancellation

Managed and Self-Hosted Environments

Next Steps