Custom Rollout Guide

Use the workspace’s Agent Skills to turn a task, dataset, or existing agent into a runnable Osmosis rollout. For a known-good example with no design work, start with the Multiply Quickstart.

This guide assumes you have completed Onboarding: your workspace repository is cloned, the CLI is installed and authenticated, and your AI coding environment is open in the workspace directory.

How Workspace Skills Work

Platform-created workspace repositories include project-local Agent Skills under .agents/skills/. An agent that loads these skills can select the workflow for the current stage:

plan from dataset -> create rollout -> submit evaluation run -> debug failures -> prepare training run

The skills call the Osmosis CLI for workspace checks, dataset validation, evaluation runs, and training preflight. They do not replace the CLI.

When to Use This Path

Use this path when	Run the Multiply example when
You already know the task you want to train on	You want proof that the platform works end to end
You have sample data or a platform dataset	You do not want to design a dataset yet
You want the agent to create or adapt rollout code	You want to copy commands without making product choices
You are comfortable inspecting generated code and evaluation run output	You are still learning the Osmosis workflow

Use the Skills in Your Workspace Repository

Open your platform-created workspace repository in your AI coding environment. The repository includes the workspace contract and Agent Skills alongside rollout code, configs, and data:

repository/
├── .agents/
│   └── skills/
├── .claude/
│   └── skills/
├── rollouts/
├── configs/
│   ├── eval/
│   └── training/
├── data/
├── AGENTS.md
├── CLAUDE.md
└── pyproject.toml

AGENTS.md contains the always-loaded workspace contract. .agents/skills/ contains the canonical workflow skills, and .claude/skills/<skill-name> exposes the same skills to Claude Code through symlinks back to .agents/skills/.

Verify the Workspace

Before asking the agent to write rollout code, confirm that the CLI can resolve the workspace:

osmosis doctor
osmosis auth whoami

Ask the Agent to Plan from the Dataset

Describe the task and ask the agent to begin with the workspace’s planning skill:

I want to train a model for <task> in this Osmosis workspace. Start with the `plan-training` skill: read the workspace instructions, help me settle the dataset plan, and propose the next step before creating rollouts, running evaluation runs, or submitting a training run.

The workspace skills guide the agent through these stages:

Plan training

Inspect data/, existing rollouts, and workspace config, then settle the dataset schema before writing rollout code.

Create or adapt a rollout

Write the smallest AgentWorkflow and Grader that can load, run, and score samples. Generated files should stay under rollouts/, configs/eval/, configs/training/, and data/.

Submit an evaluation run

Push the rollout to the workspace repository and submit an evaluation run as the quality gate:

git push
osmosis eval submit configs/eval/<name>.toml
osmosis eval info <eval-run-name>

Debug until the evaluation run is clean

Fix loading, dataset, grader, dependency, and reward issues before training. A healthy evaluation run is the handoff from creation to training readiness.

Prepare a training run

Once the rollout is validated, let the agent inspect the training run config and run submit-time preflight. Submit only when you are ready to start a platform training run.

Do not skip the evaluation run gate. osmosis train submit should be the step after the rollout cleanly loads, runs, and grades samples on the platform.

Workspace Skills

The workspace skills are organized around rollout creation stages:

Skill	Purpose
`plan-training`	Turn a task idea or dataset into a concrete experiment plan
`create-rollouts`	Create or adapt rollout code, graders, entrypoints, and baseline evaluation configs
`evaluate-rollouts`	Submit evaluation runs, compare baselines, and inspect failures
`debug-rollouts`	Diagnose evaluation, config, dataset, dependency, or preflight failures
`submit-training`	Prepare a training run config, submit a training run, and check training run status

You usually do not need to invoke a skill by name. Describe the outcome; an agent that loads the repository’s Agent Skills selects the matching workflow.

Next Steps

SDK Overview

Understand the AgentWorkflow and Grader contract behind generated rollout code.

Evaluation

Validate rollouts with an evaluation run before submitting a training run.

Git Sync

Push rollout changes and let the platform sync the code version used for evaluation runs and training runs.

Training Runs

Submit and monitor a training run after the evaluation run passes.

Get started

Core API

Integrations

Execution

Agent tools

How Workspace Skills Work

When to Use This Path

Use the Skills in Your Workspace Repository

Verify the Workspace

Ask the Agent to Plan from the Dataset

Workspace Skills

Next Steps

SDK Overview

Evaluation

Git Sync

Training Runs

​How Workspace Skills Work

​When to Use This Path

​Use the Skills in Your Workspace Repository

​Verify the Workspace

​Ask the Agent to Plan from the Dataset

​Workspace Skills

​Next Steps

SDK Overview

Evaluation

Git Sync

Training Runs

How Workspace Skills Work

When to Use This Path

Use the Skills in Your Workspace Repository

Verify the Workspace

Ask the Agent to Plan from the Dataset

Workspace Skills

Next Steps