Webhooks

Webhooks notify your systems when supported runs reach a terminal state. Osmosis sends a JSON POST to the workspace URL you configure for training, evaluation, and benchmark run completion events.

Setup

Open Workspace Settings → Webhooks. Any workspace member can view the page and delivery history. Only owners and admins can edit the URL, turn delivery on or off, or send a test payload:

Enter your Webhook URL. It must use HTTPS on port 443 and resolve to a public address. URLs that point at private or internal networks are rejected.
Turn on Deliver events and click Save.
Click Send test payload to confirm your receiver gets a webhook.test event.

Turn off Deliver events to pause deliveries without losing the URL. To remove the webhook, clear the Webhook URL and click Save.

Events

Event	Sent when
`training_run.completed`	A training run reaches a terminal status
`eval_run.completed`	An evaluation run reaches a terminal status
`benchmark_run.completed`	A benchmark run reaches a terminal status
`webhook.test`	You click Send test payload

Payload

Every request is a POST with Content-Type: application/json and this envelope:

{
  "event": "training_run.completed",
  "timestamp": "2026-07-15T22:14:05.123456+00:00",
  "data": {}
}

`training_run.completed` and `eval_run.completed`

The data object has the same shape for both events:

Field	Description
`id`	Run ID
`name`	Run name
`status`	Terminal run status, such as `finished`, `failed`, or `stopped`
`started_at`	Start time in ISO 8601 format, or `null` if the run never started
`completed_at`	Completion time in ISO 8601 format, or `null` when unavailable
`duration_seconds`	Total run duration in seconds, or `null` when no duration is available
`model_name`	Model name used by the run, or `null`
`dataset`	Dataset `{ "id", "name" }`, or `null`
`rollout`	Rollout `{ "id", "name" }`, or `null`
`platform_url`	Link to the run detail page
`latest_metrics`	Training: the latest logged value for each available metric. Evaluation: final aggregate results, including `reward_stats` when available

Training run example:

{
  "event": "training_run.completed",
  "timestamp": "2026-07-15T22:14:05.123456+00:00",
  "data": {
    "id": "a1b2c3d4-0000-0000-0000-000000000000",
    "name": "my-training-run",
    "status": "finished",
    "started_at": "2026-07-15T20:01:12+00:00",
    "completed_at": "2026-07-15T22:14:03+00:00",
    "duration_seconds": 7971,
    "model_name": "Qwen3-8B",
    "dataset": { "id": "d1e2f3a4-0000-0000-0000-000000000000", "name": "my-dataset" },
    "rollout": { "id": "r1s2t3u4-0000-0000-0000-000000000000", "name": "my-rollout" },
    "platform_url": "https://platform.osmosis.ai/my-workspace/training/a1b2c3d4-0000-0000-0000-000000000000",
    "latest_metrics": {
      "rollout/raw_reward": 0.82,
      "eval/validation/reward": 0.79,
      "train/entropy_loss": 1.24,
      "rollout/response_lengths": 512,
      "rollout/total_lengths": 1930,
      "rollout/truncated_ratio": 0.03
    }
  }
}

For evaluation runs, latest_metrics contains aggregate results instead:

"latest_metrics": {
  "total_runs": 200,
  "graded": 198,
  "passed": 154,
  "failed": 44,
  "skipped": 2,
  "pass_rate": 0.78,
  "pass_threshold": 0.5,
  "score": 0.71,
  "reward_stats": {
    "mean": 0.71,
    "median": 0.74,
    "std": 0.18,
    "min": 0.05,
    "max": 0.98,
    "pass_at_k": { "1": 0.78, "4": 0.91 }
  },
  "total_tokens": 1834520,
  "total_duration_ms": 5421000
}

`benchmark_run.completed`

Field	Description
`id`	Benchmark run ID
`name`	Run name
`status`	Terminal run status, such as `finished`, `failed`, or `stopped`
`benchmark_name`	Benchmark name
`started_at`	Start time in ISO 8601 format, or `null` if the run never started
`completed_at`	Completion time in ISO 8601 format, or `null` when unavailable
`duration_seconds`	Total run duration in seconds, or `null` when no duration is available
`platform_url`	Link to the benchmark run detail page
`agents`	One result summary per agent in the run

Each object in agents contains:

Field	Description
`id`	Benchmark run agent ID
`agent_index`	Agent’s zero-based position in the run configuration
`label`	Agent display label
`model`	Model identifier, or `null`
`tasks`	Number of tasks represented in the result summary
`attempts`	Number of attempts represented in the result summary
`passed`	Number of passing attempts
`pass_rate`	Passing attempts divided by attempts, or `null` when `attempts` is zero

{
  "event": "benchmark_run.completed",
  "timestamp": "2026-07-15T22:14:05.123456+00:00",
  "data": {
    "id": "b1c2d3e4-0000-0000-0000-000000000000",
    "name": "terminal-bench-smoke",
    "status": "finished",
    "benchmark_name": "Terminal-Bench 2.1",
    "started_at": "2026-07-15T21:40:00+00:00",
    "completed_at": "2026-07-15T22:14:03+00:00",
    "duration_seconds": 2043,
    "platform_url": "https://platform.osmosis.ai/my-workspace/benchmarks/runs/b1c2d3e4-0000-0000-0000-000000000000",
    "agents": [
      {
        "id": "c1d2e3f4-0000-0000-0000-000000000000",
        "agent_index": 0,
        "label": "codex",
        "model": "openai/gpt-5.2",
        "tasks": 10,
        "attempts": 10,
        "passed": 7,
        "pass_rate": 0.7
      }
    ]
  }
}

`webhook.test`

Field	Description
`workspace`	Workspace name
`message`	Confirmation message

Delivery

Each attempt times out after 5 seconds. Failed deliveries are retried up to 2 times with backoff.
Retries can produce duplicate deliveries. For run-completion events, make processing idempotent using the pair event + data.id.
Any 2xx response counts as delivered. Redirects are not followed.
Delivery is best effort. A webhook delivery failure does not affect the run, and Osmosis does not guarantee that every event will reach your receiver.
Every delivery is recorded with its final outcome in Recent Deliveries, so you can see what Osmosis sent and how your receiver answered.

Recent Deliveries

Workspace Settings → Webhooks lists every delivery from the last 30 days below the webhook form. Any workspace member can read the list; editing the webhook still requires an owner or admin. Each row shows:

Column	Description
Event	The event name and, for run events, the run name linked to its detail page
Status	Delivered when your receiver returned a 2xx, otherwise Failed
Response	The HTTP status code of the final attempt. When no response was received, the reason instead: Timed out, Unreachable, or Rejected (the URL failed validation, so nothing was sent)
Attempts	How many requests Osmosis made, including retries. `0` means the URL was rejected before the first request
Sent	When the final attempt finished

Click a row to open the delivery. The Request tab shows the exact JSON body Osmosis sent. The Response tab shows the HTTP status and the first 2 KB of your receiver’s response body, or the transport error when no response arrived. Filter the list by event or status to find a specific delivery. Test payloads sent with Send test payload appear in the list like any other event, so the fastest way to inspect a payload is to send one and open the row. Deliveries are kept for 30 days and then removed. Nothing is recorded while Deliver events is off, because no request is made.

Osmosis stores the first 2 KB of your receiver’s response body so you can debug failures. Do not echo secrets or other sensitive data in webhook responses.

Security

Webhook requests are currently unsigned, so the payload alone does not prove that a request came from Osmosis. Use a dedicated, high-entropy HTTPS URL that is not published elsewhere, validate the JSON schema and expected event values before processing, and apply the idempotency rule above. Do not trigger privileged actions solely from unvalidated payload fields. If signed authentication is required, place the webhook behind a gateway you control and enforce your own receiver policy.

Get started

Runs

Resources

Workspace

Setup

Events

Payload

`training_run.completed` and `eval_run.completed`

`benchmark_run.completed`

`webhook.test`

Delivery

Recent Deliveries

Security

​Setup

​Events

​Payload

​training_run.completed and eval_run.completed

​benchmark_run.completed

​webhook.test

​Delivery

​Recent Deliveries

​Security

Setup

Events

Payload

`training_run.completed` and `eval_run.completed`

`benchmark_run.completed`

`webhook.test`

Delivery

Recent Deliveries

Security