Preview a spec change against past feedback

Replays a spec change over past cases with feedback: it spends engine calls, stores a run, and burns the held-out cases it lists. Not a dry run of evaluate. Replays a local spec (with its weights pin and cut points) and its base over the decision’s reviewed and seeded cases in the key’s environment (a test key previews against test cases only), and returns fixes, breaks, the action mix at the cut points and an evidence label. Writes an eval run only: never an evaluation, never traffic, never acceptance. A preview does not register the candidate: before evaluating by its structure hash, register the spec with POST /v1/decisions//structures (or send the spec inline on evaluate once). Holdout cases it lists individually (fixes and breaks) are burned (they count as training from then on); unlisted holdout cases are only scored. evidence.by_split.holdout.holdout_uses counts preview runs that scored held-out cases, not burns. The base, in order: the base of suggestion_id when it is sent (then base is ignored), else base, else the version serving primary traffic in the key’s environment, else the latest version seeded there, else none. With suggestion_id, hashes_match says whether the spec is that suggestion’s candidate; another environment’s suggestion is 404. The evidence replays only cases with an outcome from the owner, reviewers or seeds; the action mix replays a sample of the environment’s recent primary traffic. For features the candidate changes, before is a fresh re-run of the base feature (the fair baseline), so engine nondeterminism can move it between previews; unchanged features replay cached answers when available. noise_rate measures outcome flips from that re-run against the cached baseline, not score drift. Holdout burns and use counts are per environment, and a test key’s preview neither reads nor fills the shared cache of engine answers. A decision nothing was registered, evaluated or seeded under is 404.

POST /v1/decisions/{decision_id}/previews

Preview a spec change against past feedback

cURL

curl --request POST \
  --url https://api.moonbounce.ai/v1/decisions/{decision_id}/previews \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "spec": {
    "format": "moonbounce.spec/0",
    "nodes": {},
    "output": "<string>",
    "guidance": "<string>",
    "target": "<string>",
    "meta": {
      "title": "<string>",
      "description": "<string>",
      "owner": "<string>",
      "tags": [
        "<string>"
      ],
      "display": {}
    }
  },
  "weights": "default",
  "outcomes": {
    "below": "<string>",
    "cuts": [
      {
        "at": 123,
        "outcome": "<string>"
      }
    ],
    "abstain": "<string>"
  },
  "suggestion_id": "<string>"
}
'

200

{
  "preview_id": "<string>",
  "decision_id": "<string>",
  "environment": "production",
  "base": {
    "structure": "<string>",
    "weights": "<string>",
    "engine_profile": "<string>"
  },
  "candidate": {
    "structure": "<string>",
    "weights": "<string>",
    "engine_profile": "<string>"
  },
  "hashes_match": {
    "structure": true,
    "weights": true
  },
  "evidence": {
    "label": "starting_examples",
    "frozen_at": "2023-11-07T05:31:56Z",
    "by_split": {
      "seed": {
        "n": 1,
        "fixes": 1,
        "breaks": 1,
        "base_correct": 1,
        "candidate_correct": 1,
        "holdout_uses": 1,
        "owner_decided": 1
      },
      "train": {
        "n": 1,
        "fixes": 1,
        "breaks": 1,
        "base_correct": 1,
        "candidate_correct": 1,
        "holdout_uses": 1,
        "owner_decided": 1
      },
      "holdout": {
        "n": 1,
        "fixes": 1,
        "breaks": 1,
        "base_correct": 1,
        "candidate_correct": 1,
        "holdout_uses": 1,
        "owner_decided": 1
      }
    },
    "noise_rate": 0.5,
    "action_mix": {
      "before": {},
      "after": {},
      "on": "<string>",
      "n": 1
    },
    "changed_items": [
      {
        "evaluation_id": "<string>",
        "feedback_id": "<string>",
        "split": "seed",
        "change": "fix",
        "before": {
          "features": {},
          "outcome": "<string>",
          "score": 123
        },
        "after": {
          "features": {},
          "outcome": "<string>",
          "score": 123
        }
      }
    ],
    "credited_feedback": [
      "<string>"
    ],
    "by_source": {
      "app_native": {
        "n": 1,
        "fixes": 1,
        "breaks": 1,
        "base_correct": 1,
        "candidate_correct": 1,
        "holdout_uses": 1,
        "owner_decided": 1
      },
      "app_resolution": {
        "n": 1,
        "fixes": 1,
        "breaks": 1,
        "base_correct": 1,
        "candidate_correct": 1,
        "holdout_uses": 1,
        "owner_decided": 1
      },
      "reviewer": {
        "n": 1,
        "fixes": 1,
        "breaks": 1,
        "base_correct": 1,
        "candidate_correct": 1,
        "holdout_uses": 1,
        "owner_decided": 1
      },
      "owner": {
        "n": 1,
        "fixes": 1,
        "breaks": 1,
        "base_correct": 1,
        "candidate_correct": 1,
        "holdout_uses": 1,
        "owner_decided": 1
      },
      "import": {
        "n": 1,
        "fixes": 1,
        "breaks": 1,
        "base_correct": 1,
        "candidate_correct": 1,
        "holdout_uses": 1,
        "owner_decided": 1
      },
      "synthetic_review": {
        "n": 1,
        "fixes": 1,
        "breaks": 1,
        "base_correct": 1,
        "candidate_correct": 1,
        "holdout_uses": 1,
        "owner_decided": 1
      }
    }
  },
  "cost": {
    "engine_calls": 1,
    "cache_hits": 1,
    "input_tokens": 1,
    "usd_micros": 1
  }
}

Authorizations

Authorization · string · header · required

An API key, sent as Authorization: Bearer <key>. Each operation lists the scope its key needs: decisions:evaluate (evaluate and read evaluations), decisions:feedback (your app's feedback) or decisions:develop (everything else, including owner feedback and the review queue). An operation listing no scope takes any scoped key. A key with no scope, or without the operation's, gets 403 INSUFFICIENT_SCOPE.

Headers

idempotency-key · string

Makes a retry safe. For 24 hours, the same key with the same request returns the first stored result (with Idempotent-Replayed: true); the same key with a different request is a 422 IDEMPOTENCY_KEY_REUSED. Keys are scoped to the organization and environment across every operation, so use a fresh key per request. A request that failed before its result was stored runs again when retried with its key.

Required string length: 1 - 255

Pattern: ^[\x20-\x7e]+$

Path Parameters

decision_id · string · required

Pattern: ^[a-z][a-z0-9_.-]{0,127}$

Body

application/json

spec · object · required

Show child attributes

weights · enum<string> string · default: default · required

Available options: default

outcomes · object · required

Show child attributes

suggestion_id · string

Pattern: ^sug_[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$

base · object

Show child attributes

Response

200

application/json

The preview

preview_id · string · required

Pattern: ^run_[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$

decision_id · string · required

Pattern: ^[a-z][a-z0-9_.-]{0,127}$

environment · enum<string> · required

The environment of the key that made the call: production or test. Evaluations, feedback, reviews and suggestions are separate per environment; registered structures are shared.

Available options: production, test

base · object | null · required

Show child attributes

candidate · object · required

Show child attributes

hashes_match · object | null · required

Show child attributes

evidence · object · required

Show child attributes

cost · object · required

Show child attributes