Suggestions and previews

How feedback becomes a proposed change to your spec, how a preview tests a change against past cases, and how an accepted change reaches production.

The examples on this page use Northpine, a fictional outdoor-gear shop whose refund-exception decision grants, escalates or denies refund requests outside its standard return policy. How Moonbounce works introduces it.

The Review page showed how reviews and your feedback settle the right outcome for past cases. This page follows those answers into a suggestion, a proposed change to your spec, and into a preview, which replays a change against past cases so you can decide whether to take it.

The gap reviews find

Northpine’s spec has no feature about shipping. After a few days of feedback and reviews, a pattern shows up: tickets where the carrier delivered late keep landing on escalate, and the owner keeps saying grant. Ticket C is typical: a tent that arrived two weeks late, used once, $189, 39 days after delivery. It scored 0.540, between the escalate cut (0.38) and the grant cut (0.60). The owner’s reason: “The carrier delay ate their return window; that’s our failure.”

No answer to the five existing features can say “the carrier was late”. The loop has to notice that a feature is missing.

What a suggestion is

A suggestion is one proposed change to your spec. Today it is one of two kinds:

• feature_rewrite: new wording for one existing feature (its prompt, criteria or guidance).
• new_feature: one new boolean feature, added as a term of the combine that feeds the output (here grant_score).

It is made against a specific version of your spec, its base (structure and weights hashes), and it carries the change as a diff against that base, the full candidate spec, and the evidence from a preview of the candidate. A suggestion never changes your cut points, and it never writes to your code: you or your coding agent apply it, and it ships when you merge.

How the suggestion writer picks a change

The suggestion writer starts from the cases it is allowed to see: your seed examples and reviewed cases outside the holdout (see Evidence labels). It replays the base on them and looks at each case whose outcome is wrong.

Which feature is to blame

For each wrong case it asks, feature by feature: if this answer were firmly the opposite (a yes of 0.5 or more becomes 0, anything lower becomes 1), would the case land on the outcome it should have? Each feature that passes this test is a suspect, and the case’s blame is split evenly among its suspects.

Two kinds of correction change the test. A feature claim that disagrees with the engine (yours, or a reviewer marking a feature wrong) names that feature directly, and no flipping is done. A feature claim that agrees with the engine confirms that answer, so that feature is never a suspect.

Ticket C shows why this matters. Flip outside_window from 0.95 to 0 and C scores 0.730, which is grant. Flip defective from 0.05 to 1 and it also scores 0.730. Without more information, blame would split between those two features. But both answers were right: the request was outside the window, and nothing was defective. The owner said so in their corrected feedback on ticket C, from Feedback:

{
  "about": { "evaluation_id": "evl_0199f2c4-1a2b-7c3d-8e4f-5a6b7c8d9e0f" },
  "claims": [
    { "kind": "outcome", "value": "grant" },
    { "kind": "feature", "feature": "outside_window", "value": true },
    { "kind": "feature", "feature": "defective", "value": false }
  ],
  "source": "owner",
  "reason": "The carrier delay ate their return window; that's our failure.",
  "supersedes": "fbk_0199f2c9-3b4c-7d5e-9f60-718293a4b5c6"
}

With those two answers confirmed, the remaining flips all move C down (to 0.410, 0.344 and 0.346), never to grant. No single feature explains C. A case like that is a missing criterion: the spec lacks a feature it needs.

Rewrite or add

Across all the wrong cases, the writer compares the most-blamed feature’s total blame with the number of missing-criterion cases. If the blame is at least as large, it proposes a rewrite of that feature; otherwise it proposes a new feature. The owner confirmed the same two answers on each carrier-delay ticket they judged, so those are all missing-criterion cases. They outnumber the blame on any single feature (a correction like ticket D’s in Review blames item_used for one case), so the writer adds a feature.

A rewrite also sees missing-criterion cases and their reasons as context, not as failures attributed to the rewritten feature. Blamed cases get priority within the shared cap of 20 wrong cases.

A language model drafts the words: new wording for the blamed feature’s prompt, or a name and prompt for the new feature. It sees the base spec and the text of the wrong and right cases it may see, with their reasons and corrections. Moonbounce, not the model, builds the diff, checks that the candidate is a valid spec with a new structure hash, and replays each draft on the seed and training cases. The draft that fixes the most cases net of breaks, if any fixes more than it breaks, gets a full preview that includes the held-out cases, and it is proposed only if that preview passes the propose rule.

The Northpine suggestion

GET /v1/suggestions/{suggestion_id} returns the suggestion. Abbreviated: the full response also has candidate_spec, each version’s engine_profile, the generator that wrote it, status_at, and the evidence counted by feedback source (by_source), and changed_items lists all five fixes, not only ticket C’s.

{
  "id": "sug_01a17b2c-4d5e-7f60-8a91-b2c3d4e5f607",
  "decision_id": "refund-exception",
  "kind": "new_feature",
  "target": "structure",
  "summary": "Adds a feature for late or lost deliveries, which the owner treated as Northpine's failure and granted even outside the window.",
  "environment": "production",
  "status": "proposed",
  "base": {
    "structure": "sha256:c21908531c5bb1e922591c1b6d2d9e238205a6ee4c87cddff4674003e880b313",
    "weights": "sha256:d1340418a598e440b1b7680807d525414e7b2ad6801caab117d123f731e62eba"
  },
  "candidate": {
    "structure": "sha256:9e2ee8b452b25c5023a874d7bcb1ce91c93e9c604454887e058f4bd9f3b09e2e",
    "weights": "sha256:9ba920828d06d7f00d2a5e2b5635f5af2f3be457f039835afb31168e6d3d95a5",
    "weights_pin": "default"
  },
  "requires": [],
  "diff": [
    {
      "op": "add",
      "path": "/nodes/carrier_delay",
      "after": { "type": "boolean", "prompt": "Did the carrier deliver the order late, or lose it?" }
    },
    {
      "op": "add",
      "path": "/nodes/grant_score/terms/-",
      "after": { "node": "carrier_delay", "effect": "raises" }
    }
  ],
  "evidence": {
    "label": "holdout",
    "frozen_at": "2026-10-27T15:02:11.482Z",
    "by_split": {
      "seed": { "n": 12, "fixes": 1, "breaks": 0, "base_correct": 9, "candidate_correct": 10 },
      "train": { "n": 10, "fixes": 3, "breaks": 0, "base_correct": 5, "candidate_correct": 8 },
      "holdout": { "n": 1, "fixes": 1, "breaks": 0, "base_correct": 0, "candidate_correct": 1, "holdout_uses": 1, "owner_decided": 1 }
    },
    "noise_rate": 0,
    "action_mix": {
      "before": { "deny": 0.22, "escalate": 0.62, "grant": 0.16, "abstain": 0 },
      "after": { "deny": 0.26, "escalate": 0.44, "grant": 0.3, "abstain": 0 },
      "on": "newest 50 of 50 primary production evaluations, last 7 days",
      "n": 50
    },
    "changed_items": [
      {
        "evaluation_id": "evl_0199f2c4-1a2b-7c3d-8e4f-5a6b7c8d9e0f",
        "feedback_id": "fbk_0199f2d1-7e8f-7a90-b1c2-d3e4f5a6b7c8",
        "split": "train",
        "change": "fix",
        "before": { "outcome": "escalate", "score": 0.54, "features": { "outside_window": 0.95, "item_used": 0.35, "defective": 0.05, "repeat_refunder": 0.02, "high_value": 0.03 } },
        "after": { "outcome": "grant", "score": 0.6116666666666666, "features": { "outside_window": 0.95, "item_used": 0.35, "defective": 0.05, "repeat_refunder": 0.02, "high_value": 0.03, "carrier_delay": 0.97 } }
      }
    ],
    "credited_feedback": ["fbk_0199f2d1-7e8f-7a90-b1c2-d3e4f5a6b7c8"]
  },
  "preview_id": "run_01a17b2c-4c90-7b11-9e35-c7a2d6f0e148",
  "created_at": "2026-10-27T15:02:14.906Z"
}

The diff is a list of edits addressed by JSON pointers into the spec, all under /nodes/. add creates a node or appends a term (/terms/-); replace and remove carry a before, the value they expect to find, so an edit made against a spec that has since changed fails instead of overwriting your work. A rewrite looks like {"op": "replace", "path": "/nodes/item_used/prompt", "before": "...", "after": "..."}.

environment is the environment of the key that created the suggestion. The whole loop runs in one environment: a suggestion learns from that environment’s cases, and only another key in the same environment can read it. Only a production suggestion can be accepted, so the developer’s agent asks with a production key. requires lists suggestions that must land first; it is empty today. credited_feedback lists the feedback the writer saw that decided a case the candidate fixed. The evidence fields are explained under Previews.

Weights for a new feature

The candidate’s weights_pin is the weights pin to write in your decision file, and candidate.weights is what that pin resolves to. Northpine pins "default", so the suggestion keeps "default", and "default" resolves to the candidate spec’s own default weights. Those are recomputed for six terms: each weight is now about 0.167 instead of 0.2, and the bias is 4/6 (about 0.667) instead of 0.8. With carrier_delay answered 0.97 for C, 0.02 for A and 0.01 for B:

Ticket	Base score	Candidate score	Outcome at Northpine’s cut points
A: zipper ripped	0.766	0.642	grant, still grant
B: kayak, changed mind	0.362	0.303	deny, still deny
C: tent, carrier late	0.540	0.612	escalate, now grant

C crosses the grant cut at 0.60. The other fixes are the same story: the tent seed example, two other carrier-delay tickets in training, and one held-out case.

Look at A too. Under default weights every term counts equally, so a sixth feature that is “no” for most tickets lowers their scores by about a sixth: A falls from 0.766 to 0.642. It’s still grant, but closer to the line. A change like this can break cases that sit just above a cut, which is why the preview counts breaks and replays recent traffic in the action mix.

If your file pins a weights hash instead, a rewrite keeps that hash. A new feature under a pinned hash would get no weight at all, so the suggestion ships a new pin: carried weights, which keep every existing weight and the bias and give the new term the average size of the others. A pin equal to your spec’s default weights hash counts as "default".

Previews

POST /v1/decisions/{decision_id}/previews replays a candidate spec and its base over the decision’s past cases and reports what would change. It takes the spec as it is in your code, the weights pin as written, and your cut points; with a decision file, jq builds the body:

curl -sS --fail-with-body -X POST "$MOONBOUNCE_API_URL/v1/decisions/refund-exception/previews" \
  -H "Authorization: Bearer $MOONBOUNCE_API_KEY" \
  -H 'Content-Type: application/json' \
  -H "Idempotency-Key: $(uuidgen)" \
  -d "$(jq '{spec, weights, outcomes, suggestion_id: "sug_01a17b2c-4d5e-7f60-8a91-b2c3d4e5f607"}' moonbounce/refund-exception.json)"

What it compares:

• The base. The suggestion’s base when you send suggestion_id; else the base you send ({"structure", "weights"}); else the version serving primary traffic in your key’s environment; else the latest version seeded there. With none, base is null and nothing changes.
• The cases. Past evaluations in your key’s environment with stored content whose right outcome was decided by you, by reviewers or by a seed example, and is one of your request’s outcomes. Your app’s own feedback never decide a case. Each item group counts once, newest first; contested cases are left out.
• The cut points. Both versions are cut at the cut points you send, never at the ones each evaluation used, so the only difference between before and after is the spec and weights.
• A fair baseline. A feature the candidate changed is asked again for the base in the same preview, so both sides carry the engine’s current noise. noise_rate is how often that fresh base answer gave a different outcome from the stored one: the noise floor. It is 0 when no feature changed, as for a pure addition. Because the changed feature is asked afresh each time, its before answer and score can move a little from one preview to the next. Features the candidate leaves alone replay cached answers when there are any; a test key’s preview keeps no cache between runs, so there every feature is asked afresh.

The response has base and candidate (each {structure, weights, engine_profile}), hashes_match (with suggestion_id: whether your spec and pin hash to that suggestion’s candidate), evidence, and cost (engine calls, cache hits, input tokens and usd_micros, in millionths of a dollar). In evidence:

• by_split counts, per set of cases, n, fixes (wrong before, right after), breaks (right before, wrong after), and how many each version got right. A case the engines failed on temporarily is left out; a version that fails a case permanently counts as wrong on it.
• action_mix replays a sample of recent primary traffic in your key’s environment (the on field says which) under both versions and gives the share of each outcome. Northpine’s grant share would rise from 16% to 30% as carrier-delay tickets stop escalating, and its deny share would edge up from 22% to 26% as other scores drop a little.
• changed_items lists every fix and break by ids, scores and answers, never content, and burns any held-out case it lists.

A preview doesn’t register the candidate. To evaluate by its structure hash, register the spec first with POST /v1/decisions/{decision_id}/structures, or send the spec inline on one evaluate call.

A preview writes a record of the run and nothing else: no evaluation, no traffic, and never an acceptance.

Changing cut points in the same change

A suggestion leaves your cut points alone, but a preview takes whatever cut points you send, so you can test a structure change and a cut-point change together. Preview Northpine’s candidate with the grant cut at 0.65 instead of 0.60, and both versions are cut there: A (0.766 before, 0.642 after) breaks from grant to escalate, and C (0.612 after) stays on escalate. That preview would show the new feature doesn’t pay off at a stricter cut, and the action mix at those cut points shows what recent traffic would get. Cut points mint no version, so the hashes still match the suggestion.

Evidence labels

Every preview gets a label for the strongest set of cases it was tested on.

Label	Tested on	How much to trust it
Starting examples	Your seed examples only	Weakest: the writer learned from them, so it shows the change is consistent with your examples, not that it generalizes
Training cases	Reviewed cases the writer could see	Real traffic, but still cases the change was made to fit
Held out	Reviewed cases the writer never saw	Strongest available: a test the change could not be tuned to

About 30% of reviewed cases are held out. An item’s side is fixed by the first feedback it gets (near-duplicate items share one side): a seed example is always a seed, your app’s own feedback put it in training, and feedback from you or a reviewer puts it in the holdout about 30% of the time, decided by a hash, never by the outcome or the answers. Held-out content never reaches the suggestion writer.

Two caveats sit in the numbers. owner_decided counts held-out cases you decided yourself: never seen by the writer is not never seen by you. And holdout_uses counts the previews that have scored the holdout; a holdout reused many times is less of an independent test.

Shown cases become training

A held-out case listed in a preview’s changed_items has been shown to you or your agent, so it is burned: it counts as a training case from then on. Previewing the Northpine suggestion again after it was proposed gives:

{
  "label": "training_cases",
  "by_split": {
    "seed": { "n": 12, "fixes": 1, "breaks": 0, "base_correct": 9, "candidate_correct": 10 },
    "train": { "n": 11, "fixes": 4, "breaks": 0, "base_correct": 5, "candidate_correct": 9 }
  }
}

The one held-out case it fixed was listed in the proposal’s evidence, so it moved to training, and with no held-out cases left the label drops. That is expected. The evidence stored on the suggestion is the observation made when it was proposed, and it doesn’t change; quote that label in your pull request beside the new preview’s numbers. Every preview that lists held-out cases burns them, so repeated previews use up the holdout.

A further label, Validated (tested on reviews that arrive after the change was frozen), is not available yet.

The propose rule

A suggestion is proposed only when its candidate fixes more cases than it breaks on its own label’s set: the seed examples for Starting examples, the training cases for Training cases. On Held out it must also beat the noise floor: fixes minus breaks must be greater than noise_rate times the number of held-out cases. Northpine’s suggestion passed on one held-out fix, no breaks and a noise floor of 0. When the best candidate fails the rule, nothing is proposed and no held-out case is burned.

Applying a suggestion

Your coding agent applies it. Over MCP (Connect your agent), apply_suggestion with the suggestion id returns step-by-step instructions, the diff and the candidate spec. The agent:

1. Finds the decision file

   Usually moonbounce/refund-exception.json; the spec may also be inline in code.
2. Checks the base

   If the local spec’s structure hash equals the suggestion’s base, it replaces the spec with the candidate. If not, it applies only the diff entries whose before still matches, and tells you which it skipped.
3. Sets the weights pin

   To weights_pin: here it stays "default", which already resolves to the candidate’s weights. Writing the candidate’s weights hash instead fails with WEIGHTS_UNKNOWN until that structure has been registered.
4. Runs the preview

   With suggestion_id. hashes_match must be {"structure": true, "weights": true} when every entry applied.
5. Opens a pull request

   On a branch, with your own git credentials, carrying the summary, the label the suggestion was proposed with, and the preview’s numbers. It then marks the suggestion pulled.

You review and merge. Nothing reaches production except through your merge: Moonbounce never writes to your repository.

Statuses and on-demand suggestions

A suggestion is proposed when made. POST /v1/suggestions/{suggestion_id}/status with {"status": "pulled"} marks it picked up; {"status": "rejected", "reason": "..."} turns it down (reason only goes with rejected). Proposed can become pulled, and proposed or pulled can become rejected; repeating the current status changes nothing, and anything else is 409. accepted is never set by you. GET /v1/decisions/{decision_id}/suggestions lists a decision’s suggestions newest first, filtered by status and paged with limit (up to 100) and cursor.

A suggestion read (get or list) carries stale: { "reason": "...", "cases": 2 } when feedback on cases it was evaluated on changed after its evidence was read: a reviewer changed or withdrew an answer, you corrected or judged one again, another reviewer answered one, or new feedback marked one’s input as broken. Its evidence is never rescored; preview it again for current numbers.

POST /v1/decisions/{decision_id}/suggestions asks for one now. The body may be empty: the base is then the version serving primary traffic in your key’s environment (else the latest version seeded there), at the cut points its latest evaluation recorded. You can send spec and weights to start from your local version (the spec must already be registered, or it is 404 VERSION_UNKNOWN), outcomes to set the cut points (400 OUTCOMES_INVALID when none are known), and kind to generate that kind even when blame points to the other. A forced new_feature uses missing-criterion cases when there are any, otherwise all visible wrong cases within the prompt cap. A forced feature_rewrite rewrites the top suspect, or returns no suggestion when there is no suspect. The replay and propose rules still decide whether the candidate is proposed. The answer is {"suggestion": ..., "reason": ...}:

• suggestion: null with a reason when there is nothing worth proposing yet: no cases to learn from, the spec already gets them right, there is no suspect to rewrite or no combine for a new feature to join, or no candidate passed the propose rule.
• The existing suggestion, instead of a duplicate, when the same candidate was proposed before.
• 404 when the decision has no evaluations or seeds to learn from.
• 503 ENGINE_UNAVAILABLE with Retry-After when no suggestion writer is configured, the writer failed, or the engines could not score the cases. Retry with the same Idempotency-Key.

Acceptance

A suggestion is accepted when its candidate pair (structure hash and weights hash) first serves primary production traffic, POST /v1/evaluate on a production key, after the suggestion was made. Previews, seeds and test keys never count. A pair that already served production before the suggestion existed is a revert, not an acceptance. Only proposed or pulled suggestions made with a production key can be accepted; a test key’s suggestions never are. It doesn’t matter how the change reached your code.

Moonbounce records acceptance when you next read the suggestion, the list or the loop status with a production key. GET /v1/decisions/{decision_id}/loop shows where the loop stands:

{
  "decision_id": "refund-exception",
  "accepted": [
    {
      "suggestion_id": "sug_01a17b2c-4d5e-7f60-8a91-b2c3d4e5f607",
      "evaluation_id": "evl_01a17c40-1a2b-7c3d-8e4f-5a6b7c8d9e0f",
      "accepted_at": "2026-10-28T14:11:52.317Z",
      "evidence_label": "holdout",
      "holdout_win": true
    }
  ],
  "reviews": 24,
  "random_lane": { "n": 8, "agree": 5, "nonresponse": 0.2, "changed_after_reveal": 0 },
  "spot_check": { "before": { "n": 8, "agree": 5 }, "after": { "n": 0, "agree": 0 } },
  "stop_check": { "reviews": 24, "threshold": 40, "any_holdout_win": true }
}

accepted carries the label each suggestion was proposed with and whether that evidence was a held-out win. reviews counts live feedback from you and reviewers. random_lane and spot_check report the blind spot checks from Review: how often they agreed with Moonbounce, on the version that served production before the first acceptance and on accepted versions. They read each reviewer’s first, blind answer; changed_after_reveal counts the spot checks whose answer was changed afterwards. stop_check is a prompt to rethink: after threshold reviews with no suggestion that won on held-out cases, more reviews of the same decision are unlikely to help.

Next

Node types and patterns lists every node type, including the ones coming soon, and shows how grouping carrier_delay with defective would have helped ticket C while diluting ticket A less.