QueryHippo Field guide / Docs
Docs menu

TOOLS

get_change_outcome

Did a change we recorded on a page work?

  • Read-only
  • No external calls

When to use it

Measures one recorded change: it compares the exact page over equal windows before and after the deploy date, leaving the deploy day out, and gives a verdict of improved, no_change, worse, insufficient_volume, not_ready or unavailable. Use it to close or follow up on a ticket once the after-window has complete data.

Ask your AI

  • Did the title rewrite on /compare/svelte-virtuallists work?
  • Which of last month's changes can be judged yet?
  • Check the outcome of the HUM-52 change with 14-day windows.

Parameters

NameTypeDetailsWhat it means
changeId(required)string1–200 characterspattern ^[a-zA-Z0-9._:-]+$The ID of the change marker, as returned by record_change or list_changes. Removed markers aren't measured.
window_daysintegerdefault 28minimum 7maximum 90How many days each window covers, from 7 to 90. Defaults to 28. The before-window ends the day before the deploy.
settle_daysintegerdefault 0minimum 0maximum 28Days after the deploy to skip before the after-window starts, from 0 to 28. Defaults to 0.
url_variants"auto" | "merge" | "exact"default "auto"auto (default) combines the page with its trailing-slash twin only when traffic moved between them; merge always combines them; exact measures only the recorded URL.

What it returns

The change (including its note and ticket reference), the windows used, whether data has reached the end of the after-window and, if not, the day it will and how many days remain. Once ready, the verdict with its confidence, the metric it was judged on, the page's clicks, impressions, click-through rate and position before and after, the site's own change over the same windows, how traffic split between the page and its trailing-slash twin, with a variant evidence ID when it was checked, the page's top queries, warnings, and page and site evidence IDs for get_evidence_for_change. With too few impressions it suggests a longer window and when that window completes. The versioned policy it used is included in every result.

Notes

  • The verdict is a fixed threshold rule, not a statistical significance test. Each window needs at least 100 impressions on the page.
  • It measures before and after; it doesn't prove the change caused the difference. If the page's average position moved by 1 or more, or the whole site moved the same way, the confidence is lowered to limited.
  • Title and meta description changes are judged on click-through rate (and the extra clicks it produced); every other kind is judged on clicks.
  • Until the after-window has complete data it returns not_ready with the day it can be called, without storing anything.
  • Google counts /page and /page/ as different pages. If most of the traffic moved from one to the other between the windows, for example after a redirect, the verdict uses both combined, confidence is limited, and each URL's own numbers are included. If only part of the traffic has moved, the verdict stays on the recorded URL but confidence is limited, because Google can take weeks to finish moving traffic after a redirect.
  • When both URLs are combined, a query listed for both URLs usually just moved with the redirect: a loss on the old URL and a gain on the new one are the same searches, not two separate findings.
  • Read-only: it reads data QueryHippo has already stored. It does not start a collection, fetch anything or call an AI model.

Description your AI sees

Measure a recorded change: compare its exact page over equal windows before and after the deploy date (deploy day excluded) and apply a disclosed, versioned verdict: improved, no_change, worse, insufficient_volume, not_ready or unavailable, with confidence, the site's own trend over the same windows, and page and site evidence IDs. Also checks the page's trailing-slash twin; when traffic moved between them (a redirect), judges the combined page with limited confidence and shows each URL's numbers; a partial move limits confidence. Says when the result can be called. Measured, not causal. No external calls.

Full schemas

Input schema 
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "changeId": {
      "type": "string",
      "minLength": 1,
      "maxLength": 200,
      "pattern": "^[a-zA-Z0-9._:-]+$"
    },
    "window_days": {
      "default": 28,
      "type": "integer",
      "minimum": 7,
      "maximum": 90
    },
    "settle_days": {
      "default": 0,
      "type": "integer",
      "minimum": 0,
      "maximum": 28
    },
    "url_variants": {
      "default": "auto",
      "type": "string",
      "enum": [
        "auto",
        "merge",
        "exact"
      ]
    }
  },
  "required": [
    "changeId"
  ],
  "additionalProperties": false
}

View as Markdown