Skip to main content
Submit a brief Menu

Measurement

We set up measurement before launch and report what the spend added: conversions in the exposed group above a holdout, with the method stated and the uncertainty shown.

You receive

Readout

The method, the test and control groups, incremental conversions with a confidence interval, cost per incremental customer next to blended CAC, contribution after media and fees, and what we recommend doing with the budget.

How a test is run

  1. Record a baseline

    We record at least 30 days before launch: orders, branded search, direct traffic and survey answers. Right Side Up recommends the same 30-day baseline for podcast campaigns (Right Side Up, updated 2023-12-09).

  2. Set up every signal before the first placement

    Per channel or show: one offer code and one vanity URL. Plus one post-purchase survey question, and pixel attribution where the publisher supports it. A signal added halfway through a flight only measures the second half.

  3. Design the holdout

    Where your volume allows, we compare an exposed group with a control that did not see the ads: matched regions for a geographic test, or exposed and unexposed households from a pixel vendor that offers lift testing. The design is written down before launch.

  4. Wait for the read window

    Podcast results are taken no sooner than day 30 after the final episode; in Magellan AI’s data a day-7 reading showed under half of the final response (Magellan AI, 2026-09). YouTube results are taken at day 90.

  5. Report the difference

    The readout shows incremental conversions, cost per incremental customer next to your blended CAC, and the confidence interval. If the result is flat, the report says so and the budget moves.

Holdout readout

Podcast flight, matched-region test

Design
8 exposed regions, 8 held out
Read
Day 30 after the last episode
Line chart. Orders in exposed and held-out regions track together before launch on day 14, then separate. By day 60 the exposed regions are about 18 orders per 10,000 households ahead, with an interval of roughly 11 to 24.04080120Day 0Day 14Day 28Day 42Day 60Episodes airExposedHoldout
Incremental orders per 10,000 households
18.0 11.2 to 24.3
Codes and URLs captured
27% of incremental orders
Survey answers naming podcasts
41% of the lift
Fig. 1 Cumulative orders per 10,000 households in exposed regions against the holdout, with the 90% interval on the difference. Specimen record. Values are invented; no client data.

Why one method is never enough

Right Side Up puts the share of podcast-driven conversions caught by codes and vanity URLs at roughly 20% to 30%, with codes leaking onto coupon sites (Right Side Up, updated 2024-02-15). Fairing gives post-purchase survey response rates of 40% to 50%, and buyers often recall the wrong source (Fairing, 2024-08-15). Household matching at Claritas works for at-home listening and little else (Claritas and Market Enginuity, 2020). No single method is complete. All of them run together, and the holdout decides between them.

Terms used on this page

Incrementality
Conversions caused by the ads, measured as the gap between an exposed group and a control group.
Holdout
A group withheld from the ads on purpose, to serve as the comparison for the exposed group.
Read window
The interval allowed after the final placement before a test result is taken.

All definitions

Briefings on this subject

  1. Measurement

    How to run an incrementality test for podcast ads

    Designing and reading an incrementality test for podcast ads: baseline, holdout design, sample size, read window, and the place of codes, surveys and pixels.     Reading time: 4 min
  2. Finance

    Reporting creator and podcast media against the P&L

    Reporting YouTube and podcast spend in the terms finance uses: incremental customers, cost per incremental customer, contribution, payback and flat results.     Reading time: 4 min

Start with one flight

Send us your category, budget range and what your finance team needs to see. We reply with how we would vet, buy and measure a first flight.

Submit a brief