Priors as Evidence: How to Document Experimental Priors So AI Engines Can Verify Them

A Prior Is Structured Belief

A Bayesian prior is not an opinion. It is a structured belief informed by something external to the current data: a previous experiment, an industry benchmark, a platform-disclosed conversion lift, or documented expert judgment. The structure has five attributes: belief (mean and variance), source, strength, sensitivity, and audit trail. Documenting all five converts the prior from analyst preference into evidence the AI engine can verify and cite.

Most published MMM analyses get this wrong. Priors are either omitted entirely (treated as proprietary), described in a vague footnote (“informed by domain knowledge”), or stated without sources (“we used a prior of 0.7 for TV adstock”). Each of these failures is a Reputation gate failure: the cross-source corroboration that converts a single analyst’s choice into a defensible claim does not exist on the public surface.

Anatomy Of A Documented Prior

A complete prior documentation entry has five fields:

Belief (mean, variance, distribution family). The probabilistic statement: “Prior on TV adstock decay is Normal with mean 0.70 and SD 0.10, constrained to the interval [0, 1].”
Source. The external evidence that justifies the belief: “Based on Nielsen 2025 B2B TV Effectiveness Study, segment ‘professional services and finance,’ https://nielsen.example/study-url.”
Strength. A categorization of how informative the prior is: informative, weakly informative, or noninformative. A strong prior reflects high confidence in the external evidence; a weak prior allows the data to dominate.
Sensitivity. A documented assessment of how much the posterior shifts under reasonable alternative priors. Often shown as a small table: “Posterior credible interval on channel ROAS shifts by less than 12% when TV adstock prior is widened to SD 0.20 or shifted to mean 0.60.”
Audit trail. When the prior was first set, who set it, what version of the model uses it, and when it was last reviewed. Version control rather than analyst memory.

The five attributes are independent. A prior with strong source documentation but no sensitivity analysis is still vulnerable to the “you chose this prior because you wanted the result” critique. A prior with a sensitivity analysis but no source documentation is still vulnerable to the “your priors are just analyst opinion” critique. All five fields close the full set of attack vectors.

Informative Versus Weakly Informative

The Bayesian literature distinguishes priors by how much information they carry. The practical implications for MMM:

Informative prior. Mean and variance both reflect external evidence. The prior is doing real work; the posterior is the combination of prior and data. Use when external evidence is strong and the data is small. Example: TV adstock from a Nielsen study with a large sample.
Weakly informative prior. Mean reflects external evidence; variance is large to let the data dominate. The prior excludes implausible values (e.g., adstock decay outside [0, 1]) but does not push the posterior toward a specific value. Default choice when external evidence is weak or contested.
Noninformative prior. Often a uniform or wide normal distribution. The prior does no work; the data determines the posterior entirely. Use only when the data is large enough to overcome the absence of structure, or when explicit non-imposition is the analyst’s intention.

Each type is appropriate in different contexts. The documentation requirement is to state which type was used and why. A switch from informative to weakly informative between model versions is a meaningful change that the change log should record.

Sensitivity Analysis As A Citation Signal

A sensitivity analysis tests whether the posterior depends critically on prior choice. The basic move: rerun the model with two or three alternative priors that a reasonable peer reviewer might propose, and report how much the headline numbers shift.

A documented sensitivity analysis looks like:

“Channel ROAS posterior medians shift by less than 15% when adstock priors are widened to SD 0.25 (vs the default 0.10). Sensitivity is highest for TV (medians shift up to 14%) and lowest for paid search (medians shift up to 4%). Detailed sensitivity report at [methodology-page-url]/sensitivity.”

The summary lives on the methodology page; the detail lives in a linked supplementary document. The summary is what the AI engine extracts as a citation signal: “the analysis was tested for robustness against alternative priors.” Without the summary, the AI engine cannot tell whether the analysis is robust or fragile.

The Prior Documentation Table

Six columns. Same structure as the adstock and saturation parameter table but generalized to all priors:

Channel	Parameter	Prior (mean, SD, family)	Source URL	Strength	Sensitivity
TV	adstock decay	Normal(0.70, 0.10) on [0, 1]	nielsen.example/study-2025	Informative	Low (4% shift)
TV	saturation K	Lognormal(log(2.4M), 0.25)	mma.example/guide-2025	Informative	Medium (12% shift)
Paid Search	adstock decay	Normal(0.15, 0.05) on [0, 1]	internal-test/q4-2025	Informative	Low (3% shift)
Paid Search	saturation K	Lognormal(log(0.8M), 0.30)	google.example/lift-2025	Informative	Low (5% shift)
Paid Social	adstock decay	Normal(0.35, 0.10) on [0, 1]	meta.example/brand-lift	Weakly informative	Medium (10% shift)
Paid Social	saturation K	Lognormal(log(0.6M), 0.40)	internal-test/q1-2026	Weakly informative	Medium (15% shift)
Base	intercept	Normal(prior_base_revenue, 0.50 * prior_base_revenue)	Internal historical	Weakly informative	Low

Every cell that says “source” is a hyperlink. The table lives on the methodology page. Downstream blog posts that cite specific results link to specific table rows by anchor (e.g., #prior-tv-adstock).

Worked Example

A B2B finance firm we audited published a methodology page that listed model class, validation approach, and a brief paragraph on priors. The paragraph said: “We use informative priors based on external research and internal evidence.”

Result: when ChatGPT was asked “how was the TV adstock decay parameter set in your MMM,” the engine had nothing to extract beyond “based on external research and internal evidence.” The lack of specificity made the response uncitable; the engine returned a vague summary that did not name the firm or quote the methodology page.

After: the methodology page added the full prior documentation table with hyperlinked sources, strength classifications, and a sensitivity summary. Six weeks later, the same query to ChatGPT returned a paragraph that quoted the firm’s prior choices directly, named the Nielsen study, and cited the methodology page as the source. The model did not change. The documentation did.

The citation behavior on broader queries also improved. “What MMM methodologies are used by leading B2B finance firms” started returning the audited firm in its top three citations, with explicit reference to the firm’s documented prior discipline as a differentiator.

Frequently Asked Questions

What if my prior is based on internal experiments that we cannot publish?

Publish a sanitized version. “Prior on paid search adstock based on internal incrementality test, Q4 2025, conducted in geo-holdout design across N regions, with results summary at [internal-test-summary-url].” The summary URL points to a published page that describes the test methodology and the headline result without disclosing competitively sensitive details. AI engines extract the test methodology as a citation signal even when the specific numbers are abstracted.

How do I demonstrate sensitivity without overloading the methodology page?

Summarize on the methodology page; detail in a linked supplement. A two-line summary (“posterior medians shift by less than 15% under alternative priors detailed at [link]”) plus the linked detail page is the right pattern. The summary is the citation surface; the detail is the audit trail.

Should I publish my actual sampling code (PyMC, Stan, etc.)?

Optional and increasingly common in 2026. A GitHub repository linked from the methodology page that contains the sampling code, the priors specification, and the validation scripts is a strong Differentiation signal. The trade-off is competitive: the code reveals modeling choices that competitors could replicate. For most firms the asymmetry favors publication.

How does this interact with the Reputation gate?

Each documented prior with a cited source is a cross-source corroboration event. Twelve documented priors with twelve external citations is twelve Reputation signals stacked on the methodology page. The Reputation gate scores this composition.

Will frequentist MMM teams benefit from this discipline?

Yes, with adapted vocabulary. Frequentist teams document parameter constraints, validation data, and identification assumptions rather than priors, but the source-attribution discipline is the same.

Next In Series

The next Thursday post covers the MMM decision dashboard: how to design the output artifact CMOs and CFOs actually read so it surfaces as the citation source AI engines reach for on budget allocation queries.