How we audit your MMM
Vendor-agnostic review of open-source and proprietary models. We focus on the decisions the model drives, not on selling a replacement stack.
Method
Stacks we regularly review
Google Meridian
Bayesian hierarchical MMM. We examine prior choices, geo / national structure, media transforms, and whether outputs are being used within the model's identification limits.
Meta Robyn
Nevergrad-optimised MMM. We check hyperparameter search design, solution clustering, spend constraints, and whether “best” models are stable under reasonable perturbations.
PyMC-Marketing
Bayesian MMM in the PyMC ecosystem. We review model structure, prior elicitation, MCMC diagnostics, and how channel effects are communicated to stakeholders.
Proprietary / vendor models
We work with the commissioning company, not the vendor. Where code is unavailable, we audit documentation, data dictionaries, contribution reports, and experimental calibration evidence.
The checklist
- Specification: business question, identification, controls, hierarchy
- Assumptions: priors, constraints, omitted variables, structural choices
- Data: coverage, leakage, seasonality, price/promo treatment
- Adstock & saturation: transforms and parameter plausibility
- Validation: holdout, sensitivity, stability across seeds/runs
- Calibration: agreement with geo-lift / incrementality (optional)
For a practitioner-oriented outline, see MMM validation checklist and common misspecifications.
Engagement
Four steps, three weeks
Standard turnaround three weeks; two-week rush available.
1
Intro + scope
Free 20-min fit call, then optional £750 fixed-fee scoping credited to the audit.
2
You share
Model code or spec, data, and experiment results under NDA.
3
We audit
Independent review against the checklist; reproduction if scoped.
4
Report + walkthrough
Written verdict, ranked issues, and a findings call.
Discuss your model stack
Tell us what you run and what decisions hang on it. We'll confirm fit and scope on a short call.