A holdout test on the largest brand in a PE-backed home services portfolio proved that 1.8x of matchback revenue from prospecting was genuinely campaign-driven.
Matchback attribution is the standard way to measure direct mail ROI — but it has a credibility problem. Critics argue it overstates impact because some conversions would have happened anyway. For a PE-backed platform evaluating whether to scale prospecting spend, the question wasn't whether matchback showed revenue. It was whether that revenue was real.
Matchback captures conversions across channels, but without a control group there's no way to separate campaign-driven revenue from organic activity.
For a PE-backed platform evaluating whether to scale prospecting spend, the question wasn't whether matchback showed revenue. It was whether that revenue was real.
Arch designed a holdout test on the brand's prospecting campaign — withholding a randomized control group from all campaign exposure to isolate the true incremental impact of direct mail.
Arch launched a direct mail prospecting campaign targeting new prospects in an expansion market where the brand had relatively low awareness.
A randomized subset of the target audience was withheld from all campaign exposure — no mailers, no emails, no touchpoints. This group served as the control.
After the campaign window closed, revenue from both groups was compared to isolate the true incremental impact of direct mail.
The holdout comparison isolated the incremental revenue driven by the campaign versus organic activity.
The holdout group was adjusted for audience size to enable a fair comparison. The ~$35,000 in incremental revenue represents the difference between the campaign-exposed group and the size-adjusted holdout baseline.
The holdout group generated just ~$8,000 (adjusted) in organic revenue, compared to ~$43,000 from the campaign-exposed group. Campaign-exposed prospects generated 1.8x the matchback revenue, proving the campaign genuinely drove results rather than capturing organic activity.
Direct mail attribution is fuzzy. A holdout test replaces guesswork with measurement, giving operators the confidence to scale.
With 1.8x matchback revenue validated as incremental, the platform now has the evidence needed to expand prospecting investment across the portfolio. A holdout-tested attribution framework that proves real ROI.
See how Arch measures incremental revenue — not just matchback — so you can scale with confidence.