Propensity Score Matching

Why the Simple Comparison Lies

Loyalty programs don't recruit customers randomly. Customers who choose to enroll tend to already be the company's best, most engaged buyers — higher prior spend, more frequent visits, longer tenure. They were going to spend more regardless of the program. When you compare enrolled vs. non-enrolled customers, you are not comparing the effect of the program. You are comparing two different kinds of people.

This is the selection bias problem. The treatment (enrollment) was not randomly assigned — customers self-selected in. And the very characteristics that made them enroll are the same characteristics that drive higher spending. So the naive gap of $630 is a mixture of two things that are impossible to disentangle without a more careful approach:

A randomized experiment would solve this cleanly by assigning enrollment randomly. But in most real marketing settings — CRM programs, ad exposure, promotional offers, app features — you simply cannot run a randomized experiment after the fact. You have observational data about what happened in the world, and you need to extract a credible causal answer from it.

How Propensity Score Matching Works

The challenge with "find a similar customer" is that people have many characteristics simultaneously. Matching on ten variables at once quickly becomes impossible — there may be no pair that agrees on everything. The propensity score collapses all that complexity into a single, matchable number.

Step 1 — Estimate the propensity score: $$ e(X_i) = \Pr(T_i = 1 \mid X_i) = \text{logit}^{-1}(\beta_0 + \beta_1 X_{1i} + \cdots + \beta_p X_{pi}) $$ Fit a logistic regression that predicts who enrolled from their observable characteristics \(X\). Each customer's fitted probability — their propensity score \(\hat{e}(X_i)\) — is a summary of "how likely someone with this profile was to enroll." Two customers with the same score are statistically equivalent on all the characteristics the model saw, even if their individual values differ.

Step 2 — Match on the score: For each customer who enrolled (treated), find the non-enrolled customer (control) with the closest propensity score. This creates matched pairs where both people looked equally likely to enroll — making treatment assignment as good as random within each pair.

Step 3 — Estimate the treatment effect: $$ \text{ATT} = \frac{1}{n_1} \sum_{i:\, T_i=1} \bigl[ Y_i - Y_{j(i)} \bigr] $$ The Average Treatment Effect on the Treated (ATT) is the mean outcome difference within matched pairs. Because the pairs are comparable, this gap is the causal program effect — not the selection artifact. If the loyalty program really does change behavior, it shows up here. If the naive $630 gap shrinks dramatically after matching, that is evidence that most of it was pre-existing customer quality, not the program working.

Where PSM is most valuable in marketing practice

PSM is the right tool whenever treatment was not randomly assigned and you need a causal answer rather than a correlation. Common marketing applications include:

• CRM & loyalty programs — did enrollment cause higher spend, or did high-spenders self-select in?
• Email & push campaigns — did the campaign cause conversions, or were recipients already highly engaged?
• Ad exposure — did the ad lift brand awareness, or were exposed users already category enthusiasts?
• App feature adoption — does using Feature X cause retention, or do retained users just happen to discover it?
• Sales interventions — did the rep's call cause the deal to close, or were those accounts already warm?

What you need: (1) a binary treatment indicator (enrolled vs. not, received vs. not), (2) an outcome to measure, and (3) pre-treatment covariates that predict who self-selected into treatment. The richer your covariate set, the more selection bias you can remove.

Key limitation: PSM only removes bias from observed characteristics. If customers who enrolled also had unmeasured motivation (e.g., stronger brand affinity you never surveyed), that hidden factor still confounds your estimate. PSM makes the comparison as clean as your data allows — but it cannot manufacture a randomized experiment from observational records. This is why domain knowledge about why people self-select matters as much as the method itself.

ATT vs. ATE: which effect are you actually estimating?

PSM with 1:1 matching estimates the Average Treatment Effect on the Treated (ATT): among the customers who actually enrolled, what was the average effect compared to their matched non-enrollees? This is the most actionable causal question for most marketing decisions, because it reflects the program's impact on the people it actually reached.

The Average Treatment Effect (ATE) asks a different question: what would the effect be on a randomly selected person from the full population, whether or not they would normally enroll? ATE is relevant when you are considering rolling the program out to everyone, including people who would never have enrolled voluntarily. PSM typically estimates ATT, not ATE — keep this distinction in mind when generalizing results.