Week 11 · Marketing Research & Analytics
Davood Wadi, PhD
Customer analytics estimates the value and predicts the behavior of individual customers, to inform decisions on acquisition, retention, and personalization.
Customer analytics is the analysis of customer-level data, such as transactions, interactions, and service records, to guide marketing decisions about individual customers.
It treats the customer, rather than the product or the campaign, as the unit of analysis.
Its principal questions are which customers are most valuable, which are likely to leave, which are likely to respond to an offer, and which products or messages each customer should receive.
It is based mainly on the predictive models introduced in Week 9, evaluated on their accuracy for new customers.
Customers differ substantially in the profit they generate over the course of their relationship with the firm.
Customer lifetime value and RFM analysis are two methods of estimating and ranking that value.
They inform how much the firm can spend to acquire a customer and which customers to prioritize for retention.
Customer lifetime value, CLV, is the present value of the future profits expected from a customer over the duration of the relationship.
In its simplest form, with a constant annual margin m, an annual retention rate r, and a discount rate d, CLV equals m × r / (1 + d − r).
For example, a customer who generates an annual margin of 100 dollars, with a retention rate of 80 percent and a discount rate of 10 percent, has a CLV of 100 × 0.8 / 0.3, or approximately 267 dollars.
CLV sets an upper limit on the amount the firm can profitably spend to acquire a customer.
In contractual settings, such as subscriptions, the end of a relationship is observed. In noncontractual settings, such as retail purchases, it is not, and the probability that a customer is still active must be estimated from the pattern of purchases.
CLV increases with the margin per customer, with the retention rate, and with a lower discount rate.
CLV is particularly sensitive to the retention rate. In the previous example, an increase in retention from 80 to 90 percent raises CLV from approximately 267 to 450 dollars.
The firm can therefore increase customer value by improving retention as well as by increasing margins.
The simple formula assumes that margins and retention rates are constant over time. Models used in practice allow both to vary across customers and over the course of the relationship.
A streaming service earns an annual margin of $60 per subscriber, retains 70% of subscribers each year, and uses a discount rate of 10%, so its CLV is 60 × 0.7 / 0.4, or $105. Which single change raises CLV the most?
RFM analysis ranks customers on three measures of past purchasing behavior.
Recency is the time since the customer’s most recent purchase. Frequency is the number of purchases within a given period. Monetary value is the amount the customer has spent within that period.
Customers are typically scored from 1 to 5 on each measure, according to the quintile in which they fall, with 5 representing the most recent, most frequent, or highest spending customers.
Customers with high scores on all three measures are the most likely to respond to future offers. Recency is typically the strongest single predictor of response.
RFM analysis is simple to compute from transaction data, but it describes past behavior only and does not consider the margin generated by each customer.
Customers with high recency, frequency, and monetary scores are the firm’s most valuable active customers.
Customers with a low recency score but high frequency and monetary scores were once valuable but have not purchased recently, and are at risk of being lost.
Customers with a high recency score but a low frequency score are recent new customers, whose future value is not yet established.
Customers with low scores on all three measures have lapsed, and the cost of attempting to reactivate them may exceed the expected return.
Each profile calls for a different action: rewarding the most valuable customers, contacting those at risk, developing new customers, and limiting spending on lapsed customers.
An online bookstore scores its customers on recency, frequency and monetary value from 1 to 5. Which profile does each customer have?
High recency, frequency and monetary scores: a most valuable active customer, to be rewarded.
High frequency and spending but low recency: once valuable, now at risk of being lost, and worth contacting.
High recency but low frequency: a recent new customer whose future value is not yet established, to be developed.
Low on all three measures: lapsed, and reactivation may cost more than it returns.
Recent, but only one purchase: frequency is low, so the customer's future value is not yet established.
High frequency and monetary value with low recency: at risk of being lost, and worth contacting.
Which concept does this case show?
Predictive models estimate, for each customer, the probability of a future behavior.
The most common applications are the prediction of churn and the prediction of response to an offer.
Logistic regression, introduced in Week 9, is the standard baseline model. Decision trees and ensemble models frequently predict more accurately.
Churn is the termination of a customer’s relationship with the firm, such as the cancellation of a subscription or the closure of an account.
A churn model estimates each customer’s probability of churning within a specified period, on the basis of variables such as usage, recent changes in usage, complaints, and contract type.
A response model estimates each customer’s probability of responding to an offer, such as a promotion or an upgrade.
The predicted probabilities are used to rank customers, so that retention efforts and offers are directed to those with the highest scores.
A decision tree divides the customers repeatedly into subgroups, at each step using the variable and the split that best separate customers who churn from those who do not.
The result can be expressed as a set of rules, such as: customers on a monthly contract who have not used the service in 30 days have a churn probability of 45 percent.
Decision trees are easy to interpret and to communicate to managers.
A large tree reproduces the random variation in the data used to estimate it, the overfitting introduced in Week 9. Trees are therefore limited in size, or pruned, and evaluated on holdout data.
An ensemble model combines the predictions of many models to produce a single, more accurate prediction.
A random forest estimates many decision trees, each on a random sample of customers and variables, and averages their predictions.
Gradient boosting estimates trees in sequence, with each tree designed to correct the errors of the trees before it.
Ensemble models are among the most accurate methods for predicting customer behavior from structured data.
Their predictions cannot be expressed as a simple set of rules, which makes the reasons for an individual prediction more difficult to establish.
A predictive model is evaluated by the accuracy of its predictions for customers whose outcomes were not used to estimate it.
The appropriate measure of accuracy depends on how the predictions will be used.
A model that predicts accurately may still fail to improve decisions, for reasons examined at the end of this part.
The data are divided into a training sample, used to estimate the model, and a holdout sample, used to evaluate it.
Cross-validation repeats this division several times and averages the results, which provides a more stable estimate of accuracy.
For behavior that unfolds over time, such as churn, the model is estimated on an earlier period and evaluated on a later period, which reflects the conditions under which it will be used.
Data leakage occurs when the model uses information that would not be available at the time of prediction, such as a record that the customer has requested a cancellation form. Leakage produces an accuracy on holdout data that cannot be achieved in practice.
A confusion matrix compares the predicted outcomes with the actual outcomes for the holdout sample.
True positives are churners correctly predicted to churn. False positives are customers predicted to churn who did not. False negatives are churners whom the model did not identify. True negatives are customers correctly predicted to stay.
Precision is the proportion of customers predicted to churn who actually churned. Recall is the proportion of actual churners whom the model identified.
For example, among 1,000 customers of whom 100 churn, a model identifies 150 customers as likely to churn, and 60 of them do churn. Its precision is 40 percent and its recall is 60 percent.
Accuracy, the proportion of correct predictions, is misleading when one outcome is rare. A model that predicts that no customer will churn is correct for 90 percent of these customers, yet identifies no churner.
The probability threshold above which a customer is classified as a churner is set according to the relative costs of false positives and false negatives.
In a bank's holdout sample of 2,000 customers, 200 close their accounts. A churn model flags 100 customers as likely to close, and 80 of them do. What are the model's precision and recall?
Customers are ranked by their predicted probability and divided into ten groups of equal size, called deciles.
Lift is the rate of the outcome in a decile divided by the rate in the customer base as a whole.
For example, if 35 percent of all churners are found in the top 10 percent of customers ranked by the model, the lift of the top decile is 3.5.
A gains chart displays the cumulative percentage of churners captured as successive deciles are targeted.
Lift informs how many customers to target: contact continues through successive deciles as long as the expected benefit exceeds the cost.
A churn model identifies the customers most likely to leave, not the customers whose decision an intervention would change.
Persuadables are customers who would leave without an intervention and stay with it.
Sure things are customers who would stay regardless, so that an intervention produces a cost without a benefit.
Lost causes are customers who would leave regardless of the intervention.
Sleeping dogs are customers who would stay without contact but are prompted to leave by it, for example by a reminder of their contract terms.
Uplift modeling estimates the effect of the intervention on each customer, using data from a randomized experiment of the kind introduced in Week 8, and directs the intervention to the persuadables.
A gym ran a randomized test of a call offering a free month to members whose contracts end soon. What type is each member?
Stays regardless, so the free month is a cost without a benefit.
Leaves regardless of the intervention.
Would leave without the call and stays with it: the customer uplift modeling directs the intervention to.
Would stay without contact but is prompted to leave by it.
The intervention changed the decision to leave.
No intervention changes the decision to leave.
Which concept does this case show?
Personalization is the tailoring of content, offers, prices, or products to the individual customer.
It depends on predictions of what each customer is likely to value.
Recommendation systems are the most widespread form of personalization in digital channels.
Content-based recommendation suggests items similar to those the customer has previously purchased or rated highly, on the basis of the items’ attributes.
Collaborative filtering suggests items purchased or rated highly by customers whose past behavior is similar to that of the customer, as in “customers who bought this item also bought.”
Hybrid systems combine both approaches, and many current systems use deep learning models trained on sequences of customer behavior.
The cold-start problem arises when a new customer or a new product has too little history for collaborative filtering to produce recommendations.
Popularity bias arises when recommendations concentrate on items that are already popular, which reduces exposure for new or specialized items and narrows the range of products customers encounter.
Recommendation systems are evaluated on holdout data for their predictive accuracy, and through A/B tests for their effect on business outcomes such as sales and retention.
A music streaming service designs its recommendations. Match each term with what happens on the service.
Term
On the service
Tap a term, then the example it goes with.
Predictive models increasingly determine which customers receive offers, prices, credit, and service.
Such models can treat groups of customers systematically differently, including groups protected by anti-discrimination law.
Firms are therefore expected to assess their models for bias, and to explain the decisions the models inform.
Algorithmic bias is a systematic difference in a model’s predictions or errors across groups of customers that is not justified by the purpose of the model.
Historical bias arises when the training data reflect past discrimination, so that the model reproduces it.
Representation bias arises when some groups are underrepresented in the training data, so that predictions for them are less accurate.
Proxy variables can reproduce the effect of a protected characteristic that is excluded from the model, as when a postal code is correlated with ethnicity.
Feedback loops arise when the model’s decisions determine the data on which it is later retrained, as when customers who never receive an offer never generate evidence that they would have responded to it.
Demographic parity requires that the rate of a favorable outcome, such as receiving an offer, is equal across groups.
Equal opportunity requires that, among customers who would qualify for the favorable outcome, the rate at which the model correctly identifies them is equal across groups.
Calibration requires that customers who receive the same predicted probability have the same actual rate of the outcome, regardless of group.
When the underlying rates of the outcome differ across groups, these criteria cannot all be satisfied at once.
The choice of fairness criterion is therefore a value judgment and a legal question, as well as a technical one, and is made explicitly and documented.
Global explanations describe how a model uses its variables across all customers, such as measures of variable importance.
Local explanations describe why the model produced a particular prediction for a particular customer. SHAP values, for example, estimate the contribution of each variable to an individual prediction.
Counterfactual explanations state the smallest change in a customer’s characteristics that would have changed the prediction, such as a higher account balance that would have qualified the customer for an offer.
Explanations describe the behavior of the model, not the causal effect of the variables on the customer’s behavior, as introduced in Week 9.
Regulation in several jurisdictions grants individuals rights concerning automated decisions that significantly affect them, including information about the logic involved.
The analyst first defines the groups to be compared and the fairness criterion appropriate to the decision.
The analyst then compares the model’s predictions and error rates across the groups on holdout data.
The analyst then examines the model’s variables for proxies of protected characteristics.
The analyst then uses local and global explanations to identify the variables responsible for differences across groups.
The analyst then revises the model, the data, or the decision rule, and repeats the comparison.
Finally, the analyst documents the audit and monitors the model’s outcomes across groups after deployment.
An insurer audits its loyalty-discount model. Put the audit steps in order.
Tap a step, then the number where it belongs.
Language models and AI agents inform decisions about which products consumers see and buy, and they are audited for bias in the same way as predictive models.
Benchmarks measure general capabilities and do not reveal systematic biases in specific marketing tasks.
Position bias is the tendency to select an alternative because of its place in a list (Wadi & Ma, 2026a).
Sponsorship bias arises when the party an agent is told it serves changes the penalty it applies to paid listings (Wadi & Ma, 2026c).
Country-of-origin bias arises when a model’s evaluations shift for particular pairings of the model’s country of development and the country being evaluated. A fully crossed factorial design separates this interaction from each model’s baseline tendency (Wadi, Ghodrat, & Philp, 2026).
Bias varies across models, so that an audit applies to the specific model and version that is deployed.
The analyst first defines the task, the population of queries, and the normative standard against which the agent’s behavior is judged.
The analyst then designs controlled variations of the prompts and the alternatives, with positions and orders randomized.
The analyst then collects a sufficient number of responses for each condition from the model version to be deployed.
The analyst then compares the agent’s choices with the normative standard and across conditions, with effect sizes and corrections for multiple comparisons.
The analyst then revises the system prompt, the information provided to the agent, or the choice of model, and repeats the comparison.
Finally, the analyst documents the audit and repeats it whenever the provider updates the model.
A travel site audits the AI agent that suggests hotels to its users. Put the steps of the audit in order.
Tap a step, then the number where it belongs.
Discussion: A retailer’s model directs its largest discounts to the customers predicted to be most price-sensitive. An audit reveals that customers in lower-income postal codes receive smaller discounts on average than customers in higher-income postal codes. Which sources of bias could explain this result, which fairness criterion is relevant, and what should the retailer do?
Customer lifetime value estimates the present value of future profits from a customer, and is particularly sensitive to the retention rate.
RFM analysis ranks customers by recency, frequency, and monetary value, and different RFM profiles call for different actions.
Churn and response models, including decision trees and ensemble models, are evaluated on holdout data with confusion matrices, precision, recall, and lift.
Predicting which customers will churn is not the same as identifying those whom an intervention would retain; uplift modeling addresses the second question.
Recommendation systems personalize offers through content-based and collaborative methods, and are evaluated through A/B tests of business outcomes.
Predictive models can be biased against groups of customers, and firms assess them against explicit fairness criteria, explain their decisions, and monitor them after deployment.
Language models and AI agents are audited for position, sponsorship, and country-of-origin biases with controlled designs, and each audit applies to the specific model version deployed.