애초에 인과관계 개념 및 H2O 에 Uplift 모델링이 지원됨에 따라, 공부 겸 정리 좀 해보자는 글.
Predictive Models vs. Prescriptive Uplift Models
There are many applications of predictive modeling where the outcome is predicted as advice only to a human decision maker, and no action is directly taken automatically from the model result.
예측 모델의 예측 결과가 사람의 의사결정에 도움(조언)이 되긴 하지만 구체적으로 어떤 행동을 하라고는 하진 않는다.
An example is workload prioritization. For example, in the telecom industry we can predict which customers are most likely to churn (cancel their contracts). In healthcare we can predict which patients are most likely to recover. For universities or charitable organizations, we can predict which prospective benefactors are most likely to donate.
Sometimes this is sufficient. For example, if the outcome of our weather prediction is that it is likely to rain, we take an umbrella. Since we can’t change the weather we can only be better prepared for it.
예측 모델의 예측 결과로 사람의 의사결정에 도움이 되고 충분한 도움이 되기도 한다.
날씨 예측의 경우, 비가 올 확률이 높다면 사람은 우산을 가져 나갈것이다.
왜냐하면 예측된 결과를 바꿀 수 없기 때문이다.
But where we can, we aim to influence the outcome one way or another. Will a live agent offering the phone customer a contract upgrade decrease their likelihood to churn? Will soliciting a fund raising prospect with a flyer in the mail improve their chances of making a donation? Will offering a moving bonus increase the likelihood that a desirable candidate will accept our employment offer? The most common example where uplift modeling has taken hold is in retail marketing where the goal is to predict not the likelihood of a customer buying, but what can be done to increase the likelihood of them making a purchase.
The salient knowledge sought is the impact of the treatment, not the estimate of the outcome.
잘 알려진 생각(선행연구)은 treatment(실험의 변인)의 임팩트이다.
결과 예측이 아니다.
For instance, would you rather spend campaign dollars trying to persuade your most loyal supporters (those with the highest probability of “buying”), or on the voters who will be swayed the most by an additional engagement? Simply predicting the expected outcome is not sufficient to optimize your use of money and resources. A few elections ago I was determined to vote for a particular candidate, who meanwhile, kept filling my mailbox with campaign material. Even though my publicly available data should have demonstrated that I was already a sure vote they could count on, they wasted many glossy flyers on me.
Uplift is Not Directly Measurable
Uplift modeling is also known as incremental modeling, treatment effects modeling, true lift modeling, or net modeling.
Uplift is the increase in likelihood of the outcome with the treatment as compared to the outcome without the treatment.
We can’t observe this difference, or causal effect, directly, but must infer it from an experiment.
Estimating Uplift
Consider a telecom example of trying to prevent customer churn as shown in figure 3.
Figure 3. Telecom Uplift model example.
The treatment is to offer an upgrade to a customer who is a potential churner.
이탈 방지를 위해 예비 이탈자들에게 일종의 혜택인 무료 업그레이드를 제시한다고 하자.
To perform uplift analysis, we conduct an experiment with 400 randomly selected test accounts to whom we offer a free upgrade, and a control group of 1600 accounts that receive no offer. (It is common to have a larger control group as it is less expensive).
In this experiment, we record 8 churns in the group that received an offer, and 160 churns in the group that did not receive an offer.
400명은 혜택을 제안받았고
1600명은 냅뒀다.
400명중에 8명은 이탈했고
1600명중에 160명이 이탈했다.
그러면 400명 그룹의 이탈율은 2%이고 (RT - R ? Treated )
1600명 그룹의 이탈율은 10%다. (RC - R ? Controlled )
이 제안의 Uplift, U = RT - RC = 2 - 10 = - 8
이다.
( Uplift가 음수가 나오는 이유는 타겟 행동이 지양하는 바라서 그렇다. 이 절대값이 음수의 절대값이 클수록, 혹은 값 그자체가 작으면 작을수록 좋다. )
This means that there is a 2% churn in the experimental group (RT) and a 10% churn in the control group (RC). The offer has a -8% uplift (U):
Overall Uplift U = RT – RC = 2% – 10% = -8%
The uplift in this case is negative because we are trying to avoid the target behavior rather than promote it.
For Uplift to be actionable in practice, we also need to know the treatment effect for each individual person uniquely, in addition to the general population.
업리프트를 실제 적용하기에 앞서, 이러한 변인들이 개개인에게 개별적으로 다른 효과가 나타날 수 있다는 점을 알아야 한다.
For example, my previous volume of online shopping may indicate that I am more persuadable to click on a particular advertisement than others in my same demographic group. Thus, we want to model how the attributes of a case impact the treatment uplift of that case.
The way such a model is created in practice is as follows:
1) predict the outcome with the treatment applied (RTi in the telecom example),
2) predict the outcome without the treatment applied (RCi in the telecom example),
3) calculate the difference in the rates as the uplift (Ui=RTi-RCi), and
4) compute the upper and lower 95% confidence limits on Ui.
Once these values are calculated, individuals can be allocated to the four quadrants of the treatment effect matrix using these rules:
- If the confidence limits of the Incremental Uplift (Ui) includes zero, the treatment effect can be thought of as unknown and not significant. Regardless of treatment, the Sure Things have a high outcome likelihood and the Lost Causes have a low outcome likelihood.
신뢰구간에 0이 포함된다면 변인의 효과가 없거나 무의미할 수 있다.
아까 Uplift 값이 (타겟 값이 우리가 지향하는 것이라면 ) 아주 크다면 혹은 (타겟 값이 우리가 지양하는 것이라면 ) 아주 작다면 ( 절대값은 크다면 ) 변인의 효과가 큰 것이었다.
반연 Uplift 값이 0에 가깝다면 변인의 효과가 무의미한 것일 것이다.
- If the Incremental Uplift (Ui) is significantly greater than zero, the predicted outcome increases because of the treatment. These are the Persuadables if the outcome is positive.
업리프트가 0보다 확연히 크다면 변인이 효과가 있다고 볼 수 있을 것이다.
- When the Incremental Uplift (Ui) is significantly less than zero, the predicted outcome is less likely because of the treatment. Traditionally, these are called the Do-Not-Disturbs.
업리프트가 0보다 확연히 작다면 변인이 효과가 있다고 볼 수 있을 것이다.
Remember, of course, that this is a modeled estimate of a, b, c, and d, and not every persuadable individual will actually be persuaded by the treatment.
기역해야 하는건 개개인에 적용시 예측 결과와 실제 결과는 달라질 수 있다는 것이다.
