Kaggle: Home Credit Default Risk 세 번째 모델 학습 및 성능 평가

prev_application 데이터 세트 기반의 EDA와 Feature Engineering 수행 후 학습 모델 생성/평가

라이브러리 및 데이터 세트 로딩. 이전 application 데이터의 FE 함수 복사

import numpy as np # 1.18.5
import pandas as pd # 0.25.1
import gc
import time
import matplotlib.pyplot as plt # 3.2.2
import seaborn as sns # 0.10.1
%matplotlib inline

import warnings
warnings.filterwarnings('ignore')

pd.set_option('display.max_rows', 100)
pd.set_option('display.max_columns', 200)

app_train = pd.read_csv('application_train.csv')
app_test = pd.read_csv('application_test.csv')

def get_apps_dataset():
    app_train = pd.read_csv('application_train.csv')
    app_test = pd.read_csv('application_test.csv')
    apps = pd.concat([app_train, app_test])
    
    return apps

apps = get_apps_dataset()

<ipython-input-3-da4fa99e2eb3>:4: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version
of pandas will change to not sort by default.

To accept the future behavior, pass 'sort=False'.

To retain the current behavior and silence the warning, pass 'sort=True'.

  apps = pd.concat([app_train, app_test])

이전 application 데이터의 feature engineering 함수 복사

def get_apps_processed(apps):
    
    # EXT_SOURCE_X FEATURE 가공
    apps['APPS_EXT_SOURCE_MEAN'] = apps[['EXT_SOURCE_1', 'EXT_SOURCE_2', 'EXT_SOURCE_3']].mean(axis=1)
    apps['APPS_EXT_SOURCE_STD'] = apps[['EXT_SOURCE_1', 'EXT_SOURCE_2', 'EXT_SOURCE_3']].std(axis=1)
    apps['APPS_EXT_SOURCE_STD'] = apps['APPS_EXT_SOURCE_STD'].fillna(apps['APPS_EXT_SOURCE_STD'].mean())
    
    # AMT_CREDIT 비율로 Feature 가공
    apps['APPS_ANNUITY_CREDIT_RATIO'] = apps['AMT_ANNUITY']/apps['AMT_CREDIT']
    apps['APPS_GOODS_CREDIT_RATIO'] = apps['AMT_GOODS_PRICE']/apps['AMT_CREDIT']
    
    # AMT_INCOME_TOTAL 비율로 Feature 가공
    apps['APPS_ANNUITY_INCOME_RATIO'] = apps['AMT_ANNUITY']/apps['AMT_INCOME_TOTAL']
    apps['APPS_CREDIT_INCOME_RATIO'] = apps['AMT_CREDIT']/apps['AMT_INCOME_TOTAL']
    apps['APPS_GOODS_INCOME_RATIO'] = apps['AMT_GOODS_PRICE']/apps['AMT_INCOME_TOTAL']
    apps['APPS_CNT_FAM_INCOME_RATIO'] = apps['AMT_INCOME_TOTAL']/apps['CNT_FAM_MEMBERS']
    
    # DAYS_BIRTH, DAYS_EMPLOYED 비율로 Feature 가공
    apps['APPS_EMPLOYED_BIRTH_RATIO'] = apps['DAYS_EMPLOYED']/apps['DAYS_BIRTH']
    apps['APPS_INCOME_EMPLOYED_RATIO'] = apps['AMT_INCOME_TOTAL']/apps['DAYS_EMPLOYED']
    apps['APPS_INCOME_BIRTH_RATIO'] = apps['AMT_INCOME_TOTAL']/apps['DAYS_BIRTH']
    apps['APPS_CAR_BIRTH_RATIO'] = apps['OWN_CAR_AGE'] / apps['DAYS_BIRTH']
    apps['APPS_CAR_EMPLOYED_RATIO'] = apps['OWN_CAR_AGE'] / apps['DAYS_EMPLOYED']
    
    return apps

previous 데이터 로딩

prev = pd.read_csv('previous_application.csv')
print(prev.shape, apps.shape)

(1670214, 37) (356255, 122)

application와 previous outer 조인하고 누락된 집합들 확인.

prev.head()

prev_app_outer = prev.merge(apps['SK_ID_CURR'], on='SK_ID_CURR', how='outer', indicator=True)

prev_app_outer['_merge'].value_counts()

both          1670214
right_only      17398
left_only           0
Name: _merge, dtype: int64

previous 컬럼 설명

Table	컬럼명	컬럼 대분류	컬럼 중분류	컬럼 설명
previous_application.csv	SK_ID_PREV	대출	고유ID	과거 대출 고유 ID
previous_application.csv	SK_ID_CURR	대출	고유ID	현재 대출 고유 ID
previous_application.csv	NAME_CONTRACT_TYPE	대출	대출 유형	대출 유형
previous_application.csv	AMT_ANNUITY	대출	대출 금액	월 대출 지급액
previous_application.csv	AMT_APPLICATION	대출	대출 금액	대출 신청 금액
previous_application.csv	AMT_CREDIT	대출	대출 금액	대출금액(허가)
previous_application.csv	AMT_DOWN_PAYMENT	대출	대출 금액	대출 시 납부한 선금액
previous_application.csv	AMT_GOODS_PRICE	대출	대출 금액	소비자 대출상품액
previous_application.csv	WEEKDAY_APPR_PROCESS_START	고객	행동	대출 신청 시작 요일
previous_application.csv	HOUR_APPR_PROCESS_START	고객	행동	대출 신청 시작 시간대
previous_application.csv	FLAG_LAST_APPL_PER_CONTRACT	고객	행동	이전 계약의 마지막 대출 신청 여부
previous_application.csv	NFLAG_LAST_APPL_IN_DAY	고객	행동	하루중 마지막 대출 신청 여부(하루에 여러 번 대출 신청했을 경우)
previous_application.csv	NFLAG_MICRO_CASH	대출	대출 유형	소액 대출 여부
previous_application.csv	RATE_DOWN_PAYMENT	대출	대출 금액	선금 비율(정규화됨)
previous_application.csv	RATE_INTEREST_PRIMARY	대출	대출 금액	이자율
previous_application.csv	RATE_INTEREST_PRIVILEGED	대출	대출 금액	이자율
previous_application.csv	NAME_CASH_LOAN_PURPOSE	대출	대출 유형	현금 대출 목적
previous_application.csv	NAME_CONTRACT_STATUS	대출	대출 상태	대출 상태(허가, 취소)
previous_application.csv	DAYS_DECISION	대출	대출 상태	과거 신청 대비 현재 신청 결정 기간
previous_application.csv	NAME_PAYMENT_TYPE	대출	대출 유형	과거 대출 신청의 납부 방법
previous_application.csv	CODE_REJECT_REASON	대출	대출 상태	과거 신청 거절 사유
previous_application.csv	NAME_TYPE_SUITE	고객	행동(추천)	동행 고객
previous_application.csv	NAME_CLIENT_TYPE	고객	행동	신규 고객 또는 기존 대출 고객 여부
previous_application.csv	NAME_GOODS_CATEGORY	대출	대출 유형	대출 상품 중분류 유형
previous_application.csv	NAME_PORTFOLIO	대출	대출 유형	현금대출/POS/CAR 대출 유형
previous_application.csv	NAME_PRODUCT_TYPE	채널	판매 유형	고객이 찾아온 대출인가, 영업 대출인가
previous_application.csv	CHANNEL_TYPE	채널	채널 유형	채널 유형
previous_application.csv	SELLERPLACE_AREA	채널	채널 유형	판매자 판매 지역
previous_application.csv	NAME_SELLER_INDUSTRY	채널	채널 유형	판매자 Industry
previous_application.csv	CNT_PAYMENT	대출	대출 금액	이전 대출 신청의 대출금액 관련 Term
previous_application.csv	NAME_YIELD_GROUP	대출	대출 금액	집단 금리 적용 유형
previous_application.csv	PRODUCT_COMBINATION	대출	대출 유형	이전 대출 결합 상품
previous_application.csv	DAYS_FIRST_DRAWING	대출	상태	신청날짜부터 최초 대출 지급까지의 일자
previous_application.csv	DAYS_FIRST_DUE	대출	상태	신청날짜부터 마감일까지의 일자
previous_application.csv	DAYS_LAST_DUE_1ST_VERSION	대출	상태	신청날짜부터 첫 만기일까지의 일자
previous_application.csv	DAYS_LAST_DUE	대출	상태	신청날짜부터 마지막 만기일까지의 일자
previous_application.csv	DAYS_TERMINATION	대출	상태	현 대출 신청일자 대비 대출 예상 종료 일자
previous_application.csv	NFLAG_INSURED_ON_APPROVAL	대출	상태	대출 신청 시 보험가입 요청여부

previous 컬럼과 Null 값 조사

prev.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1670214 entries, 0 to 1670213
Data columns (total 37 columns):
SK_ID_PREV                     1670214 non-null int64
SK_ID_CURR                     1670214 non-null int64
NAME_CONTRACT_TYPE             1670214 non-null object
AMT_ANNUITY                    1297979 non-null float64
AMT_APPLICATION                1670214 non-null float64
AMT_CREDIT                     1670213 non-null float64
AMT_DOWN_PAYMENT               774370 non-null float64
AMT_GOODS_PRICE                1284699 non-null float64
WEEKDAY_APPR_PROCESS_START     1670214 non-null object
HOUR_APPR_PROCESS_START        1670214 non-null int64
FLAG_LAST_APPL_PER_CONTRACT    1670214 non-null object
NFLAG_LAST_APPL_IN_DAY         1670214 non-null int64
RATE_DOWN_PAYMENT              774370 non-null float64
RATE_INTEREST_PRIMARY          5951 non-null float64
RATE_INTEREST_PRIVILEGED       5951 non-null float64
NAME_CASH_LOAN_PURPOSE         1670214 non-null object
NAME_CONTRACT_STATUS           1670214 non-null object
DAYS_DECISION                  1670214 non-null int64
NAME_PAYMENT_TYPE              1670214 non-null object
CODE_REJECT_REASON             1670214 non-null object
NAME_TYPE_SUITE                849809 non-null object
NAME_CLIENT_TYPE               1670214 non-null object
NAME_GOODS_CATEGORY            1670214 non-null object
NAME_PORTFOLIO                 1670214 non-null object
NAME_PRODUCT_TYPE              1670214 non-null object
CHANNEL_TYPE                   1670214 non-null object
SELLERPLACE_AREA               1670214 non-null int64
NAME_SELLER_INDUSTRY           1670214 non-null object
CNT_PAYMENT                    1297984 non-null float64
NAME_YIELD_GROUP               1670214 non-null object
PRODUCT_COMBINATION            1669868 non-null object
DAYS_FIRST_DRAWING             997149 non-null float64
DAYS_FIRST_DUE                 997149 non-null float64
DAYS_LAST_DUE_1ST_VERSION      997149 non-null float64
DAYS_LAST_DUE                  997149 non-null float64
DAYS_TERMINATION               997149 non-null float64
NFLAG_INSURED_ON_APPROVAL      997149 non-null float64
dtypes: float64(15), int64(6), object(16)
memory usage: 471.5+ MB

주요 컬럼 EDA 수행

SK_ID_CURR당 평균 SK_ID_PREV 건수 구하기

• groupby 로 평균 건수 구함.
• boxplot으로 시각화

prev.groupby('SK_ID_CURR')['SK_ID_CURR'].count().mean()

4.928964135313716

sns.boxplot(prev.groupby('SK_ID_CURR')['SK_ID_CURR'].count())

숫자형 피처들의 Histogram을 TARGET 유형에 따라 비교

• application_train의 TARGET 값을 가져오기 위해 prev와 app_train을 inner join 후 TARGET 유형에 따라 비교
• 숫자형 컬럼명 필터링

app_prev = prev.merge(app_train[['SK_ID_CURR', 'TARGET']], on='SK_ID_CURR', how='left')
app_prev.shape

(1670214, 38)

def show_hist_by_target(df, columns):
    cond_1 = (df['TARGET'] == 1)
    cond_0 = (df['TARGET'] == 0)
    
    for column in columns:
        fig, axs = plt.subplots(nrows=1, ncols=2, figsize=(12, 4), squeeze=False)
        sns.violinplot(x='TARGET', y=column, data=df, ax=axs[0][0] )
        sns.distplot(df[cond_0][column], ax=axs[0][1], label='0', color='blue')
        sns.distplot(df[cond_1][column], ax=axs[0][1], label='1', color='red')   

num_columns = app_prev.dtypes[app_prev.dtypes != 'object'].index.tolist()
num_columns = [ column for column in num_columns if column not in ['SK_ID_PREV', 'SK_ID_CURR', 'TARGET']]
print(num_columns)

['AMT_ANNUITY', 'AMT_APPLICATION', 'AMT_CREDIT', 'AMT_DOWN_PAYMENT', 'AMT_GOODS_PRICE', 'HOUR_APPR_PROCESS_START', 'NFLAG_LAST_APPL_IN_DAY', 'RATE_DOWN_PAYMENT', 'RATE_INTEREST_PRIMARY', 'RATE_INTEREST_PRIVILEGED', 'DAYS_DECISION', 'SELLERPLACE_AREA', 'CNT_PAYMENT', 'DAYS_FIRST_DRAWING', 'DAYS_FIRST_DUE', 'DAYS_LAST_DUE_1ST_VERSION', 'DAYS_LAST_DUE', 'DAYS_TERMINATION', 'NFLAG_INSURED_ON_APPROVAL']

show_hist_by_target(app_prev, num_columns)

/Users/andylee/opt/anaconda3/lib/python3.8/site-packages/seaborn/distributions.py:369: UserWarning: Default bandwidth for data is 0; skipping density estimation.
  warnings.warn(msg, UserWarning)
/Users/andylee/opt/anaconda3/lib/python3.8/site-packages/seaborn/distributions.py:369: UserWarning: Default bandwidth for data is 0; skipping density estimation.
  warnings.warn(msg, UserWarning)
/Users/andylee/opt/anaconda3/lib/python3.8/site-packages/seaborn/distributions.py:369: UserWarning: Default bandwidth for data is 0; skipping density estimation.
  warnings.warn(msg, UserWarning)
/Users/andylee/opt/anaconda3/lib/python3.8/site-packages/seaborn/distributions.py:369: UserWarning: Default bandwidth for data is 0; skipping density estimation.
  warnings.warn(msg, UserWarning)

• AMT_ANNUITY, AMT_CREDIT, AMT_APPLICATION, AMT_GOODS_CREDIT는 TARGET=1일 경우에 소액 비율이 약간 높음(큰 차이는 아님)
• RATE_DOWN_PAYMENT는 큰 차이 없음.
• RATE_INTEREST_PRIMARY, RATE_INTEREST_PRIVILEGED 는 NULL값이 매우 많아서 판단 어려움
• DAYS_DECISION은 TARGET=1일 때 0에 가까운(최근일)값이 약간 더 많음.
• DAYS_FIRST_DRAWING, DAYS_FIRST_DUE, DAYS_LAST_DUE_1ST_VERSION, DAYS_LAST_DUE, DAYS_TERMINATION은 365243 값이 매우 많음.

Category 피처들의 Histogram을 TARGET 유형에 따라 비교

object_columns = app_prev.dtypes[app_prev.dtypes=='object'].index.tolist()
object_columns

['NAME_CONTRACT_TYPE',
 'WEEKDAY_APPR_PROCESS_START',
 'FLAG_LAST_APPL_PER_CONTRACT',
 'NAME_CASH_LOAN_PURPOSE',
 'NAME_CONTRACT_STATUS',
 'NAME_PAYMENT_TYPE',
 'CODE_REJECT_REASON',
 'NAME_TYPE_SUITE',
 'NAME_CLIENT_TYPE',
 'NAME_GOODS_CATEGORY',
 'NAME_PORTFOLIO',
 'NAME_PRODUCT_TYPE',
 'CHANNEL_TYPE',
 'NAME_SELLER_INDUSTRY',
 'NAME_YIELD_GROUP',
 'PRODUCT_COMBINATION']

def show_category_by_target(df, columns):
    for column in columns:
        chart = sns.catplot(x=column, col="TARGET", data=df, kind="count")
        chart.set_xticklabels(rotation=65)
        
show_category_by_target(app_prev, object_columns)

• NAME_CONTRACT_TYPE은 TARGET=1일때 CASH_LOAN의 비중이 약간 높음
• NAME_CONTRACT_STATUS(대출허가상태)는 TARGET=1일때 상대적으로 TARGET=0 대비 (당연히) Refused의 비율이 높음.
• NAME_PAYMENT_TYPE(대출납부방법)는 TARGET=1일때 상대적으로 TARGET=0 대비 XNA의 비율이 약간 높음.

prev 데이터 세트 feature engineering 수행.

SQL 대비 Pandas groupby 사용 로직 비교

SQL로 SK_ID_CURR별 건수, 평균 AMT_CREDIT, 최대 AMT_CREDIT, 최소 AMT_CREDIT 구하기

select sk_id_curr, count(*), avg(amt_credit) , max(amt_credit), min(amt_credit) from previous group by sk_id_curr

pandas groupby 단일 aggregation 함수 사용

• groupby SK_ID_CURR
• SK_ID_CURR별 건수, AMT_CREDIT에 대한 평균, 최대 값

prev.groupby('SK_ID_CURR')

<pandas.core.groupby.generic.DataFrameGroupBy object at 0x7fcfc8bfba90>

prev_group = prev.groupby('SK_ID_CURR')
prev_agg = pd.DataFrame()
prev_agg['CNT'] = prev_group['SK_ID_CURR'].count()
prev_agg['AVG_CREDIT'] = prev_group['AMT_CREDIT'].mean()
prev_agg['MAX_CREDIT'] = prev_group['AMT_CREDIT'].max()
prev_agg['MIN_CREDIT'] = prev_group['AMT_CREDIT'].min()

prev_agg.head()

groupby agg()함수를 이용하여 여러개의 aggregation 함수 적용

prev_group = prev.groupby('SK_ID_CURR')
prev_agg1 = prev_group['AMT_CREDIT'].agg(['mean', 'max', 'min'])
prev_agg2 = prev_group['AMT_ANNUITY'].agg(['mean', 'max', 'min'])

prev_agg = prev_agg1.merge(prev_agg2, on='SK_ID_CURR', how='inner')
prev_agg.head()

agg()에 dictionary를 이용하여 groupby 적용

agg_dict = {
    'SK_ID_CURR':['count'],
    'AMT_CREDIT':['mean', 'max', 'sum'],
    'AMT_ANNUITY':['mean', 'max', 'sum'], 
    'AMT_APPLICATION':['mean', 'max', 'sum'],
    'AMT_DOWN_PAYMENT':['mean', 'max', 'sum'],
    'AMT_GOODS_PRICE':['mean', 'max', 'sum']
}
prev_group = prev.groupby('SK_ID_CURR')
prev_amt_agg = prev_group.agg(agg_dict)
prev_amt_agg.head()

grouby agg 로 만들어진 Multi index 컬럼 변경.

• MultiIndex로 되어 있는 컬럼명 확인
• MultiIndex 컬럼명을 _로 연결하여 컬럼명 변경.

칼럼 인덱스가 계층 구조로 되어있어서 예를 들면 AMT_CREDIT_MEAN 이런식으로 바꾸려 함

prev_amt_agg.columns

MultiIndex([(      'SK_ID_CURR', 'count'),
            (      'AMT_CREDIT',  'mean'),
            (      'AMT_CREDIT',   'max'),
            (      'AMT_CREDIT',   'sum'),
            (     'AMT_ANNUITY',  'mean'),
            (     'AMT_ANNUITY',   'max'),
            (     'AMT_ANNUITY',   'sum'),
            ( 'AMT_APPLICATION',  'mean'),
            ( 'AMT_APPLICATION',   'max'),
            ( 'AMT_APPLICATION',   'sum'),
            ('AMT_DOWN_PAYMENT',  'mean'),
            ('AMT_DOWN_PAYMENT',   'max'),
            ('AMT_DOWN_PAYMENT',   'sum'),
            ( 'AMT_GOODS_PRICE',  'mean'),
            ( 'AMT_GOODS_PRICE',   'max'),
            ( 'AMT_GOODS_PRICE',   'sum')],
           )

# [(column[0]+'_'+column[1].upper()) for column in prev_amt_agg.columns]
prev_amt_agg.columns = ['PREV_' + ('_').join(column).upper() for column in prev_amt_agg.columns.ravel()]

prev_amt_agg.head()

prev 피처 가공. 대출 신청액 대비 다른 금액 차이 및 비율 생성.

prev['PREV_CREDIT_DIFF'] = prev['AMT_APPLICATION'] - prev['AMT_CREDIT']
prev['PREV_GOODS_DIFF'] = prev['AMT_APPLICATION'] - prev['AMT_GOODS_PRICE']
prev['PREV_CREDIT_APPL_RATIO'] = prev['AMT_CREDIT']/prev['AMT_APPLICATION']
prev['PREV_ANNUITY_APPL_RATIO'] = prev['AMT_ANNUITY']/prev['AMT_APPLICATION']
prev['PREV_GOODS_APPL_RATIO'] = prev['AMT_GOODS_PRICE']/prev['AMT_APPLICATION']

DAYS_XXX 피처의 365243 을 NULL로 변환하고, 첫번째 만기일과 마지막 만기일까지의 기간 가공

prev['DAYS_FIRST_DRAWING'].replace(365243, np.nan, inplace=True)
prev['DAYS_FIRST_DUE'].replace(365243, np.nan, inplace= True)
prev['DAYS_LAST_DUE_1ST_VERSION'].replace(365243, np.nan, inplace= True)
prev['DAYS_LAST_DUE'].replace(365243, np.nan, inplace= True)
prev['DAYS_TERMINATION'].replace(365243, np.nan, inplace= True)
# 첫번째 만기일과 마지막 만기일까지의 기간
prev['PREV_DAYS_LAST_DUE_DIFF'] = prev['DAYS_LAST_DUE_1ST_VERSION'] - prev['DAYS_LAST_DUE']

기존 이자율 관련 컬럼이 null이 많아서 새롭게 간단한 이자율을 대출 금액과 대출 금액 납부 횟수를 기반으로 계산

prev.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1670214 entries, 0 to 1670213
Data columns (total 43 columns):
SK_ID_PREV                     1670214 non-null int64
SK_ID_CURR                     1670214 non-null int64
NAME_CONTRACT_TYPE             1670214 non-null object
AMT_ANNUITY                    1297979 non-null float64
AMT_APPLICATION                1670214 non-null float64
AMT_CREDIT                     1670213 non-null float64
AMT_DOWN_PAYMENT               774370 non-null float64
AMT_GOODS_PRICE                1284699 non-null float64
WEEKDAY_APPR_PROCESS_START     1670214 non-null object
HOUR_APPR_PROCESS_START        1670214 non-null int64
FLAG_LAST_APPL_PER_CONTRACT    1670214 non-null object
NFLAG_LAST_APPL_IN_DAY         1670214 non-null int64
RATE_DOWN_PAYMENT              774370 non-null float64
RATE_INTEREST_PRIMARY          5951 non-null float64
RATE_INTEREST_PRIVILEGED       5951 non-null float64
NAME_CASH_LOAN_PURPOSE         1670214 non-null object
NAME_CONTRACT_STATUS           1670214 non-null object
DAYS_DECISION                  1670214 non-null int64
NAME_PAYMENT_TYPE              1670214 non-null object
CODE_REJECT_REASON             1670214 non-null object
NAME_TYPE_SUITE                849809 non-null object
NAME_CLIENT_TYPE               1670214 non-null object
NAME_GOODS_CATEGORY            1670214 non-null object
NAME_PORTFOLIO                 1670214 non-null object
NAME_PRODUCT_TYPE              1670214 non-null object
CHANNEL_TYPE                   1670214 non-null object
SELLERPLACE_AREA               1670214 non-null int64
NAME_SELLER_INDUSTRY           1670214 non-null object
CNT_PAYMENT                    1297984 non-null float64
NAME_YIELD_GROUP               1670214 non-null object
PRODUCT_COMBINATION            1669868 non-null object
DAYS_FIRST_DRAWING             62705 non-null float64
DAYS_FIRST_DUE                 956504 non-null float64
DAYS_LAST_DUE_1ST_VERSION      903285 non-null float64
DAYS_LAST_DUE                  785928 non-null float64
DAYS_TERMINATION               771236 non-null float64
NFLAG_INSURED_ON_APPROVAL      997149 non-null float64
PREV_CREDIT_DIFF               1670213 non-null float64
PREV_GOODS_DIFF                1284699 non-null float64
PREV_CREDIT_APPL_RATIO         1333446 non-null float64
PREV_ANNUITY_APPL_RATIO        1296344 non-null float64
PREV_GOODS_APPL_RATIO          1277830 non-null float64
PREV_DAYS_LAST_DUE_DIFF        761917 non-null float64
dtypes: float64(21), int64(6), object(16)
memory usage: 547.9+ MB

CNT_PAYMENT 는 전체 납부해야하는 횟수, AMT_ANNUITY 월 별 납부해야하는 금액 -> 이것으로 간단하게 이자율 계산

all_pay = prev['AMT_ANNUITY'] * prev['CNT_PAYMENT']
prev['PREV_INTERESTS_RATE'] = (all_pay/prev['AMT_CREDIT'] - 1)/prev['CNT_PAYMENT']

prev.iloc[:, -7:].head(10)

기존 대출 금액, 대출 상태 관련 피처들과 이들을 가공하여 만들어진 새로운 컬럼들로 aggregation 수행.

agg_dict = {
     # 기존 컬럼. 
    'SK_ID_CURR':['count'],
    'AMT_CREDIT':['mean', 'max', 'sum'],
    'AMT_ANNUITY':['mean', 'max', 'sum'], 
    'AMT_APPLICATION':['mean', 'max', 'sum'],
    'AMT_DOWN_PAYMENT':['mean', 'max', 'sum'],
    'AMT_GOODS_PRICE':['mean', 'max', 'sum'],
    'RATE_DOWN_PAYMENT': ['min', 'max', 'mean'],
    'DAYS_DECISION': ['min', 'max', 'mean'],
    'CNT_PAYMENT': ['mean', 'sum'],
    # 가공 컬럼
    'PREV_CREDIT_DIFF':['mean', 'max', 'sum'], 
    'PREV_CREDIT_APPL_RATIO':['mean', 'max'],
    'PREV_GOODS_DIFF':['mean', 'max', 'sum'],
    'PREV_GOODS_APPL_RATIO':['mean', 'max'],
    'PREV_DAYS_LAST_DUE_DIFF':['mean', 'max', 'sum'],
    'PREV_INTERESTS_RATE':['mean', 'max']
}

prev_group = prev.groupby('SK_ID_CURR')
prev_amt_agg = prev_group.agg(agg_dict)
prev_amt_agg.columns = ['PREV_'+('_').join(column).upper() for column in prev_amt_agg.columns.ravel()]

prev_amt_agg.head(10)

prev_amt_agg.shape

(338857, 39)

SK_ID_CURR별로 NAME_CONTRACT_STATUS가 Refused 일 경우의 건수 및 과거 대출건 대비 비율

Group by 기준 컬럼 기반에서 다른 컬럼들의 기준에 따라 세분화된 aggregation 수행.

prev['NAME_CONTRACT_STATUS'].value_counts()

Approved        1036781
Canceled         316319
Refused          290678
Unused offer      26436
Name: NAME_CONTRACT_STATUS, dtype: int64

SQL Group by Case when 과 pandas의 차이

SK_ID_CURR레벨로 groupby 된 count와 name_contract_status가 Refused일 때의 count

select sk_id_curr, cnt_refused/cnt
from
(
    select sk_id_curr, count(*) cnt, count(case when name_contract_status == 'Refused' end) cnt_refused
    from previous group by sk_id_curr
) 

Pandas는 원 DataFrame 에 groupby 적용된 DataFrame 과 세부기준으로 filtering 된 DataFrame에 groupby 적용된 DataFrame 을 조인하여 생성.

• NAME_CONTRACT_STATUS == 'Refused' 세부 기준으로 filtering 및 filtering 된 DataFrame에 groupby 적용
• groupby 완료 후 기존 prev_amt_agg와 조인
• 효율적인 오류 방지를 위해서 groupby 시 적용후 groupby key값을 DataFrame의 Index가 아닌 일반 컬럼으로 변경.

cond_refused = (prev['NAME_CONTRACT_STATUS'] == 'Refused')
prev_refused = prev[cond_refused]
prev_refused.shape, prev.shape

((290678, 44), (1670214, 44))

prev_refused_agg = prev_refused.groupby('SK_ID_CURR')['SK_ID_CURR'].count()
prev_refused_agg.shape, prev_amt_agg.shape

((118277,), (338857, 39))

prev_refused_agg

SK_ID_CURR
100006     1
100011     1
100027     1
100030    10
100035     8
          ..
456244     1
456247     1
456249     1
456250     1
456255     2
Name: SK_ID_CURR, Length: 118277, dtype: int64

# 칼럼과 인덱스 이름이 같아서 안된다. 일부러 error 냈음
prev_amt_agg.merge(prev_refused_agg, how='left', on='SK_ID_CURR')

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

<ipython-input-54-3bccc0d949fd> in <module>
----> 1 prev_amt_agg.merge(prev_refused_agg, how='left', on='SK_ID_CURR')


~/opt/anaconda3/lib/python3.8/site-packages/pandas/core/frame.py in merge(self, right, how, on, left_on, right_on, left_index, right_index, sort, suffixes, copy, indicator, validate)
   7319         from pandas.core.reshape.merge import merge
   7320 
-> 7321         return merge(
   7322             self,
   7323             right,


~/opt/anaconda3/lib/python3.8/site-packages/pandas/core/reshape/merge.py in merge(left, right, how, on, left_on, right_on, left_index, right_index, sort, suffixes, copy, indicator, validate)
     66     validate=None,
     67 ):
---> 68     op = _MergeOperation(
     69         left,
     70         right,


~/opt/anaconda3/lib/python3.8/site-packages/pandas/core/reshape/merge.py in __init__(self, left, right, how, on, left_on, right_on, axis, left_index, right_index, sort, suffixes, copy, indicator, validate)
    624             self.right_join_keys,
    625             self.join_names,
--> 626         ) = self._get_merge_keys()
    627 
    628         # validate the merge keys dtypes. We may need to coerce


~/opt/anaconda3/lib/python3.8/site-packages/pandas/core/reshape/merge.py in _get_merge_keys(self)
    973                     if not is_rkey(rk):
    974                         if rk is not None:
--> 975                             right_keys.append(right._get_label_or_level_values(rk))
    976                         else:
    977                             # work-around for merge_asof(right_index=True)


~/opt/anaconda3/lib/python3.8/site-packages/pandas/core/generic.py in _get_label_or_level_values(self, key, axis)
   1767 
   1768         if self._is_label_reference(key, axis=axis):
-> 1769             self._check_label_or_level_ambiguity(key, axis=axis)
   1770             values = self.xs(key, axis=other_axes[0])._values
   1771         elif self._is_level_reference(key, axis=axis):


~/opt/anaconda3/lib/python3.8/site-packages/pandas/core/generic.py in _check_label_or_level_ambiguity(self, key, axis)
   1727                 label_type=label_type,
   1728             )
-> 1729             raise ValueError(msg)
   1730 
   1731     def _get_label_or_level_values(self, key, axis=0):


ValueError: 'SK_ID_CURR' is both an index level and a column label, which is ambiguous.

pd.DataFrame(prev_refused_agg)

prev_refused_agg.reset_index(name='PREV_REFUSED_COUNT')

prev_refused_agg = prev_refused_agg.reset_index(name='PREV_REFUSED_COUNT')
prev_amt_agg = prev_amt_agg.reset_index()

prev_amt_refused_agg = prev_amt_agg.merge(prev_refused_agg, on='SK_ID_CURR', how='left')
prev_amt_refused_agg.head(10)

prev_amt_refused_agg['PREV_REFUSED_COUNT'].value_counts(dropna=False)

NaN     220580
1.0      54616
2.0      26793
3.0      14025
4.0       8243
5.0       4859
6.0       3147
7.0       2069
8.0       1247
9.0        871
10.0       656
11.0       437
12.0       340
13.0       233
14.0       157
15.0       149
16.0        94
17.0        62
18.0        54
19.0        48
20.0        28
21.0        25
22.0        21
24.0        18
25.0        12
23.0        11
26.0        11
27.0         8
28.0         5
29.0         5
32.0         4
41.0         3
31.0         3
37.0         3
30.0         3
39.0         2
35.0         2
34.0         2
33.0         2
36.0         2
68.0         1
47.0         1
43.0         1
54.0         1
51.0         1
64.0         1
58.0         1
Name: PREV_REFUSED_COUNT, dtype: int64

계산된 PREV_REFUSED_COUNT 중 Null값은 0 으로 변경하고 SK_ID_CURR 개별 건수 대비 PREV_REFUSED_COUNT 비율 계산

prev_amt_refused_agg = prev_amt_refused_agg.fillna(0)
prev_amt_refused_agg['PREV_REFUSE_RATIO'] = prev_amt_refused_agg['PREV_REFUSED_COUNT'] / prev_amt_refused_agg['PREV_SK_ID_CURR_COUNT']
prev_amt_refused_agg.head(10)

세부 레벨 groupby 와 unstack()을 이용하여 SQL Group by Case when 구현. 세부 조건이 2개 이상일때

SQL 일 경우

SELECT COUNT(CASE WHEN == 'Approved' END) , COUNT(CASE WHEN == 'Refused' END) FROM PREV GROUP BY SK_ID_CURR

Pandas로 수행

prev_refused_appr_group = prev[prev['NAME_CONTRACT_STATUS'].isin(['Approved', 'Refused'])].groupby(['SK_ID_CURR', 'NAME_CONTRACT_STATUS'])
prev_refused_appr_agg = prev_refused_appr_group['SK_ID_CURR'].count().unstack()
prev_refused_appr_agg.head(10)

컬럼명 변경, Null 처리, 그리고 기존의 prev_amt_agg와 조인 후 데이터 가공

prev_refused_appr_agg = prev_refused_appr_agg.fillna(0)
prev_refused_appr_agg.columns = ['PREV_APPROVED_COUNT', 'PREV_REFUSED_COUNT']
prev_refused_appr_agg = prev_refused_appr_agg.reset_index()
prev_refused_appr_agg.head(10)

prev_amt_agg.head()

# prev_amt_agg와 조인. prev_amt_agg와 prev_refused_appr_agg 모두 SK_ID_CURR을 INDEX로 가지고 있음. 
prev_agg = prev_amt_agg.merge(prev_refused_appr_agg, on='SK_ID_CURR', how='left')
# SK_ID_CURR별 과거 대출건수 대비 APPROVED_COUNT 및 REFUSED_COUNT 비율 생성. 
prev_agg['PREV_REFUSED_RATIO'] = prev_agg['PREV_REFUSED_COUNT']/prev_agg['PREV_SK_ID_CURR_COUNT']
prev_agg['PREV_APPROVED_RATIO'] = prev_agg['PREV_APPROVED_COUNT']/prev_agg['PREV_SK_ID_CURR_COUNT']
# 'PREV_REFUSED_COUNT', 'PREV_APPROVED_COUNT' 컬럼 drop 
prev_agg = prev_agg.drop(['PREV_REFUSED_COUNT', 'PREV_APPROVED_COUNT'], axis=1)
# prev_amt_agg와 prev_refused_appr_agg INDEX인 SK_ID_CURR이 조인 후 정식 컬럼으로 생성됨. 
prev_agg.head(30)

가공된 최종 데이터 세트 생성

이전에 application 데이터 세트의 feature engineering 수행 후 새롭게 previous 데이터 세트로 가공된 데이터를 조인.

apps_all =  get_apps_processed(apps)

print(apps_all.shape, prev_agg.shape)
apps_all = apps_all.merge(prev_agg, on='SK_ID_CURR', how='left')
print(apps_all.shape)

(356255, 135) (338857, 42)
(356255, 176)

apps_all.head()

apps_all.info()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 356255 entries, 0 to 356254
Columns: 176 entries, AMT_ANNUITY to PREV_APPROVED_RATIO
dtypes: float64(120), int64(40), object(16)
memory usage: 481.1+ MB

데이터 레이블 인코딩, NULL값은 LightGBM 내부에서 처리하도록 특별한 변경하지 않음.

object_columns = apps_all.dtypes[apps_all.dtypes == 'object'].index.tolist()
for column in object_columns:
    apps_all[column] = pd.factorize(apps_all[column])[0]

apps_all.info()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 356255 entries, 0 to 356254
Columns: 176 entries, AMT_ANNUITY to PREV_APPROVED_RATIO
dtypes: float64(120), int64(56)
memory usage: 481.1 MB

학습 데이터와 테스트 데이터 다시 분리

apps_all_train = apps_all[~apps_all['TARGET'].isnull()]
apps_all_test = apps_all[apps_all['TARGET'].isnull()]

apps_all_test = apps_all_test.drop('TARGET', axis=1)

apps_all_train.columns.tolist()

['AMT_ANNUITY',
 'AMT_CREDIT',
 'AMT_GOODS_PRICE',
 'AMT_INCOME_TOTAL',
 'AMT_REQ_CREDIT_BUREAU_DAY',
 'AMT_REQ_CREDIT_BUREAU_HOUR',
 'AMT_REQ_CREDIT_BUREAU_MON',
 'AMT_REQ_CREDIT_BUREAU_QRT',
 'AMT_REQ_CREDIT_BUREAU_WEEK',
 'AMT_REQ_CREDIT_BUREAU_YEAR',
 'APARTMENTS_AVG',
 'APARTMENTS_MEDI',
 'APARTMENTS_MODE',
 'BASEMENTAREA_AVG',
 'BASEMENTAREA_MEDI',
 'BASEMENTAREA_MODE',
 'CNT_CHILDREN',
 'CNT_FAM_MEMBERS',
 'CODE_GENDER',
 'COMMONAREA_AVG',
 'COMMONAREA_MEDI',
 'COMMONAREA_MODE',
 'DAYS_BIRTH',
 'DAYS_EMPLOYED',
 'DAYS_ID_PUBLISH',
 'DAYS_LAST_PHONE_CHANGE',
 'DAYS_REGISTRATION',
 'DEF_30_CNT_SOCIAL_CIRCLE',
 'DEF_60_CNT_SOCIAL_CIRCLE',
 'ELEVATORS_AVG',
 'ELEVATORS_MEDI',
 'ELEVATORS_MODE',
 'EMERGENCYSTATE_MODE',
 'ENTRANCES_AVG',
 'ENTRANCES_MEDI',
 'ENTRANCES_MODE',
 'EXT_SOURCE_1',
 'EXT_SOURCE_2',
 'EXT_SOURCE_3',
 'FLAG_CONT_MOBILE',
 'FLAG_DOCUMENT_10',
 'FLAG_DOCUMENT_11',
 'FLAG_DOCUMENT_12',
 'FLAG_DOCUMENT_13',
 'FLAG_DOCUMENT_14',
 'FLAG_DOCUMENT_15',
 'FLAG_DOCUMENT_16',
 'FLAG_DOCUMENT_17',
 'FLAG_DOCUMENT_18',
 'FLAG_DOCUMENT_19',
 'FLAG_DOCUMENT_2',
 'FLAG_DOCUMENT_20',
 'FLAG_DOCUMENT_21',
 'FLAG_DOCUMENT_3',
 'FLAG_DOCUMENT_4',
 'FLAG_DOCUMENT_5',
 'FLAG_DOCUMENT_6',
 'FLAG_DOCUMENT_7',
 'FLAG_DOCUMENT_8',
 'FLAG_DOCUMENT_9',
 'FLAG_EMAIL',
 'FLAG_EMP_PHONE',
 'FLAG_MOBIL',
 'FLAG_OWN_CAR',
 'FLAG_OWN_REALTY',
 'FLAG_PHONE',
 'FLAG_WORK_PHONE',
 'FLOORSMAX_AVG',
 'FLOORSMAX_MEDI',
 'FLOORSMAX_MODE',
 'FLOORSMIN_AVG',
 'FLOORSMIN_MEDI',
 'FLOORSMIN_MODE',
 'FONDKAPREMONT_MODE',
 'HOUR_APPR_PROCESS_START',
 'HOUSETYPE_MODE',
 'LANDAREA_AVG',
 'LANDAREA_MEDI',
 'LANDAREA_MODE',
 'LIVE_CITY_NOT_WORK_CITY',
 'LIVE_REGION_NOT_WORK_REGION',
 'LIVINGAPARTMENTS_AVG',
 'LIVINGAPARTMENTS_MEDI',
 'LIVINGAPARTMENTS_MODE',
 'LIVINGAREA_AVG',
 'LIVINGAREA_MEDI',
 'LIVINGAREA_MODE',
 'NAME_CONTRACT_TYPE',
 'NAME_EDUCATION_TYPE',
 'NAME_FAMILY_STATUS',
 'NAME_HOUSING_TYPE',
 'NAME_INCOME_TYPE',
 'NAME_TYPE_SUITE',
 'NONLIVINGAPARTMENTS_AVG',
 'NONLIVINGAPARTMENTS_MEDI',
 'NONLIVINGAPARTMENTS_MODE',
 'NONLIVINGAREA_AVG',
 'NONLIVINGAREA_MEDI',
 'NONLIVINGAREA_MODE',
 'OBS_30_CNT_SOCIAL_CIRCLE',
 'OBS_60_CNT_SOCIAL_CIRCLE',
 'OCCUPATION_TYPE',
 'ORGANIZATION_TYPE',
 'OWN_CAR_AGE',
 'REGION_POPULATION_RELATIVE',
 'REGION_RATING_CLIENT',
 'REGION_RATING_CLIENT_W_CITY',
 'REG_CITY_NOT_LIVE_CITY',
 'REG_CITY_NOT_WORK_CITY',
 'REG_REGION_NOT_LIVE_REGION',
 'REG_REGION_NOT_WORK_REGION',
 'SK_ID_CURR',
 'TARGET',
 'TOTALAREA_MODE',
 'WALLSMATERIAL_MODE',
 'WEEKDAY_APPR_PROCESS_START',
 'YEARS_BEGINEXPLUATATION_AVG',
 'YEARS_BEGINEXPLUATATION_MEDI',
 'YEARS_BEGINEXPLUATATION_MODE',
 'YEARS_BUILD_AVG',
 'YEARS_BUILD_MEDI',
 'YEARS_BUILD_MODE',
 'APPS_EXT_SOURCE_MEAN',
 'APPS_EXT_SOURCE_STD',
 'APPS_ANNUITY_CREDIT_RATIO',
 'APPS_GOODS_CREDIT_RATIO',
 'APPS_ANNUITY_INCOME_RATIO',
 'APPS_CREDIT_INCOME_RATIO',
 'APPS_GOODS_INCOME_RATIO',
 'APPS_CNT_FAM_INCOME_RATIO',
 'APPS_EMPLOYED_BIRTH_RATIO',
 'APPS_INCOME_EMPLOYED_RATIO',
 'APPS_INCOME_BIRTH_RATIO',
 'APPS_CAR_BIRTH_RATIO',
 'APPS_CAR_EMPLOYED_RATIO',
 'PREV_SK_ID_CURR_COUNT',
 'PREV_AMT_CREDIT_MEAN',
 'PREV_AMT_CREDIT_MAX',
 'PREV_AMT_CREDIT_SUM',
 'PREV_AMT_ANNUITY_MEAN',
 'PREV_AMT_ANNUITY_MAX',
 'PREV_AMT_ANNUITY_SUM',
 'PREV_AMT_APPLICATION_MEAN',
 'PREV_AMT_APPLICATION_MAX',
 'PREV_AMT_APPLICATION_SUM',
 'PREV_AMT_DOWN_PAYMENT_MEAN',
 'PREV_AMT_DOWN_PAYMENT_MAX',
 'PREV_AMT_DOWN_PAYMENT_SUM',
 'PREV_AMT_GOODS_PRICE_MEAN',
 'PREV_AMT_GOODS_PRICE_MAX',
 'PREV_AMT_GOODS_PRICE_SUM',
 'PREV_RATE_DOWN_PAYMENT_MIN',
 'PREV_RATE_DOWN_PAYMENT_MAX',
 'PREV_RATE_DOWN_PAYMENT_MEAN',
 'PREV_DAYS_DECISION_MIN',
 'PREV_DAYS_DECISION_MAX',
 'PREV_DAYS_DECISION_MEAN',
 'PREV_CNT_PAYMENT_MEAN',
 'PREV_CNT_PAYMENT_SUM',
 'PREV_PREV_CREDIT_DIFF_MEAN',
 'PREV_PREV_CREDIT_DIFF_MAX',
 'PREV_PREV_CREDIT_DIFF_SUM',
 'PREV_PREV_CREDIT_APPL_RATIO_MEAN',
 'PREV_PREV_CREDIT_APPL_RATIO_MAX',
 'PREV_PREV_GOODS_DIFF_MEAN',
 'PREV_PREV_GOODS_DIFF_MAX',
 'PREV_PREV_GOODS_DIFF_SUM',
 'PREV_PREV_GOODS_APPL_RATIO_MEAN',
 'PREV_PREV_GOODS_APPL_RATIO_MAX',
 'PREV_PREV_DAYS_LAST_DUE_DIFF_MEAN',
 'PREV_PREV_DAYS_LAST_DUE_DIFF_MAX',
 'PREV_PREV_DAYS_LAST_DUE_DIFF_SUM',
 'PREV_PREV_INTERESTS_RATE_MEAN',
 'PREV_PREV_INTERESTS_RATE_MAX',
 'PREV_REFUSED_RATIO',
 'PREV_APPROVED_RATIO']

학습 데이터를 검증 데이터로 분리하고 LGBM Classifier로 학습 수행.

from sklearn.model_selection import train_test_split

ftr_app = apps_all_train.drop(['SK_ID_CURR', 'TARGET'], axis=1)
target_app = apps_all_train['TARGET']

train_x, valid_x, train_y, valid_y = train_test_split(ftr_app, target_app, test_size=0.3, random_state=2020)
train_x.shape, valid_x.shape

((215257, 174), (92254, 174))

from lightgbm import LGBMClassifier

clf = LGBMClassifier(
        n_jobs=-1,
        n_estimators=1000,
        learning_rate=0.02,
        num_leaves=32,
        subsample=0.8,
        max_depth=12,
        silent=-1,
        verbose=-1
        )

clf.fit(train_x, train_y, eval_set=[(train_x, train_y), (valid_x, valid_y)], eval_metric= 'auc', verbose= 100, 
        early_stopping_rounds= 50)

Training until validation scores don't improve for 50 rounds
[100]	training's auc: 0.76611	training's binary_logloss: 0.245964	valid_1's auc: 0.753701	valid_1's binary_logloss: 0.248684
[200]	training's auc: 0.789025	training's binary_logloss: 0.237779	valid_1's auc: 0.766044	valid_1's binary_logloss: 0.244034
[300]	training's auc: 0.804705	training's binary_logloss: 0.232402	valid_1's auc: 0.770975	valid_1's binary_logloss: 0.242307
[400]	training's auc: 0.81739	training's binary_logloss: 0.228102	valid_1's auc: 0.773264	valid_1's binary_logloss: 0.241518
[500]	training's auc: 0.828598	training's binary_logloss: 0.224327	valid_1's auc: 0.773917	valid_1's binary_logloss: 0.241253
[600]	training's auc: 0.839337	training's binary_logloss: 0.220751	valid_1's auc: 0.77427	valid_1's binary_logloss: 0.241111
[700]	training's auc: 0.848905	training's binary_logloss: 0.217397	valid_1's auc: 0.774436	valid_1's binary_logloss: 0.241006
[800]	training's auc: 0.857485	training's binary_logloss: 0.214237	valid_1's auc: 0.774647	valid_1's binary_logloss: 0.240922
[900]	training's auc: 0.866027	training's binary_logloss: 0.211076	valid_1's auc: 0.774776	valid_1's binary_logloss: 0.24087
Early stopping, best iteration is:
[853]	training's auc: 0.862238	training's binary_logloss: 0.212526	valid_1's auc: 0.774799	valid_1's binary_logloss: 0.240865





LGBMClassifier(learning_rate=0.02, max_depth=12, n_estimators=1000,
               num_leaves=32, silent=-1, subsample=0.8, verbose=-1)

from lightgbm import plot_importance

plot_importance(clf, figsize=(16, 32))

학습된 Classifier를 이용하여 테스트 데이터 예측하고 결과를 Kaggle로 Submit 수행.

preds = clf.predict_proba(apps_all_test.drop('SK_ID_CURR', axis=1))[:, 1 ]
apps_all_test['TARGET'] = preds
apps_all_test[['SK_ID_CURR', 'TARGET']].to_csv('prev_baseline_01.csv', index=False)

차수	Private Score	Public Score	설명
1차	0.74088	0.74448	application 데이터 세트 기본 preprocessing
2차	0.75458	0.75882	application 데이터 세트 Feature Engineering
3차	0.76396	0.77579	previous 데이터 세트 Feature Engineering

지금까지 만든 로직을 별도의 함수로 생성.

from sklearn.model_selection import train_test_split
from lightgbm import LGBMClassifier

# 신청금액과 실제 대출액, 상품금액과의 차이 비율, 만기일 차이 비교, 이자율 계산등의 주요 컬럼 가공 생산. 
def get_prev_processed(prev):
    # 대출 신청 금액과 실제 대출액/대출 상품금액 차이 및 비율
    prev['PREV_CREDIT_DIFF'] = prev['AMT_APPLICATION'] - prev['AMT_CREDIT']
    prev['PREV_GOODS_DIFF'] = prev['AMT_APPLICATION'] - prev['AMT_GOODS_PRICE']
    prev['PREV_CREDIT_APPL_RATIO'] = prev['AMT_CREDIT']/prev['AMT_APPLICATION']
    # prev['PREV_ANNUITY_APPL_RATIO'] = prev['AMT_ANNUITY']/prev['AMT_APPLICATION']
    prev['PREV_GOODS_APPL_RATIO'] = prev['AMT_GOODS_PRICE']/prev['AMT_APPLICATION']
    
    prev['DAYS_FIRST_DRAWING'].replace(365243, np.nan, inplace= True)
    prev['DAYS_FIRST_DUE'].replace(365243, np.nan, inplace= True)
    prev['DAYS_LAST_DUE_1ST_VERSION'].replace(365243, np.nan, inplace= True)
    prev['DAYS_LAST_DUE'].replace(365243, np.nan, inplace= True)
    prev['DAYS_TERMINATION'].replace(365243, np.nan, inplace= True)
    # 첫번째 만기일과 마지막 만기일까지의 기간
    prev['PREV_DAYS_LAST_DUE_DIFF'] = prev['DAYS_LAST_DUE_1ST_VERSION'] - prev['DAYS_LAST_DUE']
    # 매월 납부 금액과 납부 횟수 곱해서 전체 납부 금액 구함. 
    all_pay = prev['AMT_ANNUITY'] * prev['CNT_PAYMENT']
    # 전체 납부 금액 대비 AMT_CREDIT 비율을 구하고 여기에 다시 납부횟수로 나누어서 이자율 계산. 
    prev['PREV_INTERESTS_RATE'] = (all_pay/prev['AMT_CREDIT'] - 1)/prev['CNT_PAYMENT']
        
    return prev

# 기존 컬럼및 위에서 가공된 신규 컬럼들에 대해서 SK_ID_CURR 레벨로 Aggregation 수행.  
def get_prev_amt_agg(prev):

    agg_dict = {
         # 기존 주요 컬럼들을 SK_ID_CURR 레벨로 Aggregation 수행. . 
        'SK_ID_CURR':['count'],
        'AMT_CREDIT':['mean', 'max', 'sum'],
        'AMT_ANNUITY':['mean', 'max', 'sum'], 
        'AMT_APPLICATION':['mean', 'max', 'sum'],
        'AMT_DOWN_PAYMENT':['mean', 'max', 'sum'],
        'AMT_GOODS_PRICE':['mean', 'max', 'sum'],
        'RATE_DOWN_PAYMENT': ['min', 'max', 'mean'],
        'DAYS_DECISION': ['min', 'max', 'mean'],
        'CNT_PAYMENT': ['mean', 'sum'],
        # 신규 가공 컬럼들을 SK_ID_CURR 레벨로 Aggregation 수행. .
        'PREV_CREDIT_DIFF':['mean', 'max', 'sum'], 
        'PREV_CREDIT_APPL_RATIO':['mean', 'max'],
        'PREV_GOODS_DIFF':['mean', 'max', 'sum'],
        'PREV_GOODS_APPL_RATIO':['mean', 'max'],
        'PREV_DAYS_LAST_DUE_DIFF':['mean', 'max', 'sum'],
        'PREV_INTERESTS_RATE':['mean', 'max']
    }

    prev_group = prev.groupby('SK_ID_CURR')
    prev_amt_agg = prev_group.agg(agg_dict)

    # multi index 컬럼을 '_'로 연결하여 컬럼명 변경
    prev_amt_agg.columns = ["PREV_"+ "_".join(x).upper() for x in prev_amt_agg.columns.ravel()]
    
    # 'SK_ID_CURR'로 조인하기 위해 SK_ID_CURR을 컬럼으로 변환  
    prev_amt_agg = prev_amt_agg.reset_index()
    
    return prev_amt_agg

# NAME_CONTRACT_STATUS의 SK_ID_CURR별 Approved, Refused의 건수 계산.  
def get_prev_refused_appr_agg(prev):
    # 원래 groupby 컬럼 + 세부 기준 컬럼으로 groupby 수행. 세분화된 레벨로 aggregation 수행 한 뒤에 unstack()으로 컬럼레벨로 변형. 
    prev_refused_appr_group = prev[prev['NAME_CONTRACT_STATUS'].isin(['Approved', 'Refused'])].groupby([ 'SK_ID_CURR', 'NAME_CONTRACT_STATUS'])
    prev_refused_appr_agg = prev_refused_appr_group['SK_ID_CURR'].count().unstack()
    # 컬럼명 변경. 
    prev_refused_appr_agg.columns = ['PREV_APPROVED_COUNT', 'PREV_REFUSED_COUNT' ]
    # NaN값은 모두 0으로 변경. 
    prev_refused_appr_agg = prev_refused_appr_agg.fillna(0)
    
    # 'SK_ID_CURR'로 조인하기 위해 SK_ID_CURR을 컬럼으로 변환  
    prev_refused_appr_agg = prev_refused_appr_agg.reset_index()
    
    return prev_refused_appr_agg

    
# 앞에서 구한 prev_amt_agg와 prev_refused_appr_agg를 조인하고 SK_ID_CURR별 APPROVED_COUNT 및 REFUSED_COUNT 비율 생성
def get_prev_agg(prev):
    prev = get_prev_processed(prev)
    prev_amt_agg = get_prev_amt_agg(prev)
    prev_refused_appr_agg = get_prev_refused_appr_agg(prev)
    
    # prev_amt_agg와 조인. 
    prev_agg = prev_amt_agg.merge(prev_refused_appr_agg, on='SK_ID_CURR', how='left')
    # SK_ID_CURR별 과거 대출건수 대비 APPROVED_COUNT 및 REFUSED_COUNT 비율 생성. 
    prev_agg['PREV_REFUSED_RATIO'] = prev_agg['PREV_REFUSED_COUNT']/prev_agg['PREV_SK_ID_CURR_COUNT']
    prev_agg['PREV_APPROVED_RATIO'] = prev_agg['PREV_APPROVED_COUNT']/prev_agg['PREV_SK_ID_CURR_COUNT']
    # 'PREV_REFUSED_COUNT', 'PREV_APPROVED_COUNT' 컬럼 drop 
    prev_agg = prev_agg.drop(['PREV_REFUSED_COUNT', 'PREV_APPROVED_COUNT'], axis=1)
    
    return prev_agg

# apps와 previous 데이터 세트를 SK_ID_CURR레벨로 다양한 컬럼이 aggregation되어 있는 prev_agg 조인
def get_apps_all_with_prev_agg(apps, prev):
    apps_all =  get_apps_processed(apps)
    prev_agg = get_prev_agg(prev)
    print('prev_agg shape:', prev_agg.shape)
    print('apps_all before merge shape:', apps_all.shape)
    apps_all = apps_all.merge(prev_agg, on='SK_ID_CURR', how='left')
    print('apps_all after merge with prev_agg shape:', apps_all.shape)
    
    return apps_all

# Label 인코딩 수행. 
def get_apps_all_encoded(apps_all):
    object_columns = apps_all.dtypes[apps_all.dtypes == 'object'].index.tolist()
    for column in object_columns:
        apps_all[column] = pd.factorize(apps_all[column])[0]
    
    return apps_all

# 학습 데이터와 테스트 데이터 세트 분리. 
def get_apps_all_train_test(apps_all):
    apps_all_train = apps_all[~apps_all['TARGET'].isnull()]
    apps_all_test = apps_all[apps_all['TARGET'].isnull()]

    apps_all_test = apps_all_test.drop('TARGET', axis=1)
    
    return apps_all_train, apps_all_test

# 학습 수행. 
def train_apps_all(apps_all_train):
    ftr_app = apps_all_train.drop(['SK_ID_CURR', 'TARGET'], axis=1)
    target_app = apps_all_train['TARGET']

    train_x, valid_x, train_y, valid_y = train_test_split(ftr_app, target_app, test_size=0.3, random_state=2020)
    print('train shape:', train_x.shape, 'valid shape:', valid_x.shape)
    clf = LGBMClassifier(
            n_jobs=-1,
            n_estimators=1000,
            learning_rate=0.02,
            num_leaves=32,
            subsample=0.8,
            max_depth=12,
            silent=-1,
            verbose=-1
                )

    clf.fit(train_x, train_y, eval_set=[(train_x, train_y), (valid_x, valid_y)], eval_metric= 'auc', verbose= 100, 
                early_stopping_rounds= 100)
    
    return clf

함수를 호출하여 재학습 및 평가

def get_dataset():
    app_train = pd.read_csv('application_train.csv')
    app_test = pd.read_csv('application_test.csv')
    
    apps = pd.concat([app_train, app_test])
    prev = pd.read_csv('previous_application.csv')
    return apps, prev

apps, prev = get_dataset()
apps_all = get_apps_all_with_prev_agg(apps, prev)
apps_all = get_apps_all_encoded(apps_all)
apps_all_train, apps_all_test = get_apps_all_train_test(apps_all)
clf = train_apps_all(apps_all_train)

prev_agg shape: (338857, 42)
apps_all before merge shape: (356255, 135)
apps_all after merge with prev_agg shape: (356255, 176)
train shape: (215257, 174) valid shape: (92254, 174)
Training until validation scores don't improve for 100 rounds
[100]	training's auc: 0.76611	training's binary_logloss: 0.245964	valid_1's auc: 0.753701	valid_1's binary_logloss: 0.248684
[200]	training's auc: 0.789025	training's binary_logloss: 0.237779	valid_1's auc: 0.766044	valid_1's binary_logloss: 0.244034
[300]	training's auc: 0.804705	training's binary_logloss: 0.232402	valid_1's auc: 0.770975	valid_1's binary_logloss: 0.242307
[400]	training's auc: 0.81739	training's binary_logloss: 0.228102	valid_1's auc: 0.773264	valid_1's binary_logloss: 0.241518
[500]	training's auc: 0.828598	training's binary_logloss: 0.224327	valid_1's auc: 0.773917	valid_1's binary_logloss: 0.241253
[600]	training's auc: 0.839337	training's binary_logloss: 0.220751	valid_1's auc: 0.77427	valid_1's binary_logloss: 0.241111
[700]	training's auc: 0.848905	training's binary_logloss: 0.217397	valid_1's auc: 0.774436	valid_1's binary_logloss: 0.241006
[800]	training's auc: 0.857485	training's binary_logloss: 0.214237	valid_1's auc: 0.774647	valid_1's binary_logloss: 0.240922
[900]	training's auc: 0.866027	training's binary_logloss: 0.211076	valid_1's auc: 0.774776	valid_1's binary_logloss: 0.24087
[1000]	training's auc: 0.873512	training's binary_logloss: 0.208114	valid_1's auc: 0.774799	valid_1's binary_logloss: 0.240861
Did not meet early stopping. Best iteration is:
[1000]	training's auc: 0.873512	training's binary_logloss: 0.208114	valid_1's auc: 0.774799	valid_1's binary_logloss: 0.240861

preds = clf.predict_proba(apps_all_test.drop(['SK_ID_CURR'], axis=1))[:, 1 ]
apps_all_test['TARGET'] = preds
apps_all_test[['SK_ID_CURR', 'TARGET']].to_csv('prev_baseline_02.csv', index=False)

차수	Private Score	Public Score	설명
1차	0.74088	0.74448	application 데이터 세트 기본 preprocessing
2차	0.75458	0.75882	application 데이터 세트 Feature Engineering
3차	0.76396	0.77579	previous 데이터 세트 Feature Engineering
4차	0.76420	0.77583	previous 데이터 세트 Feature Engineering 함수화

Source: inflearn / 캐글 Advanced 머신러닝 실전 박치기