df.info() # 각 컬럼별로 결측치가 있는지, 데이터 타입은 무엇인지 등

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 200 entries, 0 to 199
Data columns (total 4 columns):
 #   Column      Non-Null Count  Dtype
---  ------      --------------  -----
 0   id          200 non-null    int64
 1   num_awards  200 non-null    int64
 2   prog        200 non-null    int64
 3   math        200 non-null    int64
dtypes: int64(4)
memory usage: 6.4 KB

1. 종속변수가 count variable 인가

df['num_awards'].unique()

array([0, 1, 3, 2, 5, 4, 6])

2. 종속변수의 평균과 분산은?

model_nb = smf.negativebinomial('num_awards ~ math', data = df).fit() #음이항 회귀모델을 돌려보자
print(model_nb.summary())

Optimization terminated successfully.
         Current function value: 0.940841
         Iterations: 20
         Function evaluations: 26
         Gradient evaluations: 26
                     NegativeBinomial Regression Results                      
==============================================================================
Dep. Variable:             num_awards   No. Observations:                  200
Model:               NegativeBinomial   Df Residuals:                      198
Method:                           MLE   Df Model:                            1
Date:                Mon, 12 Jun 2023   Pseudo R-squ.:                  0.1343
Time:                        04:07:07   Log-Likelihood:                -188.17
converged:                       True   LL-Null:                       -217.37
Covariance Type:            nonrobust   LLR p-value:                 2.132e-14
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
Intercept     -5.4458      0.671     -8.121      0.000      -6.760      -4.132
math           0.0881      0.011      7.850      0.000       0.066       0.110
alpha          0.2694      0.177      1.518      0.129      -0.078       0.617
==============================================================================

X와 y의 관계에 대해서 대략적으로 그림으로 살펴보기

- 수학성적과 상받은 횟수에 대한 관계(독립변수 1개만 고려)

plt.figure(figsize = (5, 3))
plt.plot(df['math'], df['num_awards'], '*', alpha = 0.3)
plt.title('Math score and Number of received Awards')
plt.xlabel('Math Score')
plt.ylabel('Awards')
plt.show()

푸아송 회귀모델

model_poisson = smf.poisson('num_awards ~ math', data = df).fit()
print(model_poisson.summary())

Optimization terminated successfully.
         Current function value: 0.950190
         Iterations 6
                          Poisson Regression Results                          
==============================================================================
Dep. Variable:             num_awards   No. Observations:                  200
Model:                        Poisson   Df Residuals:                      198
Method:                           MLE   Df Model:                            1
Date:                Mon, 12 Jun 2023   Pseudo R-squ.:                  0.1804
Time:                        04:07:15   Log-Likelihood:                -190.04
converged:                       True   LL-Null:                       -231.86
Covariance Type:            nonrobust   LLR p-value:                 5.903e-20
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
Intercept     -5.3335      0.591     -9.021      0.000      -6.492      -4.175
math           0.0862      0.010      8.902      0.000       0.067       0.105
==============================================================================

푸아송 회귀모델

$\lambda = e^{\beta_{0} + \beta_{1}x_{1}}$

$\lambda = e^{-5.3335 + 0.0862x_{1}}$

푸아송 확률함수

$P(Y = y) = \frac{e^{- \lambda}\lambda^{Y}}{Y!}$ where ($\lambda = e^{-5.3335 + 0.0862x_{1}}$)

Incidence rate ratio (IRR)

• 독립변수가 1단위 증가할 때 단위시간당 평균발생횟수의 비율이 어떻게 변하는가?

IRR = np.exp(0.0862)
IRR #독립변수가 1단위 증가할 때 단위시간당 평균발생횟수가 어떻게 변하는가?

1.0900243113683694

수학성적이 1점 증가할때 1학기 평균 상받는 횟수는 1.09배 만큼 증가한다.

실제 데이터와 구한 푸아송 회귀모델 그림그리기

X = df['math'].sort_values()
y = np.exp(-5.3335 + 0.0862 * X) #푸아송 회귀모델

plt.figure(figsize = (5, 3))
plt.plot(df['math'], df['num_awards'], 'go', alpha = 0.3, label = 'True data')
plt.plot(X, y, linewidth = 3, label = 'Poisson Regression Line')
plt.title('Math score and Number of received Awards')
plt.xlabel('Math Score')
plt.ylabel('Awards')
plt.legend()
plt.tight_layout()
plt.show()

포아송회귀모델과 음이항회귀모델의 비교

model_nb.params #음이항 모델의 회귀계수 뽑기

Intercept   -5.445762
math         0.088123
alpha        0.269401
dtype: float64

• 푸아송모델: $\lambda = e^{-5.3335 + 0.0862x_{1}}$
• 음이항모델: $\lambda = e^{-5.4458 + 0.0881x_{1}}$