[Stanford CS229] Lecture4 - Perceptron & Generalized Linear Model

1. Generalized Linear Models

이번 장에서는 베르누이 분포, 정규 분포 등과 같은 Generalized Linear Model(GLM)에 대해서 알아보고자 한다.

2. Exponential Family

n is called the natural parameter(also called the canonical parameter) of the distribution : T(y) is the sufficient statistic, and a(n) is the log partition function.

위와 같은 분포에 대해 parameter만 바꾸어 주면 베르누이 분포를 얻을 수 있다.

Let's now move on to consider the Gaussian distribution. Recall that, when deriving linear regression, the value of o^2 had no effect on our final choice of theta and h(x). Thus we can choose an arbitrary value for o^2 without changing anything. To simplify the derivation below, Let's set o^2 = 1.

Thus we see that the Gaussian is in the exponential family, with

위의 베르누이, 가우시안, 정규 분포 뿐 만 아니라 exponential family에는 multinomial, Poisson, Gamma, Beta, Dirichlet 분포가 포함된다.

3. Constructing GLMs

In this Section we will describe a method for constructing GLM models for problems.
1. y|x;theta ~ ExponentialFamily(n). I.e., given x and theta, the distribution of y follows some exponential family distribution, with parameter n.
2. Given x, our goal is to predict the expected value of T(y) given x. In most of our exampes, we will have T(y) = y, so this means we would like the prediction h(x) output by our learned hypothesis h to satisfy h(x) = E[y|x].
3. The natural parameter n and the inputs x are related linearly

위와 같은 방식으로 GLM의 형태로 바꿀 수 있을 것이다.

강의 링크 : Stanford CSS229 Youtube