Bayes theorem for classification in machine learning

Marginal probability $P(A)$ : 다른 변수들을 무시하고 부분 집합$X (A\in X)$ 내의 event $A$가 발생할 확률

Joint probability $P(A,B)$ : event $A,B$가 동시에 발생할 확률

Conditional probability $P(A|B)$: event $B$가 발생한 이후 event $A$가 발생할 확률

Properties

$P(A,B) = P(A|B)*P(B)$
$P(A,B) = P(B,A)$
$P(A|B) = \frac{P(A,B)}{P(B)}$
$P(A|B) \not= P(B|A)$

Bayes Theorem

• Joint Probability없이 Conditional probability를 계산하는 방법

$P(B|A) = \frac{P(A|B)P(B)}{P(A)}$

이러한 관점에서 특별히 아래와 같이 용어가 정립되어있다.
$P(A|B)$ : posterior probability
$P(B|A)$ : Likelihood
$P(A)$ : prior probability
$P(B)$ : Evidence

Naive Bayes classification

예측값을 결정짓는 feature들간의 독립을 가정한 조건부 확률모델이다.

$P(C_k|\vec{x}) = \frac{P(C_k)P(\vec{x}|C_k)}{p(\vec{x})}$

$P(\vec{x}) = Z$ : constant

${\displaystyle P(C_{k}\vert x_{1},\dots ,x_{n})={\frac {1}{Z}}P(C_{k})\prod _{i=1}^{n}P(x_{i}\vert C_{k})}$

${\displaystyle {\hat {y}}={\underset {k\in \{1,\dots ,K\}}{\operatorname {argmax} }}\ P(C_{k})\displaystyle \prod _{i=1}^{n}P(x_{i}\vert C_{k}).}$

Bayesian Belief Networks

https://machinelearningmastery.com/introduction-to-bayesian-belief-networks/

References

https://machinelearningmastery.com/bayes-theorem-for-machine-learning/
https://ko.wikipedia.org/wiki/%EB%82%98%EC%9D%B4%EB%B8%8C_%EB%B2%A0%EC%9D%B4%EC%A6%88_%EB%B6%84%EB%A5%98