분포의 모수가 상수가 아니라 모수도 어떤 분포를 따른다고 봄.
=> $f(x,\theta) = f(x|\theta)h(\theta)$

• Joint pdf of $(\underline{X},\theta)$

  $g(x_1,...,x_n,\theta) = \prod_i f(x_i|\theta)h(\theta)=L(\theta;\underline{x})h(\theta)$

• Marginal pdf of $\underline{X}$

  $g_X(\underline{x}) = \int g(\underline{x},\theta) d\theta$

• Posterior distriburion of $\theta$ given the sample

  $k(\theta|\underline{x}) = \frac{g(\underline{x},\theta)}{g_X(\underline{x})}$

ex1) $X_i|\theta \overset{iid}{\sim} Poisson(\theta), \theta \sim Gamma(\alpha,\beta) \space (\alpha,\beta:known)$

Bayesian point estimation

• $\delta(\underline{x})$: decision function
• $l(\theta,\delta(\underline{x}))$: loss function
  ex) $(\theta - \delta(\underline{x}))^2,|\theta - \delta(\underline{x})|,(\theta/\delta(\underline{x}) - 1)^2$
• Bayes estimator minimizes the posterior expected loss
  $E[l(\theta,\delta(\underline{x}))|\underline{X}=\underline{x}]=\int l(\theta,\delta(\underline{x}))k(\theta|\underline{x}) d\theta$

ex1) $X_i|\theta \overset{iid}{\sim} Poisson(\theta), \theta \sim Gamma(\alpha,\beta) \space (\alpha,\beta:known)$
$\theta| \underline{x} \sim Gamma(\sum_i x_i + \alpha, \frac{\beta}{n \beta + 1}) \\ When \space l(\theta, \delta(\underline{x})) = (\theta - \delta(\underline{x}))^2, \space \hat{\theta}=E(\theta | \underline{x}) \\ \hat{\theta}^B = E(\theta | \underline{x}) = \frac{\beta(\sum_i x_i + \alpha)}{n \beta + 1} = \frac{\sum_i x_i + \alpha}{n + \frac{1}{\beta}} = \frac{n}{n + \frac{1}{\beta}} \overline{x} + \frac{1/ \beta}{n + \frac{1}{\beta}} \alpha \beta$ : weighted sum of MLE & mean of prior distritbution

ex2) $X_i|\theta \overset{iid}{\sim} b(1,\theta),\theta \sim Beta(\alpha,\beta)(\alpha,\beta:known)$

✔︎ conjugate prior
: If the posterior pdfs is in the same family of distributions as the prior, the prior is called conjugate prior

ex)
① When $X \sim Poisson(\theta)$, Gamma is conjugate prior
② When $X \sim b(n,\theta)$, Beta is conjugate prior
③ When $X \sim N(\mu,1)$, Normal is conjugate prior