2.1 결정이론의 요소들
관측치 모형
모형: ( X , Y , ( P θ ) θ ∈ Θ ) (\mathcal{X}, \mathcal{Y}, (\mathbf{P}_\theta)_{\theta \in \Theta}) ( X , Y , ( P θ ) θ ∈ Θ ) \quad\quad → \rightarrow → X \mathcal{X} X Y \quad \mathcal{Y} Y X \mathcal{X} X σ − f i e l d \sigma-field σ − f i e l d ( P θ ) θ ∈ Θ \quad (\mathbf{P}_\theta)_{\theta \in \Theta} ( P θ ) θ ∈ Θ Y \mathcal{Y} Y X ∼ P θ \mathbf{X} \sim \mathcal{P}_\theta X ∼ P θ X = ( X 1 , … , X n ) \mathbf{X}=(X_1,\ldots,X_n) X = ( X 1 , … , X n ) P θ = Π i = 1 n f θ \quad \mathcal{P}_\theta=\Pi_{i=1}^nf_\theta P θ = Π i = 1 n f θ
행동 공간 (Action Space)
A \mathcal{A} A
Example X ∼ P θ \quad\mathbf{X} \sim \mathcal{P}_\theta X ∼ P θ θ \theta θ A = Θ \quad\mathcal{A}=\Theta A = Θ
(ii) 검정H 0 : θ ∈ Θ 0 \quad H_0:\theta \in \Theta_0\quad H 0 : θ ∈ Θ 0 H 1 : θ ∈ Θ 1 \quad H_1:\theta \in \Theta_1 H 1 : θ ∈ Θ 1 A = { H 0 , H 1 } \quad A=\{H_0,H_1\} A = { H 0 , H 1 }
(iii) 순위 ranking\quad A = { ( a , b , c ) , ( a , c , b ) , ⋯ } \quad \mathcal{A}=\{(a,b,c), (a,c,b), \cdots\} A = { ( a , b , c ) , ( a , c , b ) , ⋯ }
손실함수(Loss function)
L : Θ × A → R L:\Theta \times \mathcal{A}\rightarrow\mathbf{R}\quad L : Θ × A → R ( θ , a ) ⟼ L ( θ , a ) \quad (\theta, a)\longmapsto L(\theta, a) ( θ , a ) ⟼ L ( θ , a ) L ( θ , a ) \quad L(\theta, a) L ( θ , a ) θ \ \theta θ a a a
Example. 추정 L ( θ , a ) = ( θ − a ) 2 L(\theta, a)=(\theta-a)^2 L ( θ , a ) = ( θ − a ) 2 L ( θ , a ) = ∣ θ − a ∣ L(\theta, a)=|\theta-a| L ( θ , a ) = ∣ θ − a ∣ L ( θ , a ) = ∥ θ − a ∥ 2 L(\theta, a)=\|\theta-a\|^2 L ( θ , a ) = ∥ θ − a ∥ 2 θ , a ∈ R k \theta, a\in\mathbf{R}^k θ , a ∈ R k L ( f , a ) = ∫ ( f ( x ) − a ( x ) ) 2 d P ( x ) L(f, a)=\int(f(x)-a(x))^2dP(x) L ( f , a ) = ∫ ( f ( x ) − a ( x ) ) 2 d P ( x ) f , a f, a f , a L ( Σ , A ) = ∥ Σ − a ∥ F 2 L(\Sigma, A)=\|\Sigma-a\|_F^2\quad L ( Σ , A ) = ∥ Σ − a ∥ F 2
Example. 가설검정 H 0 : θ ∈ Θ 0 H_0:\theta\in\Theta_0\quad H 0 : θ ∈ Θ 0 H 1 : θ ∈ Θ 1 \quad H_1:\theta\in\Theta_1 H 1 : θ ∈ Θ 1 A = { H 0 , H 1 } , { 0 , 1 } \mathcal{A}=\{H_0, H_1\}, \{0, 1\} A = { H 0 , H 1 } , { 0 , 1 } L ( θ , a ) = 0 L(\theta, a)=0 L ( θ , a ) = 0 L ( θ , a ) = { 0 , θ ∈ a 1 , θ ∉ a L(\theta, a) = \begin{cases} 0 &,\; \theta \in a \\ 1 &,\; \theta \notin a \end{cases} L ( θ , a ) = { 0 1 , θ ∈ a , θ ∈ / a
결정규칙 (Decision rule)
δ : X ⟼ A \delta:\mathcal{X}\longmapsto\mathcal{A} δ : X ⟼ A x ⟼ δ ( x ) \quad\; x \longmapsto \delta(x) x ⟼ δ ( x ) → \rightarrow → x x x
Example. δ ( x ) = x ˉ \delta(x)=\bar{x} δ ( x ) = x ˉ δ ( x ) = 1 n Σ ( x i − x ˉ ) 2 \delta(x)=\frac{1}{n}\Sigma(x_i-\bar{x})^2 δ ( x ) = n 1 Σ ( x i − x ˉ ) 2
(ii) 검정δ ( x ) = { H 0 , n σ ( x ˉ − σ 0 ) > z α H 1 , n σ ( x ˉ − σ 0 ) ≤ z α \delta(x) = \begin{cases} H_0 &,\; \frac{\sqrt{n}}{\sigma}(\bar{x}-\sigma_0)>z_\alpha \\ H_1 &,\; \frac{\sqrt{n}}{\sigma}(\bar{x}-\sigma_0)\leq z_\alpha \end{cases} δ ( x ) = { H 0 H 1 , σ n ( x ˉ − σ 0 ) > z α , σ n ( x ˉ − σ 0 ) ≤ z α
랜덤화된 결정규칙 (Randomized decision rule)
δ : X ⟼ P ( A ) \delta:\mathcal{X}\longmapsto \mathbf{P(\mathcal{A})} δ : X ⟼ P ( A ) A \; \mathcal{A} A x ⟼ δ ( x , ⋅ ) \quad\;\; x \longmapsto \delta(x,\cdot) x ⟼ δ ( x , ⋅ ) A \; \mathcal{A} A x x x a ∼ δ ( x , ⋅ ) a\sim\delta(x, \cdot) a ∼ δ ( x , ⋅ ) a a a δ ; X × ( A \delta; \mathcal{X}\times(\mathcal{A} δ ; X × ( A σ − f i e l d ) → [ 0 , 1 ] \sigma-field) \rightarrow [0,1] σ − f i e l d ) → [ 0 , 1 ] ( x , A ) ⟼ δ ( x , A ) ∈ [ 0 , 1 ] \quad\quad\quad\quad\quad\quad\quad(x,A)\longmapsto\delta(x,A)\in[0,1] ( x , A ) ⟼ δ ( x , A ) ∈ [ 0 , 1 ]
손실함수의 계산L ( θ , δ ( x , ⋅ ) ) = ∫ L ( θ , a ) δ ( x , d a ) L(\theta,\delta(x,\cdot))=\int L(\theta, a)\delta(x,da) L ( θ , δ ( x , ⋅ ) ) = ∫ L ( θ , a ) δ ( x , d a )
위험함수 (Risk function)
\quad\quad
θ \theta θ R ( θ , δ ) = ∫ L ( θ , δ ( x ) ) P θ ( d x ) = E θ [ L ( θ , δ ( x ) ) ] \begin{aligned}R(\theta,\delta)&=\int L(\theta, \delta(x))\mathcal{P_\theta(dx)}\\ &=\mathbf{E}_\theta[ L(\theta,\delta(x))] \end{aligned} R ( θ , δ ) = ∫ L ( θ , δ ( x ) ) P θ ( d x ) = E θ [ L ( θ , δ ( x ) ) ]
Example. 추정 ν ( θ ) \nu(\theta) ν ( θ ) δ ( x ) = ν ^ ( x ) \delta(x)=\hat{\nu}(x) δ ( x ) = ν ^ ( x ) ν \nu ν L ( θ , a ) = ( ν ( θ ) − a ) 2 L(\theta,a)=(\nu(\theta)-a)^2 L ( θ , a ) = ( ν ( θ ) − a ) 2 R ( θ , δ ) = E θ [ ν ( θ ) − δ ( x ) ] 2 = E θ [ ν ( θ ) − E θ δ ( x ) + E θ δ ( x ) − δ ( x ) ] 2 = E θ [ ν ( θ ) − E θ δ ( x ) ] 2 + E θ [ δ ( x ) − E θ δ ( x ) ] 2 = V a r ( δ ( x ) ) + { ν ( θ ) − E θ δ ( x ) } 2 = V a r θ δ ( x ) + B i a s 2 ( δ ( x ) ) ■ = M S E ( θ , δ ) \begin{aligned} R(\theta,\delta)=\mathbf{E}_\theta[\nu(\theta)-\delta(x)]^2&=\mathbf{E}_\theta[\nu(\theta)-\mathbf{E}_\theta\delta(x)+\mathbf{E}_\theta\delta(x)-\delta(x)]^2\\ &=\mathbf{E}_\theta[\nu(\theta)-\mathbf{E}_\theta\delta(x)]^2+\mathbf{E}_\theta[\delta(x)-\mathbf{E}_\theta\delta(x)]^2\\ &=Var(\delta(x))+\{\nu(\theta)-\mathbf{E}_\theta\delta(x)\}^2\\ &=Var_\theta\delta(x)+Bias^2(\delta(x))_\blacksquare \\ &= MSE(\theta,\delta) \end{aligned} R ( θ , δ ) = E θ [ ν ( θ ) − δ ( x ) ] 2 = E θ [ ν ( θ ) − E θ δ ( x ) + E θ δ ( x ) − δ ( x ) ] 2 = E θ [ ν ( θ ) − E θ δ ( x ) ] 2 + E θ [ δ ( x ) − E θ δ ( x ) ] 2 = V a r ( δ ( x ) ) + { ν ( θ ) − E θ δ ( x ) } 2 = V a r θ δ ( x ) + B i a s 2 ( δ ( x ) ) ■ = M S E ( θ , δ )
Example. 두 추정량의 비교 μ : \mu: μ : μ 0 : \mu_0: μ 0 :
X 1 , ⋯ , X n ∼ N ( μ , σ 2 ) : X_1,\cdots,X_n\sim N(\mu, \sigma^2): X 1 , ⋯ , X n ∼ N ( μ , σ 2 ) : { δ 1 = 0.2 μ 0 + 0.8 X ˉ δ 2 = X ˉ \begin{cases} \delta_1=0.2\mu_0+0.8\bar{X}&\\ \delta_2=\bar{X}& \end{cases} { δ 1 = 0 . 2 μ 0 + 0 . 8 X ˉ δ 2 = X ˉ
{ M S E μ ( δ 1 ) = R ( μ , δ 1 ) = 0.64 σ 2 n + 0.04 ( μ 0 − μ ) 2 M S E μ ( δ 2 ) = V a r μ ( X ˉ ) = σ 2 n = R ( μ , δ 2 ) \begin{cases} MSE_\mu(\delta_1)=R(\mu,\delta_1)=0.64\frac{\sigma^2}{n}+0.04(\mu_0-\mu)^2&\\ MSE_\mu(\delta_2)=Var_\mu(\bar{X})=\frac{\sigma^2}{n}=R(\mu,\delta_2)& \end{cases} { M S E μ ( δ 1 ) = R ( μ , δ 1 ) = 0 . 6 4 n σ 2 + 0 . 0 4 ( μ 0 − μ ) 2 M S E μ ( δ 2 ) = V a r μ ( X ˉ ) = n σ 2 = R ( μ , δ 2 )
δ 1 , δ 2 \delta_1,\delta_2 δ 1 , δ 2 R ( μ , δ ) R(\mu,\delta) R ( μ , δ )
Method 1) R ( θ , δ ) R(\theta,\delta) R ( θ , δ )
Method 2) 비교하는 δ \delta δ
Example. 검정의 예
행동\참 H0 H1 H0 0 2종 오류(1) H1 1종 오류(1) 0
검정함수 (Test function)
랜덤화된 결정규칙δ : X → [ 0 , 1 ] \delta:\mathcal{X}\rightarrow [0,1] δ : X → [ 0 , 1 ] x ⟼ δ ( x ) = H 0 \quad\;\; x \longmapsto \delta(x)=H_0 x ⟼ δ ( x ) = H 0
기각역 (Rejection region)
δ ( x ) = I ( x ∈ C ) \delta(x)=I(x\in C) δ ( x ) = I ( x ∈ C ) C : \quad C: C :
위험함수
δ ( x ) = I ( x ∈ C ) \delta(x)=I(x\in C) δ ( x ) = I ( x ∈ C )
R ( θ , δ ) = E θ L ( θ , δ ( X ) ) = { E θ δ ( x ) , if θ ∈ Θ 0 E θ ( 1 − δ ( x ) ) , if θ ∈ Θ 1 = { P θ ( X ∈ C ) , if θ ∈ Θ 0 P θ ( X ∉ C ) , if θ ∈ Θ 1 = { H 0 가 참일 때 H 1 을 선택하는 확률 , if θ ∈ Θ 0 ; 1종오류 확률 H 1 이 참일 때 H 0 를 선택하는 확률 , if θ ∈ Θ 1 ; 2종오류 확률 \begin{aligned} R(\theta,\delta) &= E_{\theta} L(\theta,\delta(X)) \\ &= \begin{cases} \mathbf{E}_\theta\delta(x), & \text{if } \theta \in \Theta_0 \\ \mathbf{E}_\theta(1-\delta(x)), & \text{if } \theta \in \Theta_1 \end{cases}\\ &= \begin{cases} P_{\theta}(X \in C), & \text{if } \theta \in \Theta_0 \\ P_{\theta}(X \notin C), & \text{if } \theta \in \Theta_1 \end{cases}\\ &= \begin{cases} H_0 \text{가 참일 때 }H_1 \text{을 선택하는 확률}, & \text{if } \theta \in \Theta_0 \quad; \text{1종오류 확률}\\ H_1 \text{이 참일 때 }H_0 \text{를 선택하는 확률}, & \text{if } \theta \in \Theta_1\quad; \text{2종오류 확률} \end{cases} \end{aligned} R ( θ , δ ) = E θ L ( θ , δ ( X ) ) = { E θ δ ( x ) , E θ ( 1 − δ ( x ) ) , if θ ∈ Θ 0 if θ ∈ Θ 1 = { P θ ( X ∈ C ) , P θ ( X ∈ / C ) , if θ ∈ Θ 0 if θ ∈ Θ 1 = { H 0 가 참일 때 H 1 을 선택하는 확률 , H 1 이 참일 때 H 0 를 선택하는 확률 , if θ ∈ Θ 0 ; 1 종오류 확률 if θ ∈ Θ 1 ; 2 종오류 확률
2.2 결정규칙의 비교
δ \delta δ δ ′ \delta' δ ′ ⟺ ( i ) R ( θ , δ ) ≤ R ( θ , δ ′ ) , ∀ θ ∈ Θ ( i i ) R ( θ , δ ) < R ( θ , δ ′ ) 인 θ ∈ Θ 존재 . \begin{aligned} \iff &(i)\; R(\theta,\delta) \leq R(\theta,\delta'), \; \forall\theta \in \Theta\\ &(ii)\; R(\theta,\delta)<R(\theta,\delta')인\;\; \theta\in\Theta\;\; 존재. \end{aligned} ⟺ ( i ) R ( θ , δ ) ≤ R ( θ , δ ′ ) , ∀ θ ∈ Θ ( i i ) R ( θ , δ ) < R ( θ , δ ′ ) 인 θ ∈ Θ 존 재 .
허용가능(admissible), 허용불가능(inadmissible)
δ : \delta: δ : ⟺ \iff ⟺ δ \delta δ δ ′ \delta' δ ′ \quad\quad δ : \delta: δ : ⟺ \iff ⟺ δ \delta δ \quad\quad
베이즈 규칙 (Bayes rule)
(i) 베이즈 위험(Bayes risk)
r ( π , δ ) = ∫ Θ R ( θ , δ ) π ( d θ ) \quad r(\pi,\delta)=\int_\Theta R(\theta,\delta)\pi(d\theta) r ( π , δ ) = ∫ Θ R ( θ , δ ) π ( d θ ) π : Θ \quad \pi:\Theta π : Θ
(ii) 사전분포 π \pi π
δ B : = arg min δ r ( π , δ ) \quad \delta^B:= \underset{\delta}{\arg\min}\ r(\pi,\delta) δ B : = δ arg min r ( π , δ )
최소최대규칙 (Minimax rule)
δ m : = arg min δ max θ ∈ Θ r ( π , δ ) \delta^m:= \underset{\delta}{\arg\min}\;\underset{\theta \in\Theta}{\max}\; r(\pi,\delta) δ m : = δ arg min θ ∈ Θ max r ( π , δ ) ⇒ \quad\quad\quad \Rightarrow\quad ⇒
완비모임정리 (Complete class theorem)
통계모형 ( X , Y , ( P θ ) θ ∈ Θ ) (\mathcal{X},\mathcal{Y},(\mathbf{P}_\theta)_{\theta \in\Theta}) ( X , Y , ( P θ ) θ ∈ Θ )
(a) 확률의 정규성 (regularity)
P : Y × Θ → [ 0 , 1 ] \mathbf{P}:\mathcal{Y}\times\Theta \rightarrow [0,1] P : Y × Θ → [ 0 , 1 ] A ∈ Y A \in \mathcal{Y} A ∈ Y θ ⟼ P θ ( A ) \theta \longmapsto \mathbf{P}_\theta(A) θ ⟼ P θ ( A ) \quad\ \; θ ∈ Θ \theta \in \Theta θ ∈ Θ θ \theta θ P θ \mathbf{P}_\theta P θ Y \mathcal{Y} Y
(b) 모형의 연속성
모수공간 Θ \Theta Θ θ ⟼ P θ \theta \longmapsto \mathbf{P}_\theta θ ⟼ P θ L 1 − n o r m L_1-norm L 1 − n o r m ∥ P θ 1 − P θ 2 ∥ L 1 = ∫ ∣ f θ 1 ( x ) − f θ 2 ( x ) ∣ d μ ( x ) \|\mathbf{P}_{\theta_1}-\mathbf{P}_{\theta_2}\|_{L_1}=\int|f_{\theta_1}(x)-f_{\theta_2}(x)|d\mu(x) ∥ P θ 1 − P θ 2 ∥ L 1 = ∫ ∣ f θ 1 ( x ) − f θ 2 ( x ) ∣ d μ ( x ) A : \mathcal{A}: A : P ( A ) : A \quad \mathcal{P}(\mathcal{A}):\mathcal{A} P ( A ) : A
완비모임정리
모수공간 Θ : \Theta: Θ : A : \mathcal{A}: A : L ( θ , a ) L(\theta,a) L ( θ , a ) ( θ , a ) (\theta,a) ( θ , a ) ⇒ \Rightarrow ⇒ δ ∈ P ( A ) \delta\in\mathcal{P}(A) δ ∈ P ( A ) \quad δ k ⇝ δ 0 \delta_k\rightsquigarrow\delta_0 δ k ⇝ δ 0 \quad R ( θ , δ 0 ) ≤ R ( θ , δ ) , ∀ θ ∈ Θ R(\theta,\delta_0)\leq R(\theta,\delta),\quad \forall\theta\in\Theta R ( θ , δ 0 ) ≤ R ( θ , δ ) , ∀ θ ∈ Θ \quad δ 0 ∈ P ( A ) \delta_0\in\mathcal{P}(A) δ 0 ∈ P ( A ) π k \pi_k π k δ k \quad\delta_k δ k π k \pi_k π k
cf) 완비모임정리는 모든 결정규칙 δ \delta δ δ \delta δ
