CNN

CNN

Naive Conv

7중 for문 연산

for batch
	for out_h
    	for out_w
        	for out_c
            	for kernel_c
                	for kernel_h
                    	for kernel_w

IM2COL

nD data $\rarr$ 2D data
GEMM : General Matrix to Matrix Multiplication
np.matmul()로 한 번에 연산

Torch

Pooling

Max pooling

Average pooling

FCN

Activation

sigmoid

$\sigma(x)=\displaystyle\frac{1}{1+e^{-x}}$

tanh

$\tanh(x)=\displaystyle\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}$

ReLU

$(x)^+=\max(0,x)$

LeakyReLU

$LeakyReLU=\begin{cases} x &\text{if } x \geq 0 \\ \alpha x &\text{if } otherwise \end{cases}$
$\alpha=0.02$

Shallow CNN