Lec-03 2 How to minimize cost

remind

• Hypothesis => $H(x)=Wx\,(+\,b)$
• Cost Function => $\text{cost}(W)=\frac{1}{m}\sum_{i=1}^{m}\big(H(x_i)-y_i\big)^2$

cost function in python

import numpy as np

X = np.array([1, 2, 3])
Y = np.array([1, 2, 3])

def cost_func(W, X, Y):
    c = 0
    for i in range(len(X)):
        c += (W * X[i] - Y[i]) ** 2
    return c / len(X)

for feed_W in np.linspace(-3, 5, num=15):
    curr_cost = cost_func(feed_W, X, Y)
    print("W: {:6.3f}, cost: {:10.5f}".format(feed_W, curr_cost))

cost function in tensorflow

import tensorflow as tf
import numpy as np

X = np.array([1, 2, 3], dtype=np.float32)
Y = np.array([1, 2, 3], dtype=np.float32)

def cost_func(W, X, Y):
    hypothesis = W * X
    return tf.reduce_mean(tf.square(hypothesis - Y))

W_values = tf.linspace(-3.0, 5.0, num=15)
cost_values = []
for feed_W in W_values:
    curr_cost = cost_func(feed_W, X, Y)
    cost_values.append(curr_cost)
    print("W: {:6.3f}, cost: {:10.5f}".format(feed_W.numpy(), curr_cost.numpy()))

Gradient descent 변환

• $\text{cost}(W)=\frac{1}{m}\sum_{i=1}^{m}\big(Wx_i-y_i\big)^2$
• $\Rightarrow\ W\ \leftarrow\ W\ -\ a\cdot\frac{1}{m}\sum_{i=1}^{m}2\big(Wx_i-y_i\big)x_i$

# 주의: 아래 구현은 cost를 (1/2m)로 두었을 때의 기울기에 해당합니다.
alpha = 0.01
# gradient ≈ mean((W*X - Y) * X)  ← (1/2m) 정의와 일치
gradient = tf.reduce_mean((W * X - Y) * X)
descent = W - alpha * gradient
W.assign(descent)

출처: 모두를 위한 딥러닝 강좌 2
https://www.youtube.com/watch?v=7eldOrjQVi0&list=PLQ28Nx3M4Jrguyuwg4xe9d9t2XE639e5C