Learning Rate

learning rate : too low, just right, too high

learning rate scheduler 도입
일정한 rate vs decay ---> 줄였다 늘렸다 하는 것이 더 성능향상
step decay, cosine decay

Reference
6-4. Learning Rate Scheduler