[Paper Review] Fundamentals in Deep Learning (Norm, TTA, etc.)

Author : Sergey Ioffe, Christian SzegedyConference : ICML 2015(International Conference on Machine Learning)

Zhu, Jiachen, et al. "Transformers without normalization." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025. 위 논문을 읽으면서, no

(background)modern neural networks에서, Normalization layers는 어디서든 사용되고(unbiquitous) 필수적으로 고려되어 왔다.(이 논문의 핵심)이 연구에서는 매우 간단한 technique을 사용하여, Transformer

https://arxiv.org/abs/2102.06171(배경)BN은 image classification model에서 key componenet이지만,the batch size and interactions between examples에 대한 depen

https://arxiv.org/abs/2003.00152DL techniques은 training affine transformations of features를 학습하는데 의존한다.이러한 것들 중 가장 prominent한 것은 feature normaliz