[Paper Review] Transformer for NLP

Authors : Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhinsubject : neural inf

GPT1은 transformer의 decode 부분의 Masked multi-head self-attention만을 사용함.unlabeld corpus는 labeled corpus보다 훨씬 많으니까, unlabeld corpus(대량 언어 데이터)에 pretrainin