NLP
1.[NLP] mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs (EMNLP, 2021)
1 Introduction & Summary Challenge seq2seq 모델인 mT5는 cross-lingual task에서 괄목할만한 성능 보여줌 MT5가 이미 성능이 좋은데, 어떻게 하면 tranlation data 사용해서 MT5를 뛰어넘을 수 있을까
2.[NLP] Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation (ACL, 2022)
Summary Background: Neural Machine Translation (NMT) task에서는 training 데이터 수에 따라서 generaliation performance가 달라진다. 그래서 training data를 늘리기 위해 Augmen
3.[NLP] SimCSE: Simple Contrastive Learning of Sentence Embeddings (EMNLP, 2021)
한줄 요약: unlabeled or labeled data 모두에서 setence embedding 뽑을 수 있다? Paper: https://aclanthology.org/2021.emnlp-main.552/Code: https://github.co
4.[NLP] BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (ACL, 2020)
Paper: https://aclanthology.org/2020.acl-main.703/ > Code: > - Ko-BART : https://github.com/SKT-AI/KoBART
5.[NLP] Mutual Information Alleviates Hallucinations in Abstractive Summarization (EMNLP, 2022)
paper: https://aclanthology.org/2022.emnlp-main.399.pdfcode: https://github.com/VanderpoelLiam/CPMIBackground: hallucinate: 요약한게 source docu
6.[NLP] How to Adapt Your Pretrained Multilingual Model to 1600 Languages (ACL, 2021)
Goal: 1600가지 언어 데이터셋을 활용해서 기존의 PMM 모델들 performance 검증Challenge: 1) 소량 2) narrow domainResult: XLM-R good
7.[NLP #8] How Far are We from Robust Long Abstractive Summarization? (EMNLP, 2022)
goal: abstractive summarization system의 수행결과를 human annotation 결과랑 비교해서 long document에 대해 얼마나 잘했는지 평가해본다. result: 비교해보니까 ROUGE result에서는 굿이었음. 그러니까 re
8.[NLP] On Transferability of Prompt Tuning for Natural Language Processing (NAACL, 2022)
paper: https://aclanthology.org/2022.naacl-main.290/ > velog: https://velog.io/@zvezda/On-Transferability-of-Prompt-Tuning-for-Natural-Language-Proces
9.[NLP] Efficient Dialogue State Tracking by Selectively Overwriting Memory (ACL, 2020)
Dialogue systemopen-domain dialogue system : 대화주제 자유롭게task oriented dialogue (TOD) system: 특정 task 수행TODsingle domain TOD: 하나의 작업만 수행Multi domain TOD:
10.[NLP] GPT-2: Language Models are Unsupervised Multitask Learners
GPT-2 Review
11.[NLP] Story Generation - Genre-Controllable Story Generation via Supervised Contrastive Learning (WWW, 2022)
Challenge : Pretraine language model 등의 발전으로 controllable text genration이 각광받고 있다. 하지만 story-specific controllability를 잘하기 위해선 아직 부족하다!
12.[NLP] When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer (NAACL, 2022)
InversionPermutationTransliterationSyntaxBilingual ModelMonolingual Model
13.[NLP] Training language models to follow instructions with human feedback (NeurIPS, 2022)
paper: https://proceedings.neurips.cc/paper_files/paper/2022/file/b1efde53be364a73914f58805a001731-Paper-Conference.pdfBackground: LM 모델을 더 크게 만든