1. Noisy Student Training(NST)
[Xie et al.,2020] Self-training with Noisy Student improves ImageNet classification
[Li et al.,2017] Large-Scale Domain Adaptation via Teacher-Student Learning, Google
[Hwang et al.,2022]Large-Scale ASR Domain Adaptation Using Self- And Semi-Supervised Learning
[Zhang et al.,2020] Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition
2. Domain Adversarial Training
[Ganin et al.,2016] Domain-Adversarial Training of Neural Networks
[Li et al.,2021] Accent-Robust Automatic Speech Recognition Using Supervised and Unsupervised Wav2Vec Embeddings, Facebook AI
Objective
(i) discriminativeness
(ii) domain-invariance
How?
(i) Minimize the loss of the label classifier
(ii) Maximize the loss of the domain classifier = 우리가 보고있는 sample의 feature representation이 source domain에서 왔는지 target domain에서 왔는지 구별 못하게 domain discriminator를 약화하는 방향(gradient reverse @backprop)으로 학습시키겠다는 것.
Gradient Reversal Layer(GRL)
