Deeper Conversational AI - Part 3

Deeper Conversational AI

1. Reinforcement Learning / Self-Chat ⇒ Human In the Loop
   hard to learn directly from human interaction

   • learning a reward signal directly from human responses and transforming into meaningful update ⇒ but only toy datasets

2. Few-Shot / Zero-Shot Learning ⇒ Better Strategy
   collecting dataets is hard
   less data-intensive is crucial

   how do we learn to converse in new domain by learning from another domain

   • Zero-Shot learning
     no train data for this domain⇒need to learn from another domain and adapt to this
   • Few-shots learning
     • use pre-trained language model and fine-tune with few examples ex) ToD-BERT, SC-GPT
       
     • apply Meta-Learning solve new learning tasks using only a small number of training examples Model Agnostic Meata-Learning
       
     • without persona description, just by using previous dialogue by the same user

3. Lifelong Learning ⇒ User experience
   relatively unexplored area

   • extract attributes from the previous conversatoin and being able to remeber it and use it
   • add new domain or dialogue skills without retraining all data
     

4. Mitigating Inappropriate Response ⇒ On the Model

   • misleading response
   • toxic response
   • gender bias response

   build classifiers to filter out inappropritate examples(preprocessing)
   block n-gram from sensitive word list(decoding)

5. Multimodal ⇒ still challenge
   talk & able to see
   conversations grounded on images, VR environment

6. Evaluation ⇒ Better Automatic evalution
   N-gram based ⇒ fail to capture semantic meaning
   turn-level ⇒ fail to capture repetition, consistency between turns
   perplexity vs human evaluation
   

7. Shared Tasks&Datasets ⇒ need more