S3-Rec -리뷰

S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization (CIKM, 2020)

논문

• https://dl.acm.org/doi/abs/10.1145/3340531.3411954

코드

• (논문 저자 코드) https://github.com/aHuiWang/CIKM2020-S3Rec
• (RecBole library) https://github.com/RUCAIBox/RecBole/blob/master/recbole/model/sequential_recommender/s3rec.py

참고자료

• Mutual Information Maximization 관련
  • https://youtu.be/tVXCkAaiY58
  • https://velog.io/@tobigs16gm/Self-Supervised-Learning#3-1-2-maximize-mutual-informationmi

문제의식

• researchers have incorporated rich contextual information (such as item attributes) to neural sequential recommenders
• Existing neural sequential recommendation models usually rely on the item prediction loss
  1. be prone to suffer from data sparsity problem.
  2. association between data (sequence) has not been well captured

아이디어

• utilize the intrinsic data correlation
  • learn the correlations among item, attribute and sequence
  • utilizing the mutual information maximization (MIM)
    • MIM : capture correlation between different types of data
      • Self-supervised learning 방법론

모델 구조

• Propose Self-Supervised learning for Sequential Recommendation
  based on the self-attentive recommender architecture

0. item embedding matrix & attribute embedding matrix :
   project the high-dimensional one-hot representation of an item or attribute
   to low-dimensional dense representations

1. pretrain the sequential recommender with self-supervised signals

   • devise four self-supervised optimization objectives
     • bidirectional Transformer architectures
   • Modeling Item-Attribute Correlation
     • item $i$ and the attribute set $A_i$ → maximizes the mutual information
   • Modeling Sequence-Item Correlation
     • predict the masked items from the original sequence
       based on the surrounding context in both directions. (like BERT)
   • Modeling Sequence-Attribute Correlation
     • recover the attributes of a masked item based on surrounding contexts.
   • Modeling Sequence-Segment Correlation
     • Unlike word sequence, a single item may not be highly related to surrounding contexts
       : an item segment reflects more clear, stable user preference
     • extend the Cloze strategy from a single item to item subsequence :
       the state representations of the last position in a sequence.

2. fine-tune the model parameters according to the recommendation task

   • utilize the learned parameters from the pretrained stage
     to initialize the parameters of the unidirectional Transformer

프리트레인 효과성

(개인의견) 한계점

• 프리트레인 과정의 효율성