S3-Rec -리뷰

seonjin2·2023년 8월 22일

researchers have incorporated rich contextual information (such as item attributes) to neural sequential recommenders
Existing neural sequential recommendation models usually rely on the item prediction loss
1. be prone to suffer from data sparsity problem.
2. association between data (sequence) has not been well captured

utilize the intrinsic data correlation
- learn the correlations among item, attribute and sequence
- utilizing the mutual information maximization (MIM)
  - MIM : capture correlation between different types of data
    - Self-supervised learning 방법론

Propose Self-Supervised learning for Sequential Recommendation
based on the self-attentive recommender architecture

item embedding matrix & attribute embedding matrix :
project the high-dimensional one-hot representation of an item or attribute
to low-dimensional dense representations
pretrain the sequential recommender with self-supervised signals
- devise four self-supervised optimization objectives
  - bidirectional Transformer architectures
- Modeling Item-Attribute Correlation
  - item $i$ and the attribute set $A_i$ → maximizes the mutual information
- Modeling Sequence-Item Correlation
  - predict the masked items from the original sequence
    based on the surrounding context in both directions. (like BERT)
- Modeling Sequence-Attribute Correlation
  - recover the attributes of a masked item based on surrounding contexts.
- Modeling Sequence-Segment Correlation
  - Unlike word sequence, a single item may not be highly related to surrounding contexts
    : an item segment reflects more clear, stable user preference
  - extend the Cloze strategy from a single item to item subsequence :
    the state representations of the last position in a sequence.
fine-tune the model parameters according to the recommendation task
- utilize the learned parameters from the pretrained stage
  to initialize the parameters of the unidirectional Transformer