Exemplar-FreeSOLO to enhance unsupervised instance segmentation by exploiting a limited number of unannotated and unsegmented exemplars
New exemplar embedding contrastive module is designed to enhance the discriminative capability of the segmentation model
FreeSOLO: hardly overcome the detrimental effects of the considerable noise in pseudo labels without any guidance

Unsupervised segmentation models are influenced by the noisy pseudo-labels

Unsupervised nature makes it difficult to learn discriminative information

Same type of target tends to lead to fragmentation problem

To solve the problem, exemplar learning mechanism developed

Guide of Top-down architecture and enhance discriminative capability

Exemplar-FreeSOLO for unsupervised instance segmentation

FreeSOLO - Pseudo-label noise, not optimized discriminate same objects

Top of Free SOLO, EKA(Exemplar Knowledge Abstraction) and EEC(Exemplar Embedding Contrastive) Module

Embedding Backbone, Segmentation Head, EEC Module

Embedding features extracted, fed into two branches: segmentation head, EEC



EKA

1. Each selected exemplar image Xu

2. Unsupervised object segmentation method(Grabcut 등)
   Extract the coarse foreground mask then crop out foreground

3. Deploy a trained FreeSOLO to extract a refined object mask from the exemplar object region and apply: Xe
   Treated as representatives of pivots for the corresponding hidden categories

EEC

Leveraging the exemplar object pool produced by the EKA module, EEC module exploits the exemplar objects as guidance knowledge to construct contrastive embedding

EEC takes the feature embeddings I of the unlabeled images and the feature embedding pool

Each unlabeled image, compute a mask-guided feature embedding

L2-normalization and up-sampling of Q

Prediction Mask M as FreeSOLO

Mask-guided feature embedding I by multiplying the prediction mask with a transformed feature I

Loss

Alpha보다 크면 positive
Beta보다 작으면n egative로 간주
Can determine positive and negative samples in an unsupervised manner
Maximize similarities between the positive pairs(Contrastive Loss)

Loss Function

Mask segmentation loss, Category Loss, exemplar embedding Loss
Trade-off Hyperparameter

Mask loss is Constrain the predicted mask of the segmentation head

Categoey loss is defined for foreground/background binary classification and semantic embedding learning

FreeSOLO에서 Q, K를 이용하여 구하는 부분은 동일하나, 이것을 이용하여 Exemplar를 생성하고 Contrastive Learning을 통해 Loss를 생성해내는 부분이 흥미로웠다.

구현 시에 이 부분의 계산이 구현되지 않는 경우가 있었는데 어떤 문제인지 알아보는 과정이 선행되어야 할 것 같다.