Review] Multi-Similarity and Attention Guidance for Boosting Few-Shot Segmentation(MSANet)

Suho Park·2023년 1월 2일

cv_paper_reviews

목록 보기

14/22

The problem of traditonal supervised CNN is that it needs the number of well-annotated data, the balance of class distribution and sample representation. Moreover, it is not suitable for unseen class data.
Few-shot learning can be a good solution for such problems. The goal of FSS is to segment the target region of selected category in the query image.
There are many attempts to get more guidance from prototype vectors adopting different mechanism( PANet, PFENet, SG-One Net ... )
However, such approach( PANet, PFENet, SG-One Net ... ) can lose fine information of an image due to masked average pooling operation. Therefore, this paper tried to deal with this problem with Multi-Similarity module and Attention module.

The proposed model has two guiding modules. First, multi similiary module finds a visual correspondence between the support image and query image. On the other hand attention module will make FSS network focus on target object of the query image.
The following image shows the overall structure of proposed model.

First, a pair of query image and support image are input to the backborn network( VGG, ResNet ). The last three blocks of these backborn model will extract feature maps from support image and query image repectively as following equation 1, 2.
To activate related object region, feature maps are masked with corresponding masks with bi-linear interpolation that interpolates the masks to appropriate channel.
To generate a visual correspondence, the paper compute pixel-wise cosine distance between feature map of support image and query image as equation 5. Figure 4 shows the overall process of computing this visual correspondance(similarity map)

Due to the limited number of support set image, the paper proposed a lightweight attention module. This module will extract the class related information from the few support sample images and guide the model to focus on the target region.
The extracted feature from block 2 and 3 are input to the attention module. Before feeding feature map, those who features are concatenated and reduced dimension. Actual process is as folloing figure.