[Title]

Quick Look

Authors & Affiliation:

[Authors]: Shaoyu Chen, Jiemin Fang, Qian Zhang, Wenyu Liu, Xinggang Wang

[Affiliations]: School of EIC, Huazhong University of Science & Technology,
Institute of AI, Huazhong University of Science & Technology, Horizon Robotics

Link : https://arxiv.org/pdf/2108.02350.pdf

Comments: Published at ICCV2021

TLDR: 3D instance segmentation using hierarchical aggregation

Relevance: 4

Research Topic

• Category (General) : Computer Vision
• Category (Specific) : 3D instance segmentation

Paper summary (What)

[Summary of the paper - a few sentences with bullet points. What did they do?]

• a novel bottom-up framework with the hierarchical aggregation for instance segmentation on 3D point cloud
• 1st on the leaderboard of ScanNetv2, SOTA on S3DIS
• highest efficiency among all existing methods (2021 기준), average per-frame inference time on ScanNet v2 is only 410ms

Issues addressed by the paper (Why)

[What are the issues that the paper addresses? Describe the problem. Why did they write this paper?]

In extending 2D instance segmentation to 3D scenes, most existing 3D methods adopt a totally different bottom-up pipeline, which generates instances through clustering.

the difficulties of directly clustering a point cloud into multiple instances

Becuase:

1. a point cloud usually contains a large number of points
2. the number of instances in a point cloud has large variations for different 3D scenes
3. the size of instances vary significantly
4. each point has a very weak feature

Detailed Information (How)

Methodology

1. The point-wise prediction network extracts features from point clouds
2. Predicts point-wise semantic labels and center shift vectors
3. The point aggregation module forms preliminary instace predictions based on point-wise prediction results
4. The set aggregation module expands incomplete instances to cover missing parts
5. The intra instance prediction network smooths instances to filter out outliers

Assumptions

[What assumptions were made and are these assumptions valid?]

Prominent Formulas

Prominent Figures

Results

[Theoretical or empirical results (any main tables) ]

Limitations

[Did the authors mention any limitations to their work? Do you see any limitations of their work?]

1. It was validated only for objects, not for human body or face which is more elaborate an detailed
2. They throw away some raw data in the filtering processing which is in intra-instance prediction.

Confusing aspects of the paper

[Is there anything that is confusing and could need better explanations or references?]

Conclusions

The author's conclusions

HAIS is a concise bottom-up approach for 3D instance segmentation.

The effectiveness and generalization of the method are demonstrated by Experiments on ScanNet v2 and S3DIS

HAIS retains much better inference speed than all existing methods, showing its practicability expecially latency-sensitive ones.

→ HAIS is concise, effective and fast.

My Conclusion

[What do you think about the work presented in the article? Did the authors manage to achieve what they set out to achieve?]

I agree that the extension of 2D to 3D has many difficulties as they points out.

In this point, the trial to address the issue and achievement of this study matter.

But, what I want more is that the 3D point data doesn’t loss its raw data.

It will have some trouble when the 3D points are rendered into 3D model

Rating

Wow

Possible future work / improvements

[Can you think of ways to improve this paper or ideas for future work?]

1. 사람 신체 구조에 대한 data set은 face parsing으로 현재 구축이 되어있으므로, 이 데이터로 instance segmentation 학습을 진행하면 우리 연구에 맞는 결과값을 도출해낼 수 있을 것 같음

2. raw data를 잃어서 3d reconstrunction을 위한 정보도 함께 소실되는 문제를 해결하기 위해, point-base MVS 기술을 이용해 3d depth map을 얻는 해결책을 고안해볼 수 있음.

   (민서 리뷰 링크 추가하기)

Extra

• Cited references to follow up on / related papers: NSFC
• Source code/ blog/ twitter thread/ other links: https://github.com/hustvl/HAIS

---