EfficientNetv2 성능 개선

Datasts

• train
  • normal : 8589
  • abnormal : 4959
• val
  • normal : 181
  • abnormal : 118
• test
  • normal : 190
  • abnormal : 109

Image Classification 성능 개선 방법

ResNet, EfficientNet 모델을 torchvision.models로 불러와 추가 작업없이 단순히 훈련을 진행하는 경우 maximum 정확도를 끌어내지 못할 가능성이 있다. 다시말해 그냥 가져와서 이미지 데이터셋 입출력을 설정하는 방법으로 모델을 사용하게 되면, 해당 모델의 최고의 성능을 뽑아내지 못할 수도 있다.

모델을 불러와서 다음과 같은 최적화 방법들을 고려해야한다. 이들을 통해 가장 높은 정확도를 끌어올릴 수 있는 최적의 조합을 찾아야 한다.

• optimizer
• scheduler
• dropout
• weight initialize

---

Setting1-1

optimizer = torch.optim.Adam(base_model.parameters(), lr=0.001)
lrp_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)

Result

• adam 적용

Validset Accuracy(mean): 78.260870 %

Testset Accuracy(mean): 73.578595 %

Confusion Matirx : 
[[170  20]
 [ 59  50]]
- Sensitivity :  45.87155963302752
- Specificity :  89.47368421052632
- Precision:  71.42857142857143
- NPV:  74.235807860262
- F1 :  55.8659217877095

Setting1-2

optimizer = torch.optim.Adam(base_model.parameters(), lr=0.001)
lrp_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)

Result

• adam 적용

Validset Accuracy(mean): 80.267559 %

Testset Accuracy(mean): 72.909699 %

Confusion Matirx : 
[[156  34]
 [ 47  62]]
- Sensitivity :  56.88073394495413
- Specificity :  82.10526315789474
- Precision:  64.58333333333334
- NPV:  76.84729064039408
- F1 :  60.48780487804879

Setting2

• adam + L2 적용

optimizer = torch.optim.Adam(base_model.parameters(), lr=0.001)
lrp_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)

Result

Validset Accuracy(mean): 76.588629 %

Testset Accuracy(mean): 71.571906 %

Confusion Matirx : 
[[133  57]
 [ 28  81]]
- Sensitivity :  74.31192660550458
- Specificity :  70.0
- Precision:  58.69565217391305
- NPV:  82.6086956521739
- F1 :  65.58704453441295

Setting3

• drop all 적용

optimizer = torch.optim.Adam(base_model.parameters(), lr=0.001)
lrp_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)

Result

Validset Accuracy(mean): 77.257525 %

Testset Accuracy(mean): 75.919732 %

Confusion Matirx : 
[[164  26]
 [ 46  63]]
- Sensitivity :  57.798165137614674
- Specificity :  86.31578947368422
- Precision:  70.78651685393258
- NPV:  78.0952380952381
- F1 :  63.636363636363626

---

Setting4 efficientnet m

Result

Validset Accuracy(mean): 78.260870 %

Testset Accuracy(mean): 72.240803 %

Confusion Matirx : 
[[135  55]
 [ 28  81]]
- Sensitivity :  74.31192660550458
- Specificity :  71.05263157894737
- Precision:  59.55882352941176
- NPV:  82.82208588957054
- F1 :  66.12244897959184

Setting5 efficientnet l

optimizer = torch.optim.Adam(base_model.parameters(), lr=0.001)
lrp_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)

Result

Validset Accuracy(mean): 77.257525 %

Testset Accuracy(mean): 70.234114 %

Confusion Matirx : 
[[166  24]
 [ 65  44]]
- Sensitivity :  40.36697247706422
- Specificity :  87.36842105263159
- Precision:  64.70588235294117
- NPV:  71.86147186147186
- F1 :  49.717514124293785

---

Setting 6-1 efficient-s

• adam_l2

optimizer = torch.optim.Adam(base_model.parameters(), lr=0.001)
lrp_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)

Result

Validset Accuracy(mean): 78.595318 %

Testset Accuracy(mean): 75.585284 %

Confusion Matirx : 
[[169  21]
 [ 52  57]]
- Sensitivity :  52.293577981651374
- Specificity :  88.94736842105263
- Precision:  73.07692307692307
- NPV:  76.47058823529412
- F1 :  60.962566844919785

Setting 5-2 efficientv2-m

• adam+l2

optimizer = torch.optim.Adam(base_model.parameters(), lr=0.001)
lrp_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)

Result

Validset Accuracy(mean): 80.267559 %

Testset Accuracy(mean): 73.578595 %

Confusion Matirx : 
[[151  39]
 [ 40  69]]
- Sensitivity :  63.30275229357798
- Specificity :  79.47368421052632
- Precision:  63.888888888888886
- NPV:  79.05759162303664
- F1 :  63.59447004608294

Setting 5-3 efficientv2-l

• adam+l2

optimizer = torch.optim.Adam(base_model.parameters(), lr=0.001)
lrp_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)

Result

Validset Accuracy(mean): 82.943144 %

Testset Accuracy(mean): 75.585284 %

Confusion Matirx : 
[[159  31]
 [ 42  67]]
- Sensitivity :  61.46788990825688
- Specificity :  83.6842105263158
- Precision:  68.36734693877551
- NPV:  79.1044776119403
- F1 :  64.7342995169082

---

Setting Val 수정

train
  -normal (8589)
  -abnormal (4959)
  
result
  -train 
    -normal (2975)
    -abnormal (5153)
  -val 
  	-normal (991)
    -abnormal (1717)
  -test 
    - normal (993)
    - abnormal (1719)

optimizer = torch.optim.Adam(base_model.parameters(), lr=0.001)
lrp_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)

Result

Validset Accuracy(mean): 96.491876 %

Testset Accuracy(mean): 96.497050 %

Confusion Matirx : 
[[1661   58]
 [  37  956]]
- Sensitivity :  96.27391742195368
- Specificity :  96.6259453170448
- Precision:  94.28007889546352
- NPV:  97.82096584216725
- F1 :  95.26656701544593