모델의 설명 및 각각의 layer역활은 다른 페이지에서 작성하고 여기서는 각각의 모델 사용법 및 폴더 구조를 소개하고 있고 간단한 train code test code inference code에 대한 예시가 있다. det의 경우 mAP에 대해서 git hub에서 다운 받아야하고 각각의 custom data에 대한 학습과 test코드는 yaml 및 config 파일이 수정되어야 하므로 이에 대한것은 github를 참고 바랍니다.
개발 언어 및 프레임워크
1.문서 개요
본 문서는 노후시설물 상태판정 모델 개발과 관련한 기술적인 사항을 담고 있다.
2.사용 기술
- 모델 개발언어 : Python
- AI 프레임워크
| 용도 | 모델 | 개발 AI Framework |
|---|---|---|
| 시설물 상태판정 | EfficientNet | Tensorflow |
| 시설물 객체 및 파손부 감지 | EfficientDet | Pytorch |
3.AI 개발 환경
- 개발 H/W
| 용도 | 모델 | 개발 AI Framework |
|---|---|---|
| 시설물 상태판정 | EfficientNet | Tensorflow |
| 시설물 객체 및 파손부 감지 | EfficientDet | Pytorch |
- 개발 S/W
- OS : Ubuntu 18.04 LTS
- S/W : CUDA 10.2, 11.0 / cuDNN 8.0
- 개발툴 : Pycharm (Community Edtion), Tensorboard 등 활용
학습 모델 및 학습 조건
1.문서 개요
본 문서는 노후시설물 상태판정을 위한 인공지능 모델 개발 시 사용하는 AI 학습 모델과 이에 적용되는 학습 조건을 설명한다.
2. AI 학습 모델
| 학습 모델 | 논문 Reference | 비고 |
|---|---|---|
| EfficientNet | https://arxiv.org/pdf/1905.11946.pdf | 2020.9 발표 |
| EfficientDet | https://arxiv.org/pdf/1911.09070v1.pdf | 2019.11 발표 |
3.모델 학습 과정
-
EfficientNet
-
사용된 코드의 폴더 구조
efficientnet
├── LICENSE
├── init.py
├── config
├── efficientnet
├── inference
│ └── inference.py
├── model
│ ├── 2020-12-24_gnerate_model_task42_modelB4.h5
├── scripts
├── tensorboard
├── tests
│ └── test.py
├── train
│ └── train.py
├── utils
│ ├── category_classification.py
│ └── landmark_split_test.py
└── weight
└── checkpoint
- Train Code
```
import os
import sys
os.environ["CUDA_VISIBLE_DEVICES"] = '1'
sys.path.append('/home/dh3978/taskfolder/efficientnet')
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from efficientnet.tfkeras import EfficientNetB4
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.callbacks import EarlyStopping
from tensorflow.keras.callbacks import ModelCheckpoint,ReduceLROnPlateau
import datetime
from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True
train_datagen = ImageDataGenerator(rescale=1./255,
rotation_range=20,
horizontal_flip=True,
fill_mode='nearest'
)
val_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
'/data/task42-data/generator/images/train',
target_size=(300,300),
shuffle=True,
seed=66,
batch_size=2,
class_mode = 'categorical'
)
val_generator = val_datagen.flow_from_directory(
'/data/task42-data/generator/images/test',
target_size=(300,300),
batch_size=2,
class_mode='categorical'
)
# 1. model
model = Sequential()
model.add(EfficientNetB4(include_top=False,pooling = 'avg',weights = 'imagenet'))
model.add(Dense(119, activation='softmax',name='result'))
model.summary()
checkpoint_path = '../weight/gnerate_model__task42_modelB4_best.ckpt'
checkpoint_path = checkpoint_path[:10] + datetime.datetime.today().strftime('%Y-%m-%d') + '_' + checkpoint_path[10:]
modelcheckpoint = ModelCheckpoint(checkpoint_path, monitor='val_loss', mode='min', save_best_only=True, save_weights_only=True, verbose=1)
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=5, verbose=1)
earlystopping = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=10)
# 2. compile, fit
model.compile(optimizer = 'adam', loss = tf.keras.losses.categorical_crossentropy, metrics = ['acc'])
hist = model.fit_generator(train_generator,
steps_per_epoch=len(train_generator),
epochs=50,
validation_data=val_generator,
validation_steps=len(val_generator),
callbacks=[earlystopping, modelcheckpoint, reduce_lr])
model.load_weights(checkpoint_path)
model_path ='../model/gnerate_model_task42_modelB4.h5'
model_path = model_path[:9] + datetime.datetime.today().strftime('%Y-%m-%d') + '_' + model_path[9:]
model.save(model_path)
# 3. visualize
import matplotlib.pyplot as plt
plt.plot(hist.history['acc'])
plt.plot(hist.history['val_acc'])
plt.plot(hist.history['loss'])
plt.plot(hist.history['val_loss'])
plt.title('loss & acc')
plt.ylabel('loss, acc')
plt.xlabel('epoch')
plt.legend(['train acc','val acc', 'train loss', 'val loss'])
plt.show()
```- Test Code
import os
import cv2
import numpy as np
import tensorflow as tf
from tensorflow.keras.backend import shape
from tensorflow.keras.layers import Dropout
from sklearn.metrics import accuracy_score
import pandas as pd
os.environ["CUDA_VISIBLE_DEVICES"] = '2'
class FixedDropout(Dropout):
def _get_noise_shape(self, inputs):
if self.noise_shape is None:
return self.noise_shape
return tuple([shape(inputs)[i] if sh is None else sh for i, sh in enumerate(self.noise_shape)])
def swish(x):
return (tf.keras.activations.sigmoid(x) * x)
customObjects = {
'swish': swish,
'FixedDropout': FixedDropout
}
time_pd = pd.DataFrame(columns=["class","accuracy","train_set","test_set"])
# 1. data load
path = f'/data/task42-data/generator/images/infer'
train_path = f'/data/task42-data/generator/images/train'
category_ls = os.listdir(path)
category_ls.sort()
model_path = '../model/2020-12-24_gnerate_model_task42_modelB4.h5'
model = tf.keras.models.load_model(model_path, custom_objects=customObjects)
for idx, name in enumerate(category_ls):
images_ls = os.listdir(os.path.join(path,name))
images_train_ls = os.listdir(os.path.join(train_path,name))
train_set = len(images_train_ls)
x_test = list()
for image in images_ls:
img = cv2.imread(os.path.join(path, name, image))
img = cv2.resize(img, (300, 300))
# print(img)
x_test.append(img.tolist())
y_true = [idx for i in range(len(images_ls))]
y_true= np.array(y_true)
test_set = len(y_true) # .next()[1]))
x_test = np.array(x_test) / 255.0
y_predict = model.predict(x_test)
y_predict = np.argmax(y_predict, axis=1)
acc = accuracy_score(y_predict, y_true)
print('정확도 : {:.4f}'.format(acc))
time_pd.loc[idx] = [name,acc,train_set,test_set]
print("time_pd",time_pd)
time_pd.to_csv("/data/task42-data/csv/inference.csv", index=True)
- Inference Code
import os
import cv2
import numpy as np
import tensorflow as tf
from tensorflow.keras.backend import shape
from tensorflow.keras.layers import Dropout
from sklearn.metrics import accuracy_score
import pandas as pd
os.environ["CUDA_VISIBLE_DEVICES"] = '2'
class FixedDropout(Dropout):
def _get_noise_shape(self, inputs):
if self.noise_shape is None:
return self.noise_shape
return tuple([shape(inputs)[i] if sh is None else sh for i, sh in enumerate(self.noise_shape)])
def swish(x):
return (tf.keras.activations.sigmoid(x) * x)
customObjects = {
'swish': swish,
'FixedDropout': FixedDropout
}
time_pd = pd.DataFrame(columns=["class","accuracy","train_set","test_set"])
# 1. data load
path = f'/data/task42-data/generator/images/infer'
train_path = f'/data/task42-data/generator/images/train'
category_ls = os.listdir(path)
category_ls.sort()
model_path = '../model/2020-12-24_gnerate_model_task42_modelB4.h5'
model = tf.keras.models.load_model(model_path, custom_objects=customObjects)
for idx, name in enumerate(category_ls):
images_ls = os.listdir(os.path.join(path,name))
images_train_ls = os.listdir(os.path.join(train_path,name))
train_set = len(images_train_ls)
x_test = list()
for image in images_ls:
img = cv2.imread(os.path.join(path, name, image))
img = cv2.resize(img, (300, 300))
x_test.append(img.tolist())
y_true = [idx for i in range(len(images_ls))]
y_true= np.array(y_true)
test_set = len(y_true)
x_test = np.array(x_test) / 255.0
y_predict = model.predict(x_test)
y_predict = np.argmax(y_predict, axis=1)- EfficientDet
-
사용된 코드의 폴더 구조
efficientnet Yet-Another-EfficientDet-Pytorch
efficientDet
├── data
│ ├── check_obj.py
│ └── data_check.py
├── efficientdet
│ ├── loss.py
│ ├── model.py
│ ├── project42dataset.py //시설물탐지
│ ├── project42-2dataset.py //파손탐지
│ └── utils.py
├── efficientnet
│ ├── init.py
│ ├── model.py
│ └── utils.py│
├── mAP
│ ├── input
│ │ ├── detection-results
│ │ │ ├── 0.txt //시설물탐지
│ │ │ └── 1.txt //파손탐지
│ │ └── ground-truth
│ │ ├── 0.txt //시설물탐지
│ │ └── 1.txt //파손탐지
│ ├── main.py
│ ├── output
│ │ └── output.txt
│ └── utils_train.py
├── projects
│ ├── object.yml //시설물탐지
│ └── damage.yml //파손탐지
├── test
│ └── img_inferred_d4_1.jpg
├── utils
├── weights
│ ├── object_weight
│ ├── damage_weight
│ ├── efficientdet-d0.pth
│ ├── efficientdet-d1.pth
│ ├── efficientdet-d2.pth
│ ├── efficientdet-d3.pth
│ ├── efficientdet-d4.pth
│ └── efficientdet-d5.pth
├── backbone.py
├── create_yamlntxt.py //시설물탐지 yml 생성
├── create_yamlntxt_damage.py //파손탐지 yml 생성
├── inference_class.py //시설물탐지 or 파손 부위 탐지
├── inference.py // 시설물 및 파손탐지
├── test_object.py //시설물탐지
├── test_damage.py //파손탐지
├── train_object.py //시설물탐지
└── train_damage.py //파손부위
- train_object.py
import argparse
import datetime
import os
import traceback
import numpy as np
import torch
import yaml
from tensorboardX import SummaryWriter
from torch import nn
from torch.utils.data import DataLoader
from torchvision import transforms
from tqdm.autonotebook import tqdm
from backbone import EfficientDetBackbone
from efficientdet.utils import BBoxTransform, ClipBoxes
from utils.utils import postprocess
from efficientdet.project42_objcet_dataset import Project42Dataset, Resizer, Normalizer, Augmenter, collater
from efficientdet.loss import FocalLoss
from utils.sync_batchnorm import patch_replication_callback
from utils.utils import replace_w_sync_bn, CustomDataParallel, get_last_weights, init_weights, boolean_string
from mAP.utils_train import mAP_score
import shutil
import cv2
# reset mAP folder
if os.path.isdir('mAP/input/detection-results'):
shutil.rmtree('mAP/input/detection-results')
if os.path.isdir('mAP/input/ground-truth'):
shutil.rmtree('mAP/input/ground-truth')
# assign CUDA device
os.environ["CUDA_VISIBLE_DEVICES"] = '0
class Params:
def __init__(self, project_file):
self.params = yaml.safe_load(open(project_file).read())
def __getattr__(self, item):
return self.params.get(item, None)
def get_args():
parser = argparse.ArgumentParser('Yet Another EfficientDet Pytorch: SOTA object detection network - Zylo117')
parser.add_argument('-p', '--project', type=str, default='final_project', help='project file that contains parameters') # you need to change // data가 들어있는 파일의 프로젝트 명
parser.add_argument('-c', '--compound_coef', type=int, default=0, help='coefficients of efficientdet') # 모델명 변경 ex) B4
parser.add_argument('-n', '--num_workers', type=int, default=12, help='num_workers of dataloader')
parser.add_argument('--batch_size', type=int, default=16, help='The number of images per batch among all devices') # batch size 변경
parser.add_argument('--head_only', type=boolean_string, default=False,
help='whether finetunes only the regressor and the classifier, '
'useful in early stage convergence or small/easy dataset')
parser.add_argument('--lr', type=float, default=1e-4)
parser.add_argument('--optim', type=str, default='adamw', help='select optimizer for training, '
'suggest using \'admaw\' until the'
' very final stage then switch to \'sgd\'')
parser.add_argument('--num_epochs', type=int, default=500)
parser.add_argument('--val_interval', type=int, default=10, help='Number of epoches between valing phases')
parser.add_argument('--save_interval', type=int, default=10, help='Number of steps between saving')
parser.add_argument('--es_min_delta', type=float, default=0.0,
help='Early stopping\'s parameter: minimum change loss to qualify as an improvement')
parser.add_argument('--es_patience', type=int, default=0,
help='Early stopping\'s parameter: number of epochs with no improvement after which training will be stopped. Set to 0 to disable this technique.')
parser.add_argument('--data_path', type=str, default='/mnt/data/task42-data/',
help='the root folder of dataset') # you need to change
parser.add_argument('--log_path', type=str, default='/mnt/data/task42_weights/john/obj')
parser.add_argument('-w', '--load_weights', type=str, default=None,
# 모델 변경시 불러오는 weight 변경해야함
help='whether to load weights from a checkpoint, set None to initialize, set \'last\' to load last checkpoint')
# parser.add_argument('-w', '--load_weights', type=str, default='weights/efficientdet-d0.pth', # 모델 변경시 불러오는 weight 변경해야함
# help='whether to load weights from a checkpoint, set None to initialize, set \'last\' to load last checkpoint')
parser.add_argument('--saved_path', type=str, default='/mnt/data/task42_weights/john/obj')
parser.add_argument('--debug', type=boolean_string, default=False,
help='whether visualize the predicted boxes of training, '
'the output images will be in test/')
args = parser.parse_args()
return args
class ModelWithLoss(nn.Module):
def __init__(self, model, debug=False):
super().__init__()
self.criterion = FocalLoss()
self.model = model
self.debug = debug
def forward(self, imgs, annotations, obj_list=None):
_, regression, classification, anchors = self.model(imgs)
if self.debug:
cls_loss, reg_loss = self.criterion(classification, regression, anchors, annotations,
imgs=imgs, obj_list=obj_list)
else:
cls_loss, reg_loss = self.criterion(classification, regression, anchors, annotations)
return cls_loss, reg_loss, regression, classification, anchors
def train(opt):
params = Params(f'projects/object.yml')
if params.num_gpus == 0:
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
if torch.cuda.is_available():
torch.cuda.manual_seed(42)
else:
torch.manual_seed(42)
save_time = datetime.datetime.now().strftime("%Y%m%d-%H%M%S") # 날짜,시간대별로 파일 만들어줌
opt.saved_path = opt.saved_path + f'/{params.project_name}/crop/weights/{save_time}'
opt.log_path = opt.log_path + f'/{params.project_name}/crop/tensorboard/'
os.makedirs(opt.log_path, exist_ok=True)
os.makedirs(opt.saved_path, exist_ok=True)
training_params = {'batch_size': opt.batch_size,
'shuffle': True,
'drop_last': True,
'collate_fn': collater,
'num_workers': opt.num_workers}
val_params = {'batch_size': opt.batch_size,
'shuffle': False,
'drop_last': True,
'collate_fn': collater,
'num_workers': opt.num_workers}
input_sizes = [512, 640, 768, 896, 1024, 1280, 1280, 1536, 1536]
training_set = Project42Dataset(root_dir=os.path.join(opt.data_path, params.project_name), set=params.train_set, params = params,
transform=transforms.Compose([Normalizer(mean=params.mean, std=params.std),
Augmenter(),
Resizer(input_sizes[opt.compound_coef])]))
training_generator = DataLoader(training_set, **training_params)
val_set = Project42Dataset(root_dir=os.path.join(opt.data_path, params.project_name), set=params.val_set, params = params,
transform=transforms.Compose([Normalizer(mean=params.mean, std=params.std),
Resizer(input_sizes[opt.compound_coef])]))
val_generator = DataLoader(val_set, **val_params)
# labels
labels = training_set.labels
print('label:', labels)
model = EfficientDetBackbone(num_classes=len(params.obj_list), compound_coef=opt.compound_coef,
ratios=eval(params.anchors_ratios), scales=eval(params.anchors_scales))
# load last weights
if opt.load_weights is not None:
if opt.load_weights.endswith('.pth'):
weights_path = opt.load_weights
else:
weights_path = get_last_weights(opt.saved_path)
try:
last_step = int(os.path.basename(weights_path).split('_')[-1].split('.')[0])
except:
last_step = 0
try:
ret = model.load_state_dict(torch.load(weights_path), strict=False)
except RuntimeError as e:
print(f'[Warning] Ignoring {e}')
print(
'[Warning] Don\'t panic if you see this, this might be because you load a pretrained weights with different number of classes. The rest of the weights should be loaded already.')
print(f'[Info] loaded weights: {os.path.basename(weights_path)}, resuming checkpoint from step: {last_step}')
else:
last_step = 0
print('[Info] initializing weights...')
init_weights(model)
# freeze backbone if train head_only
if opt.head_only:
def freeze_backbone(m):
classname = m.__class__.__name__
for ntl in ['EfficientNet', 'BiFPN']:
if ntl in classname:
for param in m.parameters():
param.requires_grad = False
model.apply(freeze_backbone)
print('[Info] freezed backbone')
# https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
# apply sync_bn when using multiple gpu and batch_size per gpu is lower than 4
# useful when gpu memory is limited.
# because when bn is disable, the training will be very unstable or slow to converge,
# apply sync_bn can solve it,
# by packing all mini-batch across all gpus as one batch and normalize, then send it back to all gpus.
# but it would also slow down the training by a little bit.
if params.num_gpus > 1 and opt.batch_size // params.num_gpus < 4:
model.apply(replace_w_sync_bn)
use_sync_bn = True
else:
use_sync_bn = False
writer = SummaryWriter(opt.log_path + f'/{save_time}/')
# warp the model with loss function, to reduce the memory usage on gpu0 and speedup
model = ModelWithLoss(model, debug=opt.debug)
if params.num_gpus > 0:
model = model.cuda()
if params.num_gpus > 1:
model = CustomDataParallel(model, params.num_gpus)
if use_sync_bn:
patch_replication_callback(model)
if opt.optim == 'adamw':
optimizer = torch.optim.AdamW(model.parameters(), opt.lr)
else:
optimizer = torch.optim.SGD(model.parameters(), opt.lr, momentum=0.9, nesterov=True)
scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, patience=3, verbose=True)
epoch = 0
best_loss = 1e5
best_epoch = 0
step = max(0, last_step)
model.train()
num_iter_per_epoch = len(training_generator)
try:
for epoch in range(opt.num_epochs):
last_epoch = step // num_iter_per_epoch
if epoch < last_epoch:
continue
epoch_loss = []
progress_bar = tqdm(training_generator)
for iter, data in enumerate(progress_bar):
if iter < step - last_epoch * num_iter_per_epoch:
progress_bar.update()
continue
try:
imgs = data['img']
annot = data['annot']
if params.num_gpus == 1:
# if only one gpu, just send it to cuda:0
# elif multiple gpus, send it to multiple gpus in CustomDataParallel, not here
imgs = imgs.cuda()
annot = annot.cuda()
optimizer.zero_grad()
cls_loss, reg_loss, regression, classification, anchors= model(imgs, annot, obj_list=params.obj_list)
cls_loss = cls_loss.mean()
reg_loss = reg_loss.mean()
loss = cls_loss + reg_loss
if loss == 0 or not torch.isfinite(loss):
continue
loss.backward()
# torch.nn.utils.clip_grad_norm_(model.parameters(), 0.1)
optimizer.step()
# loss
epoch_loss.append(float(loss))
# mAP
threshold = 0.2
iou_threshold = 0.2
regressBoxes = BBoxTransform()
clipBoxes = ClipBoxes()
out = postprocess(imgs,
anchors, regression, classification,
regressBoxes, clipBoxes,
threshold, iou_threshold)
# mAP = mAP_score(annot, out, labels)
# mAP = mAP.results['mAP']
# progress_bar.set_description(
# 'Step: {}. Epoch: {}/{}. Iteration: {}/{}. Cls loss: {:.5f}. Reg loss: {:.5f}. Total loss: {:.5f}. mAP: {:.2f}'.format(
# step, epoch+1, opt.num_epochs, iter + 1, num_iter_per_epoch, cls_loss.item(),
# reg_loss.item(), loss.item(), mAP))
progress_bar.set_description(
'Step: {}. Epoch: {}/{}. Iteration: {}/{}. Cls loss: {:.5f}. Reg loss: {:.5f}. Total loss: {:.5f}'. format(
step, epoch + 1, opt.num_epochs, iter + 1, num_iter_per_epoch, cls_loss.item(),
reg_loss.item(), loss.item()))
writer.add_scalars('Loss', {'train': loss}, step)
writer.add_scalars('Regression_loss', {'train': reg_loss}, step)
writer.add_scalars('Classfication_loss', {'train': cls_loss}, step)
# writer.add_scalars('mAP', {'train': mAP}, step)
# log learning_rate
current_lr = optimizer.param_groups[0]['lr']
writer.add_scalar('learning_rate', current_lr, step)
step += 1
if step % opt.save_interval == 0 and step > 0:
save_checkpoint(model, f'efficientdet-d{opt.compound_coef}_{epoch}.pth')
print('checkpoint...')
except Exception as e:
print('[Error]', traceback.format_exc())
print(e)
continue
scheduler.step(np.mean(epoch_loss))
if epoch % opt.val_interval == 0:
model.eval()
loss_regression_ls = []
loss_classification_ls = []
for iter, data in enumerate(val_generator):
with torch.no_grad():
imgs = data['img']
annot = data['annot']
if params.num_gpus == 1:
imgs = imgs.cuda()
annot = annot.cuda()
cls_loss, reg_loss, regression, classification, anchors = model(imgs, annot, obj_list=params.obj_list)
cls_loss = cls_loss.mean()
reg_loss = reg_loss.mean()
loss = cls_loss + reg_loss
if loss == 0 or not torch.isfinite(loss):
continue
loss_classification_ls.append(cls_loss.item())
loss_regression_ls.append(reg_loss.item())
cls_loss = np.mean(loss_classification_ls)
reg_loss = np.mean(loss_regression_ls)
loss = cls_loss + reg_loss
# mAP
threshold = 0.2
iou_threshold = 0.2
regressBoxes = BBoxTransform()
clipBoxes = ClipBoxes()
out = postprocess(imgs,
anchors, regression, classification,
regressBoxes, clipBoxes,
threshold, iou_threshold)
# mAP = mAP_score(annot, out, labels)
# mAP = mAP.results['mAP']
print(
'Val. Epoch: {}/{}. Classification loss: {:1.5f}. Regression loss: {:1.5f}. Total loss: {:1.5f}.'.format(
epoch+1, opt.num_epochs, cls_loss, reg_loss, loss))
writer.add_scalars('Loss', {'val': loss}, step)
writer.add_scalars('Regression_loss', {'val': reg_loss}, step)
writer.add_scalars('Classfication_loss', {'val': cls_loss}, step)
# writer.add_scalars('mAP', {'val': mAP}, step)
if loss + opt.es_min_delta < best_loss:
best_loss = loss
best_epoch = epoch
save_checkpoint(model, f'efficientdet-d{opt.compound_coef}_{epoch}_{step}.pth')
model.train()
# Early stopping
if epoch - best_epoch > opt.es_patience > 0:
print('[Info] Stop training at epoch {}. The lowest loss achieved is {}'.format(epoch, best_loss))
break
except KeyboardInterrupt:
save_checkpoint(model, f'efficientdet-d{opt.compound_coef}_{epoch}_{step}.pth')
writer.close()
writer.close()
def save_checkpoint(model, name):
if isinstance(model, CustomDataParallel):
torch.save(model.module.model.state_dict(), os.path.join(opt.saved_path, name))
else:
torch.save(model.model.state_dict(), os.path.join(opt.saved_path, name))
if __name__ == '__main__':
opt = get_args()
train(opt)
- Test Code
import torch
from torch.backends import cudnn
import argparse
from backbone import EfficientDetBackbone
import os
import yaml
from torch.utils.data import DataLoader
from torchvision import transforms
from efficientdet.utils import BBoxTransform, ClipBoxes
from utils.utils import postprocess
from efficientdet.project42_objcet_dataset import Project42Dataset, Resizer, Normalizer, collater
from mAP.utils import mAP_score
parser = argparse.ArgumentParser()
parser.add_argument('-p', '--project', type=str, default='use_data', help = 'project name')
parser.add_argument('-c', '--compound_coef', type=int, default=0, help='coefficients of efficientdet')
parser.add_argument('-f', '--force_input_size', default=None, help='set None to use default size')
parser.add_argument('-w', '--weights', type=str, help='number of weights to use' )
parser.add_argument('--batch_size', type=int, default=2, help='The number of images per batch among all devices')
parser.add_argument('--data_path', type=str, default='/mnt/data/task42-data/', help='the root folder of dataset')
parser.add_argument('-n', '--num_workers', type=int, default=12, help='num_workers of dataloader')
opt = parser.parse_args()
save_time = '20201204-142948'
opt.weights = '199'
# os.environ["CUDA_VISIBLE_DEVICES"] = '0'
class Params:
def __init__(self, project_file):
self.params = yaml.safe_load(open(project_file).read())
def __getattr__(self, item):
return self.params.get(item, None)
# project informations
params = Params(f'projects/object.yml')
# replace this part with your project's anchor config
anchor_ratios = [(1.0, 1.0), (1.4, 0.7), (0.7, 1.4)]
anchor_scales = [2 ** 0, 2 ** (1.0 / 3.0), 2 ** (2.0 / 3.0)]
threshold = 0.2
iou_threshold = 0.2
use_cuda = True
use_float16 = False
cudnn.fastest = True
cudnn.benchmark = True
obj_list = params.obj_list
val_params = {'batch_size': opt.batch_size,
'shuffle': False,
'drop_last': True,
'collate_fn': collater,
'num_workers': opt.num_workers}
# tf bilinear interpolation is different from any other's, just make do
input_sizes = [512, 640, 768, 896, 1024, 1280, 1280, 1536, 1536]
input_size = input_sizes[opt.compound_coef] if opt.force_input_size is None else opt.force_input_size
val_set = Project42Dataset(root_dir=os.path.join(opt.data_path, params.project_name), set=params.val_set, params = params,
transform=transforms.Compose([Normalizer(mean=params.mean, std=params.std),
Resizer(input_sizes[opt.compound_coef])]))
val_generator = DataLoader(val_set, **val_params)
model = EfficientDetBackbone(compound_coef=opt.compound_coef, num_classes=len(obj_list),
ratios=anchor_ratios, scales=anchor_scales)
# model.load_state_dict(torch.load(f'/data/efdet/logs/{opt.project}/weights/{save_time}/efficientdet-d{opt.compound_coef}_{opt.weights}.pth', map_location='cpu'))
model.load_state_dict(torch.load(f'./weights/object_weight/object_efficientdet-d{opt.compound_coef}_{opt.weights}.pth', map_location='cpu'))
model.requires_grad_(False)
model.eval()
if use_cuda:
model = model.cuda()
if use_float16:
model = model.half()
results = []
annots = []
for iter, data in enumerate(val_generator):
with torch.no_grad():
features, regression, classification, anchors = model(data['img'].cuda()) # cuda & cup type match
regressBoxes = BBoxTransform()
clipBoxes = ClipBoxes()
out = postprocess(data['img'],
anchors, regression, classification,
regressBoxes, clipBoxes,
threshold, iou_threshold)
annot = data['annot'].detach().cpu().numpy().tolist()
annots += annot
results += out
# print(result)
print(len(results))
mAP = mAP_score(val_set, annots, results, val_params)
mAP = mAP() # call
print(mAP)- Inference Code
import time
import torch
from torch.backends import cudnn
from matplotlib import colors
from backbone import EfficientDetBackbone
import cv2
import numpy as np
import os
import yaml
from efficientdet.utils import BBoxTransform, ClipBoxes
from utils.utils import preprocess, invert_affine, postprocess, STANDARD_COLORS, standard_to_bgr, get_index_label, plot_one_box
project = 'task42_1130'
number = '199'
save_time = '20201203-174940'
compound_coef = 4
force_input_size = None # set None to use default size
# img_path = 'datasets/shape/val/900.jpg''
# img_path = os.path.join('datasets', project, 'test03.jpg') # image load
# img_path = os.path.join('/home/jain/Downloads','test', '19-1.png') # image load
img_path = os.path.join('/data/data','ex_task42', f'{project}','crop', 'images', '17384853-775f-44f6-af92-0a0de053b821.jpg') # image load
# img_name
if type(img_path) == list:
img_names = [i.split('/')[-1] for i in img_path]
else:
img_names = [img_path.split('/')[-1]]
# replace this part with your project's anchor config
anchor_ratios = [(1.0, 1.0), (1.4, 0.7), (0.7, 1.4)]
anchor_scales = [2 ** 0, 2 ** (1.0 / 3.0), 2 ** (2.0 / 3.0)]
threshold = 0.3
iou_threshold = 0.2
use_cuda = True
use_float16 = False
cudnn.fastest = True
cudnn.benchmark = True
class Params:
def __init__(self, project_file):
self.params = yaml.safe_load(open(project_file).read())
def __getattr__(self, item):
return self.params.get(item, None)
params = Params(f'projects/{project}_crop.yml')
obj_list = params.obj_list
obj_list.sort()
print(obj_list)
color_list = standard_to_bgr(STANDARD_COLORS)
# tf bilinear interpolation is different from any other's, just make do
input_sizes = [512, 640, 768, 896, 1024, 1280, 1280, 1536, 1536]
input_size = input_sizes[compound_coef] if force_input_size is None else force_input_size
ori_imgs, framed_imgs, framed_metas = preprocess(img_path, max_size=input_size)
if use_cuda:
x = torch.stack([torch.from_numpy(fi).cuda() for fi in framed_imgs], 0)
else:
x = torch.stack([torch.from_numpy(fi) for fi in framed_imgs], 0)
x = x.to(torch.float32 if not use_float16 else torch.float16).permute(0, 3, 1, 2)
model = EfficientDetBackbone(compound_coef=compound_coef, num_classes=len(obj_list),
ratios=anchor_ratios, scales=anchor_scales)
# model.load_state_dict(torch.load(f'logs/{project}/efficientdet-d{compound_coef}_{number}.pth', map_location='cpu'))
# model.load_state_dict(torch.load(f'/data/efdet/logs/{project}/efficientdet-d{compound_coef}_{number}.pth', map_location='cpu'))
model.load_state_dict(torch.load(f'/data/efdet/logs/{project}/crop/weights/{save_time}/efficientdet-d{compound_coef}_{number}.pth', map_location='cpu'))
model.requires_grad_(False)
model.eval()
if use_cuda:
model = model.cuda()
if use_float16:
model = model.half()
with torch.no_grad():
features, regression, classification, anchors = model(x)
regressBoxes = BBoxTransform()
clipBoxes = ClipBoxes()
out = postprocess(x,
anchors, regression, classification,
regressBoxes, clipBoxes,
threshold, iou_threshold)
print(out)
def display(preds, imgs, imshow=True, imwrite=False):
for i, img_name in zip(range(len(imgs)), img_names):
# if len(preds[i]['rois']) == 0: # if model dosen't detect object, not show image
# continue
imgs[i] = imgs[i].copy()
for j in range(len(preds[i]['rois'])):
x1, y1, x2, y2 = preds[i]['rois'][j].astype(np.int)
obj = obj_list[preds[i]['class_ids'][j]]
score = float(preds[i]['scores'][j])
plot_one_box(imgs[i], [x1, y1, x2, y2], label=obj,score=score,color=color_list[get_index_label(obj, obj_list)]) # web color 사용
print(obj)
if imwrite:
img = cv2.cvtColor(imgs[i], cv2.COLOR_BGR2RGB)
cv2.imwrite(f'test/img_inferred_d{compound_coef}_{img_name}', img)
if imshow:
img = cv2.cvtColor(imgs[i], cv2.COLOR_BGR2RGB)
cv2.namedWindow(f'{img_name}', cv2.WINDOW_NORMAL | cv2.WINDOW_KEEPRATIO)
cv2.imshow(f'{img_name}', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
out = invert_affine(framed_metas, out)
display(out, ori_imgs)
print("Done")
코린2