[Docker] YOLOv4 training

pre-requirment

docker, nvidia-docker

Dataset folder

Make directory that mount to docker container(folder)

dir location is up to you

$ mkdir data
$ cd data
$ mkdir train test backup cfg

Train/Test data (image, annotation file)

move your custom data pair(image,annotation file) to each train/test directory

/data/test

Make names file(.names)

custom.names is whatever you want
ex) esens.names
but you have to uniform the name with .data

$ vi <custom>.names

list up your custom dataset classes
class number
0 : person
1 : bicycle
2 : car
...
5 : truck

Make data file(.data)

custom.data is whatever you want
ex) esens.data
but you have to uniform the name

$ vi <custom>.data

class = custom data class number
train = train data list file location
test = test data list file location
names = .names file location
backup = weight are stored to backup folder

Make train/test data list file(.txt)

use this code below to list up your train/test dataset
-> train.txt, test.txt
run this code at <your_directory>/data

import os

#dataset 정합 후 train:test = 8:2 로 data/train data/test 에 저장하고 
#이코드 각 각 돌려서 위에처럼 txt파일 생성해야함 생성된 txt파일들 train돌릴때 넣어줘야함

# dir_path => "<your_directory>/data"
dir_path = "/home/jay/DataSets/coco/data"
train_data_path = "data/train"
test_data_path = "data/test"

data = []
for (root, directories, files) in os.walk(dir_path):
    for file in files:
        if '.jpg' in file:
            #file_path = os.path.join(train_data_path, file)
            file_path = os.path.join(test_data_path, file)
            print(file_path)
            data.append(file_path)

# train.txt / test.txt 
data_list = open(r'test.txt','w') 

for i in data:
    data_list.write(i + '\n')

data_list.close()

result test.txt

cfg file (.cfg)

if you use custom cfg file you can put in here
copy yolov4.cfg or yolov4-tiny.cfg ... from alexeyab/darknet github repository and change parameters

• batch : 64
• subdivision = 16 ( 메모리 부하 걸리면 32나 64로 바꾸면 됨 )
• max_batch = 2000 * = 2000
• steps = 0.8max_batch, 0.9max_batch = 1600, 1800
• width, heigth : 32의 배수로 해야 AP 좋음. 416, 416
• 각각의 [yolo] 계층 수정 ⇒ class =
• 중요! [yolo] 바로 전 [convolutional]계층 filter = ( + 5)*3 = 18
• random = 1로 하면 해상도 다른 환경에서도 AP 좋음. when error occurs change to 0

my custom data cfg file

Result

Docker image and train

Pull Docker image

$ sudo docker pull cjh2626002/yolo-ros:train

run image with mount

$ xhost +
$ sudo docker run -it -e DISPLAY=unix$DISPLAY --device /dev/video0:/dev/video0 --privileged -v /tmp/.X11-unix/:/tmp/.X11-unix/ -v <your-data-directory>/data/:/workspace/darknet/data --gpus all cjh2626002/yolo-ros:train

docker container with darknet

# cd darknet/data

you can check your local directory is mounted to container

Training

execute darknet at root directory of repo
and you have to download pre-trained weight file

yolov4 : https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.conv.137

# ./darknet detector train data/<custom>.data data/cfg/<custom>.cfg yolov4.conv.137 -map

enjoy!