[Wandb]#1 Sweep

Wandb sweep is a great tool for optimizing model by searching hyperparameters. I record my settings using sweep in this post.

1st. Create yaml

in creating yaml, it's important not to include sub configures like nn_params in my case. If you make something as parameters for sweep, the config created from wandb is saving it as a string information not as an independent configurations.

• correct
  
• wrong
  

2nd(optional). python3 command

if you are using python3, then add path to python

sudo ln -s $(which python3) /usr/local/bin/python
export PATH=/path/to/python3:$PATH

or just add command python3 to yaml, and change command options like this

3rd. Match the path, Implement

match the path where the yaml is located, and create sweep using following command on your terminal.
path ex) double_sequential/symbolic_yamls/symbolic_sweep.yaml

wandb sweep --project=SymbolicEncoding_6th --entity=clayryu path.../{yaml_name}.yaml

create instance. If you have mutiple gpus, then don't forget to add CUDA_VISIBLE_DEVICES infront of creating instances.

# Distributed Data Parallel
CUDA_VISIBLE_DEVICES=0,1 wandb agent SWEEP_ID
# Multi Instance(one model for one parameters)
CUDA_VISIBLE_DEVICES=0 wandb agent SWEEP_ID
CUDA_VISIBLE_DEVICES=1 wandb agent SWEEP_ID