Introduction

ModelPack generates training logs in TensorBoard format. To save logs we need to specify --logs parameter. This parameter should contain a valid path within the system with where the write permission permission is already set. 

Note: if this parameter is not properly set, ModelPack will automatically save logs and checkpoints within the ./out folder relative to the current directory. When running ModelPack from the docker container this might be a problem since the folder will be created inside the container. 

Invoking TensorBoard

To visualize the evolution of the training process it is needed to call tensorboard in the following way.

sudo tensorboard --logdir logs/ --port 9999 --host 0.0.0.0

To see the charts and logs go to HTTP://host-pc:9999 

To visualize validation images with bounding boxes, add the --display parameter to the training command and assign a positive value to it. This will make the training process a bit slower and logs files bigger