打开nccl的错误信息：
export NCCL_DEBUG=WARN
错误1.docker容器内运行pytorch多gpu报错 RuntimeError: NCCL Error 2: unhandled system error
在启动容器的时候加上 -e NVIDIA_VISIBLE_DEVICES=0,1,2,3
docker run --runtime=nvidia --net="host" -e NVIDIA_VISIBLE_DEVICES=0,1,2,3 --shm-size 8g -it huangzc/reid:v1 /bin/bash
错误2.RuntimeError: DataLoader worker (pid 53617) is killed by signal: Bus error. It is possible that dataloader's workers are out of shared memory. Please try to raise your shared memory limit.
启动容器的时候增加交换内存 --shm-size 8g