基于TensorRT的YOLO（V345）模型部署《方案二》

之前写过一个部署方案，即：基于TensorRT的YOLO（V345）模型部署《方案一》，以下链接

https://www.cnblogs.com/winslam/p/13816143.html

今天尝试的是另一个开源方案，部署环境和之前一样（实际上是我懒得改环境，一波三折，好在测试通过）

先把项目下载下来：

https://github.com/wang-xinyu/tensorrtx

打开D: ensorrtx-masteryolov5 准备用cmake生成vs2017工程，修改CMakeLists.txt文件如下：

 1 cmake_minimum_required(VERSION 2.6)
 2 
 3 project(yolov5) # 1
 4 set(OpenCV_DIR "D:\Program Files (x64)\opencv412\opencv\build")  #2 # 这里之前使用的是opencv with contrib，报错，所以改为原版
 5 set(TRT_DIR "D:\TensorRT-7.1.3.4")  #3
 6 
 7 add_definitions(-std=c++11)
 8 option(CUDA_USE_STATIC_CUDA_RUNTIME OFF)
 9 set(CMAKE_CXX_STANDARD 11)
10 set(CMAKE_BUILD_TYPE Debug)
11 
12 set(THREADS_PREFER_PTHREAD_FLAG ON)
13 find_package(Threads)
14 
15 # setup CUDA
16 find_package(CUDA REQUIRED)
17 message(STATUS "    libraries: ${CUDA_LIBRARIES}")
18 message(STATUS "    include path: ${CUDA_INCLUDE_DIRS}")
19 
20 include_directories(${CUDA_INCLUDE_DIRS})
21 
22 set(CUDA_NVCC_PLAGS ${CUDA_NVCC_PLAGS};-std=c++11; -g; -G;-gencode; arch=compute_75;code=sm_75)
23 ####
24 enable_language(CUDA)  # add this line, then no need to setup cuda path in vs
25 ####
26 include_directories(${PROJECT_SOURCE_DIR}/include)
27 include_directories(${TRT_DIR}\include)
28 
29 # -D_MWAITXINTRIN_H_INCLUDED for solving error: identifier "__builtin_ia32_mwaitx" is undefined
30 set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wall -Ofast -D_MWAITXINTRIN_H_INCLUDED")
31 
32 # setup opencv
33 find_package(OpenCV QUIET
34     NO_MODULE
35     NO_DEFAULT_PATH
36     NO_CMAKE_PATH
37     NO_CMAKE_ENVIRONMENT_PATH
38     NO_SYSTEM_ENVIRONMENT_PATH
39     NO_CMAKE_PACKAGE_REGISTRY
40     NO_CMAKE_BUILDS_PATH
41     NO_CMAKE_SYSTEM_PATH
42     NO_CMAKE_SYSTEM_PACKAGE_REGISTRY
43 )
44 
45 message(STATUS "OpenCV library status:")
46 message(STATUS "    version: ${OpenCV_VERSION}")
47 message(STATUS "    libraries: ${OpenCV_LIBS}")
48 message(STATUS "    include path: ${OpenCV_INCLUDE_DIRS}")
49 
50 include_directories(${OpenCV_INCLUDE_DIRS})
51 link_directories(${TRT_DIR}\lib)
52 
53 #add_executable(yolov5 ${PROJECT_SOURCE_DIR}/yolov5.cpp ${PROJECT_SOURCE_DIR}/yololayer.cu ${PROJECT_SOURCE_DIR}/yololayer.h 
54 #                ${PROJECT_SOURCE_DIR}/hardswish.cu ${PROJECT_SOURCE_DIR}/hardswish.h)   #4 # 红色的删除了，已经改为56行
55 
56 add_executable(yolov5 ${PROJECT_SOURCE_DIR}/yolov5.cpp ${PROJECT_SOURCE_DIR}/yololayer.cu ${PROJECT_SOURCE_DIR}/yololayer.h )   #4
57 
58 target_link_libraries(yolov5  "nvinfer" "nvinfer_plugin")   #5
59 target_link_libraries(yolov5 ${OpenCV_LIBS})          #6
60 target_link_libraries(yolov5 ${CUDA_LIBRARIES})   #7
61 target_link_libraries(yolov5 Threads::Threads)       #8

configure、generate之后，用ide打开，载入cuda、tensorrt的属性表，

在yolov5.cpp中指定具体模型文件路径（文件应该是原本yolov5模型转换后的文件？）：

std::map<std::string, Weights> weightMap = loadWeights("D:\tensorrtx-master\yolov5\build\yolov5s.wts");

然后在release x64模式下编译成功。

（编译可参考：https://github.com/wang-xinyu/tensorrtx/blob/master/tutorials/run_on_windows.md 记得还有个include头文件下载）

接下来在powershell上操作

生成“.engin”文件

PS D:	ensorrtx-masteryolov5uildRelease> .yolov5.exe  -s
Loading weights: D:	ensorrtx-masteryolov5uildyolov5s.wts 
Building engine, please wait for a while... 
[10/15/2020-16:02:24] [W] [TRT] Half2 support requested on hardware without native FP16 support, performance will be negatively affected.
Build engine successfully!

测试，提前将图片放在新建的samples文件夹下，复制到源码根目录：

PS D:	ensorrtx-masteryolov5uildRelease> .yolov5.exe  -d D:	ensorrtx-mastersamples
181ms
11ms
11ms                                                                                                                                                                       
11ms

效果图在exe所在文件夹。

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------

在最后，作者提供了另一个测试环境的方式，顺便也是解释整个模型部署的流程，

链接：https://github.com/wang-xinyu/pytorchx

我们将项目下载下来，放在D: ensorrtx-master下，

进入测试项目：D: ensorrtx-masterpytorchx-master

进入lenet文件夹

在python环境执行：

python lenet5.py   
python inference.py

打印以下内容正常：

接着在C++环境测试（理论输出结果和上图一样）

进入D: ensorrtx-masterlenet，拷贝其中C++源码新建工程，我直接建在这里了（懒）

和上篇博客一样加载cuda、tensorrt的属性表

将 216行的

 std::ofstream p("lenet5.engine");

改为：

 std::ofstream p("lenet5.engine", std::ios::binary);

（参考：https://github.com/wang-xinyu/tensorrtx/issues/25）

修改76行，指定具体路径

std::map<std::string, Weights> weightMap = loadWeights("D:\tensorrtx-master\pytorchx-master\lenet\lenet5.wts");

指定以下指令

PS D:	ensorrtx-masteryolov5uildx64Release> .lenet.exe -d   
 Output: 
0.0949623, 0.0998472, 0.110072, 0.0975036, 0.0965564, 0.109736, 0.0947979, 0.105618, 0.099228, 0.0916792,

这里的输出和上图差不多，是正常的，模型转换成功

参考：https://github.com/wang-xinyu/tensorrtx/blob/master/tutorials/run_on_windows.md