scrapyd-deploy 把scrapy项目打包上传部署到scrapyd服务端报错 Deploy failed (500)

Outline

在把scrapy任务部署到scrapyd服务上时，遇到问题一直不成功：

报错如下：

（Deploy failed (500):，部署失败）

scrapyd-deploy muji_data_python_spider -p muji_data_python_spider

Packing version 1617328541
scrapyd-deploy.py:259: ScrapyDeprecationWarning: Call to deprecated function retry_on_eintr.
  retry_on_eintr(check_call, [sys.executable, 'setup.py', 'clean', '-a', 'bdist_egg', '-d', d], stdout=o, stderr=e)
Deploying to project "muji_data_python_spider" in http://192.168.150.154:6800/addversion.json
Deploy failed (500):
<html><head><title>Processing Failed</title></head><body><b>Processing Failed</b></body></html>

Fix

经过查询发现问题原因是：

早于18.9的Twisted版本不支持python-3.7，而我当前的Python版本是3.7的，且twisted版本小于18.9；

如果使用的是python-3.7版本，并且twisted版本低于18.9，可以将twisted版本升至大于等于18.9。

eg：

pip install -U Twisted==18.9.0

修改twisted版本后，问题得到解决：

scrapyd-deploy muji_data_python_spider -p muji_data_python_spider

Packing version 1617328797
scrapyd-deploy.py:259: ScrapyDeprecationWarning: Call to deprecated function retry_on_eintr.
  retry_on_eintr(check_call, [sys.executable, 'setup.py', 'clean', '-a', 'bdist_egg', '-d', d], stdout=o, stderr=e)
Deploying to project "muji_data_python_spider" in http://192.168.150.154:6800/addversion.json
Server response (200):
{"node_name": "DESKTOP-8ACKO7I", "status": "ok", "project": "muji_data_python_spider", "version": "1617328797", "spiders": 24}