celery使用实例

一:准备工作

(1)安装虚拟环境工具

(2)创建虚拟环境:会在当前路径下创建一个虚拟环境,由于放置在conf里面,移出后再上层重新创建一个mooc的虚拟环境

(3)查看虚拟环境

(4)激活虚拟环境

(5)退出虚拟环境

(6)安装virtualenvwrapper

 

修改 ~/.bashrc,里面增加一个 source /usr/local/bin/virtualenvwrapper.sh

source ~/.bashrc 立即生效

使用virtualenvwrapper创建一个虚拟环境,其实virtualenvwrapper就是virtualenv封装的一个工具类,更加方便的通过命令,创建和执行虚拟环境

mkvirtualenv mooc_4:创建好之后,直接进入虚拟环境,不用 source xxxx/active这种方式了

deactive 退出  workon 进入虚拟环境

(7)安装pyenv

$curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash

提示:-bash: -L: command not found

环境变量没有配置好

pyenv 是python的版本管理包

这里先不修改,直接用系统的python

(8)安装celery

查看安装结果

pip3 install celery[redis]  会安装celery需要的依赖包,还会安装适配的redis

在指定文件夹下创建一个目录,用来编写代码

在celery_test下面创建一个app.py的文件

import time

def test():
   print("enter")
   time.sleep(10)
   print("haha")
   print("out")

if __name__ == "__main__":
   test()
                

执行后会发现,等待10s后才能输出 haha和out,如果我们想直接不等待10s直接执行呢?

在celery_test下再创建一个task.py的文件

import time
from celery import Celery

broker = "redis://localhost:6379/1"
backend = "redis://localhost:6379/2"
app = Celery("haha",broker=broker,backend=backend)

@app.task
def add():
    time.sleep(10)
    return 100

修改app.py

from task import add
print("enter app.py")
if __name__ == "__main__":
   result = add.delay()
   print(result)

运行app.py

打印的这个结果不是返回值100的原因是,这里打印的是将任务信息发送到redis里面的信息,启动worker后可以发现打印了内容

(mooc_4) [root@VM_0_13_centos celery_test]# celery worker -A task -l INFO
/root/.virtualenvs/mooc_4/lib/python3.7/site-packages/celery/platforms.py:801: RuntimeWarning: You're running the worker with superuser privileges: this is
absolutely not recommended!

Please specify a different user using the --uid option.

User information: uid=0 euid=0 gid=0 egid=0

  uid=uid, euid=euid, gid=gid, egid=egid,
 
 -------------- celery@VM_0_13_centos v4.4.2 (cliffs)
--- ***** ----- 
-- ******* ---- Linux-3.10.0-862.el7.x86_64-x86_64-with-centos-7.5.1804-Core 2020-04-19 21:54:43
- *** --- * --- 
- ** ---------- [config]
- ** ---------- .> app:         haha:0x7f8458755160
- ** ---------- .> transport:   redis://localhost:6379/1
- ** ---------- .> results:     redis://localhost:6379/2
- *** --- * --- .> concurrency: 1 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery
                

[tasks]
  . task.add

[2020-04-19 21:54:43,503: INFO/MainProcess] Connected to redis://localhost:6379/1
[2020-04-19 21:54:43,509: INFO/MainProcess] mingle: searching for neighbors
[2020-04-19 21:54:44,531: INFO/MainProcess] mingle: all alone
[2020-04-19 21:54:44,540: INFO/MainProcess] celery@VM_0_13_centos ready.
[2020-04-19 21:54:44,686: INFO/MainProcess] Received task: task.add[f73a4d9f-dc3a-4da8-a538-a138160b2fab]  
[2020-04-19 21:54:44,688: INFO/MainProcess] Received task: task.add[18d0feeb-6bd1-494f-bf68-27df49a2597c]  
[2020-04-19 21:54:44,689: INFO/MainProcess] Received task: task.add[72e77356-f81e-48d9-84f3-1638954d4dea]  
[2020-04-19 21:54:44,691: INFO/MainProcess] Received task: task.add[f3cd0d0b-c9cd-4fc6-9099-8f3e1e75b99b]  
[2020-04-19 21:54:44,691: INFO/MainProcess] Received task: task.add[02261dcf-9e60-49e5-8649-7eea15bc14c8]  
[2020-04-19 21:54:54,698: INFO/ForkPoolWorker-1] Task task.add[f73a4d9f-dc3a-4da8-a538-a138160b2fab] succeeded in 10.009396089008078s: 100
[2020-04-19 21:55:04,706: INFO/ForkPoolWorker-1] Task task.add[18d0feeb-6bd1-494f-bf68-27df49a2597c] succeeded in 10.006045571062714s: 100
[2020-04-19 21:55:14,717: INFO/ForkPoolWorker-1] Task task.add[72e77356-f81e-48d9-84f3-1638954d4dea] succeeded in 10.010685473913327s: 100
[2020-04-19 21:55:24,727: INFO/ForkPoolWorker-1] Task task.add[f3cd0d0b-c9cd-4fc6-9099-8f3e1e75b99b] succeeded in 10.009241034975275s: 100
[2020-04-19 21:55:34,730: INFO/ForkPoolWorker-1] Task task.add[02261dcf-9e60-49e5-8649-7eea15bc14c8] succeeded in 10.001496333163232s: 100

python3 app.py 就相当于发送了一条信息,等待消费,可以看出刚才 app.py这个文件别执行了5次,就相当于发送了5次信息,当我们启动 celery worker -A task -l INFO的时候,就相当于启动了消费者去消费刚才发布的五条信息。并将结果打印了。

(9)项目情况下的celery使用情况

celery_test

#  明天解决这个问题

# TODO

原文地址:https://www.cnblogs.com/meloncodezhang/p/12656752.html