Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
696 views
in Technique[技术] by (71.8m points)

python - How do I tell celery worker to stop accepting tasks? How can I check if any celery worker tasks are running?

The scenario:

  • System running on a server consisting of a Python/Flask web application and background tasks using Celery
  • Both web application and celery workers are run as upstart jobs (Web app behind Nginx)
  • Deployment to production is done with a script that:

    • Stop the upstart jobs
    • Push code to server
    • Run any db migrations
    • Start the upstart jobs

How can I enhance the deployment script so it does the following?:

  • Tell the celery worker to stop accepting tasks
  • Wait until any currently running celery tasks are finished
  • Stop the upstart jobs
  • Push code to server
  • Run any db migrations
  • Start the upstart jobs
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

The following script, run as part of the deployment solved the problem:

import time
from celery.app.control import Control
from myapp.tasks import celery # my application's Celery app

if __name__ == "__main__":
    control = Control(celery)
    control.cancel_consumer("celery") # queue name, must probably be specified once per queue, but my app uses a single queue

    inspect = control.inspect()
    while True:
        active = inspect.active()
        running_jobs = []
        for key, value in active.items():
            running_jobs.extend(value)
        if len(running_jobs) > 0:
            print("{} jobs running: {}".format(len(running_jobs), ", ".join(job["name"] for job in running_jobs)))
            time.sleep(10)
        else:
            print("No running jobs")
            break

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...