Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
716 views
in Technique[技术] by (71.8m points)

python - Can i use Tornado+ Celery+ RabbitMQ + Redis?

For real time streaming of data on the web, I'm planning to use the Redis as my Cache data layer, where data is instantaneous. Celery is the Queue manager and RabbitMQ is the broker which queues from Redis and gets to the Tornado layer. This layer then via websockets streams to the frontend.

I have never found the Redis + RabbitMQ Combination on-line. Can someone guide with a reliable solution for the same. The problem being is such integration possible and advisable?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

I'm using Tornado and RabbitMQ (without Celery) together very successfully right now. I also did some prototyping with Redis plugged into the system as a db, as well. Your best chance at success is to find asynchronous libraries that can plug into Tornado's ioloop. For example there's tornado-redis, pika (for RabbitMQ), and tornado-celery. I should add that I've never used tornado-celery, so I'm not sure of its quality.

From a purely architectural standpoint, I don't think there's anything wrong with using both Redis and RabbitMQ together. While both can be used as message brokers, RabbitMQ isn't really appropriate for use as a cache, while Redis is well-suited for that use-case. If you were using memcached instead of Redis, it wouldn't seems very unusual at all, right?

Also (and perhaps this is the best argument I can make to suggest Redis + RabbitMQ can work), Pinterest apparently use (or at least used) both to run their site: http://blog.gopivotal.com/pivotal/case-studies-2/using-redis-at-pinterest-for-billions-of-relationships.

I found another source that says they eventually replaced Celery/RabbitMQ with another solution, but clearly it was working for them for a while.

edit: Found a Quora link from 2011 that says they were actually running Tornado as well!


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...