celery的timezone设置对收集任务结果result_backend无效

问题描述:
【celery的timezone设置对收集任务结果result_backend无效】python版本3.8,celery版本4.3.0,在使用redis收集任务结果时,发现redis中存储的任务结果和实际运行的时间对不上。代码如下:

# app.py from celery import Celeryapp = Celery("elTask", broker="redis://:@192.168.10.21:6379/15", backend='celery.backends.redis.RedisBackend', changes={"redis_host": "192.168.10.21", "redis_db": 14, 'timezone': 'Asia/Shanghai', 'CELERY_ENABLE_UTC': False})@app.task(name="funcname") def async_funcname(): print("async function") return Trueif __name__ == '__main__': async_funcname.delay()

解决方法:
仔细研究发现celery.backends.redis.RedisBackend这个类继承的celery.backends.base.BaseKeyValueStoreBackend类中实现的_store_result函数与任务结果时间有关,源代码如下:
def _store_result(self, task_id, result, state, traceback=None, request=None, **kwargs):if state in self.READY_STATES: date_done = datetime.datetime.utcnow() else: date_done = Nonemeta = { 'status': state, 'result': result, 'traceback': traceback, 'children': self.current_task_children(request), 'task_id': bytes_to_str(task_id), 'date_done': date_done, }if request and getattr(request, 'group', None): meta['group_id'] = request.group if request and getattr(request, 'parent_id', None): meta['parent_id'] = request.parent_idif self.app.conf.find_value_for_key('extended', 'result'): if request: request_meta = { 'name': getattr(request, 'task_name', None), 'args': getattr(request, 'args', None), 'kwargs': getattr(request, 'kwargs', None), 'worker': getattr(request, 'hostname', None), 'retries': getattr(request, 'retries', None), 'queue': request.delivery_info.get('routing_key') if hasattr(request, 'delivery_info') and request.delivery_info else None }meta.update(request_meta)self.set(self.get_key_for_task(task_id), self.encode(meta)) return result

代码中date_done就是收集到任务的时间。在解决问题的过程中发现在当前最新版本4.4.6中也存在这个问题,但是问题代码不同,在4.4.6版本中是celery.backends.base.Backend中的_get_result_meta方法和任务时间相关。我的解决办法是继承重写这个方法。代码如下:
# app.py import datetime from celery import Celery from celery.utils.encoding import bytes_to_str from celery.backends.redis import RedisBackendclass RedisResult(RedisBackend): def _store_result(self, task_id, result, state, traceback=None, request=None, **kwargs):if state in self.READY_STATES: date_done = datetime.datetime.now() else: date_done = Nonemeta = { 'status': state, 'result': result, 'traceback': traceback, 'children': self.current_task_children(request), 'task_id': bytes_to_str(task_id), 'date_done': date_done, }if request and getattr(request, 'group', None): meta['group_id'] = request.group if request and getattr(request, 'parent_id', None): meta['parent_id'] = request.parent_idif self.app.conf.find_value_for_key('extended', 'result'): if request: request_meta = { 'name': getattr(request, 'task_name', None), 'args': getattr(request, 'args', None), 'kwargs': getattr(request, 'kwargs', None), 'worker': getattr(request, 'hostname', None), 'retries': getattr(request, 'retries', None), 'queue': request.delivery_info.get('routing_key') if hasattr(request, 'delivery_info') and request.delivery_info else None }meta.update(request_meta)self.set(self.get_key_for_task(task_id), self.encode(meta)) return resultapp = Celery("elTask", broker="redis://:@192.168.10.21:6379/15", backend='app.RedisResult', changes={"redis_host": "192.168.10.21", "redis_db": 14, 'timezone': 'Asia/Shanghai', 'CELERY_ENABLE_UTC': False})@app.task(name="funcname") def async_funcname(): print("async function") return Trueif __name__ == '__main__': async_funcname.delay()

这个文件名是app.py所以backend='app.RedisResult',可以视项目环境具体设定,包括4.4.6版本也可以用这个方式兼容解决。

    推荐阅读