Re: how to set CELERY_TASK_RESULT_EXPIRES in Airflow

2020-04-01 Thread Kamil Breguła
it's a very old ticket. We have introduced mechanisms that allow for much more flexible configurations. if you need to add only one option in the Celery configuration then you can do it using the option mentioned earlier. On Wed, Apr 1, 2020 at 8:32 PM heng gu wrote: > > thanks for your quick r

Re: how to set CELERY_TASK_RESULT_EXPIRES in Airflow

2020-04-01 Thread heng gu
thanks for your quick response.  found it on [AIRFLOW-661] Celery Task Result Expiry - ASF JIRA | | | | [AIRFLOW-661] Celery Task Result Expiry - ASF JIRA | | | It looks like the fix has not been released. On Wednesday, April 1, 2020, 12:28:00 PM EDT, Kamil Breguła wrote:

Re: how to set CELERY_TASK_RESULT_EXPIRES in Airflow

2020-04-01 Thread Kamil Breguła
Hello, I am not sure, but please look at this configuration option. This is probably the solution to your problem https://airflow.readthedocs.io/en/latest/configurations-ref.html#celery-config-options Best regards, Kamil On Wed, Apr 1, 2020 at 5:43 PM heng gu wrote: > > I am using Celery Execut

how to set CELERY_TASK_RESULT_EXPIRES in Airflow

2020-04-01 Thread heng gu
I am using Celery Executor with RabbitMQ and PostgreSQL database backend. The  celery_taskmeta table never got automatically cleaned by Airflow. In Celery configuration, you can set  CELERY_TASK_RESULT_EXPIRES, but I cannot find it in airflow.cfg or in other config templates. Does anybody know ho