Miretpl commented on issue #70784: URL: https://github.com/apache/airflow/issues/70784#issuecomment-5182099197
> I think scheduler.persistence would be cleaner than workers.local.persistence. While either could work functionally, the StatefulSet and PVC are part of the scheduler workload, so organizing the setting under scheduler seems more consistent with the resource hierarchy. I'm not fully agreeing with this. The persistence thing is connected to the executor rather than the scheduler itself, even if the executor lives alongside the scheduler. Looking at that, I would say that `workers.local.persistence` is more correct, looking at the things underneath. `workers.celery` is `CeleryExecutor`, `workers.kubernetes` is `KubernetesExecutor` and `workers.local` would be e.g. `LocalExecutor`. `scheduler. persistence` abstracts executor type, which I think is important, and our user base here is mostly platform developers, DevOps people, etc. so the consistency within the values should be kept I think. Happy to hear what others think about it. > Should the new setting fall back to workers.celery.persistence for backward compatibility, or can we simply remove the existing coupling? On `main` - no, on `chart/v1-2x-test `- yes. As this changes the parameters a bit, I think it should be backported to the 1.2x line. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
