houqp commented on PR #22698: URL: https://github.com/apache/airflow/pull/22698#issuecomment-1087910743
I remember during the initial implementation, @ashb mentioned it was introduced to reduce serialized DAG size. But good to double check with him :) > I believe this one allows non-airflow-core dependencies to be excuted in scheduler, which is something we should not really be comfortable with. Yeah, I agree with this and that's why I have the config value set to False by default. On the other hand, I think there is value in opening up this customization route for advanced users who understands the tradeoffs. In our case, this is probably the best way for us to achieve the custom scheduling logic we need without maintaining an internal fork. Our airflow instance is also fully locked down within the internal network with every code change reviewed and audited, so untrusted code execution within the scheduler process is not in our threat model :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
