We attempted to use zipped dags in the past and ran into a bunch of issues.
I've stopped trying to get them to work and just use a docker operator or
virtualenv operator when I need custom dependencies.
+1 on supporting dag serialization or remote fetching, would work around a
bunch of problems.
I know the code around this is pretty hacky (if use_zip_file then...
instead of an abstraction). I know when it was added it was a bit
controversial, I would be +1 on removing it. That being said I feel the
entire DAG parsing process needs to be moved to the client-side (users who
write DAGs),
General question: does anybody deploy their DAGs inside zip files?
I understand this functionality exists for historical reasons, but I’ve never
heard of anybody actually using it.
If there’s nobody, I think the DAG parsing could be simplified by removing this
functionality
