We do, but it brings a host of issues so I would not be disinclined to adjust our workflow if this functionality was removed. Our teams are very decentralized so we keep every dag in a separate git repository. During our deployment process (triggered by CI/CD) we zip everything up and put the zip file in a S3 bucket. Our airflow components sync their dag folder with this bucket. Having everything zipped up gives us some assurance that when a DAG is updated, the whole thing is replaced as a single unit.
On Sat, Jun 8, 2019 at 1:28 PM Bas Harenslak <basharens...@godatadriven.com> wrote: > General question: does anybody deploy their DAGs inside zip files? > > I understand this functionality exists for historical reasons, but I’ve > never heard of anybody actually using it. > > If there’s nobody, I think the DAG parsing could be simplified by removing > this functionality. > If there is somebody using it, could you explain why you deploy zip files > instead of “normal" files? > > Bas >
