Re: Apache Airflow 2.0.0 is released!

gt; customizations visible right in the Airflow UI. >> >> >> >> Our very own Jarek Potiuk has written about providers in much more detail >> <https://www.polidea.com/blog/airflow-2-providers/> on the Polidea blog. >> >> >> >> Docs on t

Re: Triage role for active contributors

;> Kamil Breguła >>> >> >> >> -- >> >> Jarek Potiuk >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >> M: +48 660 796 129 <+48660796129> >> [image: Polidea] <https://www.polidea.com/> >> >> -- Kamil Olszewski Polidea <https://www.polidea.com> | Software Engineer M: +48 503 361 783 E: kamil.olszew...@polidea.com Unique Tech Check out our projects! <https://www.polidea.com/our-work>

Re: Generic Transfer Operator

;we > > > >> >> > should > > > >> >> > > > >> > put into another AIP together with the mentioned > @spark > > > >> >decorator. > > > >> >> > > > >> > > > > >> >> > > > >> > Our proposition of transfer

Re: Generic Transfer Operator

a > > processing solution) but for the kind of quick-start scenarios I foresee > it > > might be most useful, being able to apply simple data transformation on > the > > fly by data scientist might be a big plus. > > > > Suggestion: Panda DataFrame as the format of the &qu

Generic Transfer Operator

eature to be released only after Airflow 2.0 is out. What do you think about this idea? Would you find such an operator helpful in your pipelines? Maybe you already use a similar solution or know packages that could be used to implement it? Best regards, -- Kamil Olszewski Polidea

Re: [DISCUSS] Utility DAGs

mechanism on how to run it (trigger it when disk or storage percentage reaches high value or simply periodically using cron) on every node they use. Best regards, Kamil Olszewski On 2019/04/01 09:54:15, Ash Berlin-Taylor wrote: > > For example, say we want to support retention in some A