Re: Question regarding data usage in the DAG itself

2020-10-04 Thread Chen Michaeli
an expected > behavior :) > > Cheers, > Tomek > > On Mon, Sep 21, 2020 at 11:23 PM Chen Michaeli wrote: > >> Hello, I am using Apache Airflow for my fun and experience and it is >> great! >> I hope I was meant to send questions to this address, please correct me

Re: Question regarding data usage in the DAG itself

2020-09-22 Thread Chen Michaeli
if you will request an > external API or database on DAG level (not in operator) it means that the > request will be send quite often and that's definitely not an expected > behavior :) > > Cheers, > Tomek > > On Mon, Sep 21, 2020 at 11:23 PM Chen Michaeli wrote: >

Question regarding data usage in the DAG itself

2020-09-21 Thread Chen Michaeli
Hello, I am using Apache Airflow for my fun and experience and it is great! I hope I was meant to send questions to this address, please correct me if I'm wrong. I was wondering why I shouldn't let the DAG itself do any data gathering? For example and for the sake of simplicity, I have a pipeline