I'm really excited about this feature, and I'd love to be able to provide
feedback on the proposed design.
On Thu, Jul 18, 2019 at 10:21 AM Tao Feng wrote:
> Thanks Ash. This will be huge!
>
> On Thu, Jul 18, 2019 at 4:00 AM Jarek Potiuk
> wrote:
>
> > Cool!
> >
> > On Thu, Jul 18, 2019 at 11:4
+1 to a serialization scheme. I'm happy to give early feedback.
On Mon, Jun 10, 2019 at 8:34 AM Dan Davydov
wrote:
> I know the code around this is pretty hacky (if use_zip_file then...
> instead of an abstraction). I know when it was added it was a bit
> controversial, I would be +1 on removing
> Thanks.
>
> Feng
>
> *From: *Gabriel Silk
> *Date: *Fri, May 10, 2019 at 10:15 AM
> *To: *
> *Cc: *Griselda Cuevas, Leah Cole, Trishka Fernandes
>
> Hey Feng,
> >
> > I'm the tech lead on Data Infrastructure at Dropbox, and I'd like to talk
>
Hey Feng,
I'm the tech lead on Data Infrastructure at Dropbox, and I'd like to talk
about the challenges and opportunities in migrating large-scale ETL to
Airflow from a system that was built in-house.
On Fri, May 10, 2019 at 8:23 AM Szymon Przedwojski <
szymon.przedwoj...@polidea.com> wrote:
>
Nice work!
On Thu, Apr 11, 2019 at 9:21 AM Sid Anand wrote:
> Woot! Piling on with the kudos to Ash!
>
> -s
>
> On Thu, Apr 11, 2019 at 6:55 AM Jarek Potiuk
> wrote:
>
> > Great work Ash!
> >
> > On Thu, Apr 11, 2019 at 10:19 AM Sumit Maheshwari <
> sumeet.ma...@gmail.com>
> > wrote:
> >
> > >
>
> A lot of the problems that Quantopian experiences with Airflow can't be
> tackled without either "hacks" on top of Airflow; or deep reworkings of
> Airflow components. But that kind of rework is very challenging to
> implement with the current Airflow contribution process.
Can you elaborate o
Two questions:
1) Are you eventually seeing the full log for the task, after it finishes?
2) Are you using S3 to store your logs?
On Thu, Feb 14, 2019 at 11:53 AM Dan Stoner wrote:
> More info!
>
> It appears that the Celery executor will silently fail if the
> credentials to a postgres results_
The main concern I have with FK's is the potential performance impact.
When evaluating Airflow for use at Dropbox, I ran benchmarks that showed
several bottlenecks in the query patterns being used by the scheduler,
which led me to open https://issues.apache.org/jira/browse/AIRFLOW-2430 and
submit
My two cents:
"execution_date" is definitely confusing to newcomers, and it's partly the
ambiguity of the wording, and partly the UI's fault. When I first saw
execution date, I assumed it meant *the earliest time at which the task
will execute*, which is wrong. I was confused when no tasks appeare
