Re: CSS issue on Airflow 1.10 Tree view UI

2018-11-12 Thread Adityan MS
Hi Ash, Here's the link to the screenshot. https://snag.gy/JUvcGy.jpg Actually, this issue is happening only on Airflow 1.10. We have the same list of tasks that is very tall (such that the page needs to be scrolled) and we were running Airflow 1.9, and it seemed totally fine. We upgrad

Re: CSS issue on Airflow 1.10 Tree view UI

2018-11-12 Thread Ash Berlin-Taylor
> On 9 Nov 2018, at 20:25, Adityan MS wrote: > > Hi, > > After upgrading to airflow 1.10, hovering the mouse over the rectangles in > tree view, causes the tooltip to popup much higher than it used to be. Any > quick fix for this behavior? In the screenshot, the right lower

CSS issue on Airflow 1.10 Tree view UI

2018-11-09 Thread Adityan MS
Hi, After upgrading to airflow 1.10, hovering the mouse over the rectangles in tree view, causes the tooltip to popup much higher than it used to be. Any quick fix for this behavior? In the screenshot, the right lowermost rectangle is the one that is being hovered over. Once you scroll down on

Re: Question on Running Airflow 1.10 in Kubernetes

2018-10-14 Thread Pramiti Goel
in a > > single day... > > > > Get Outlook for Android<https://aka.ms/ghei36> > > > > ____ > > From: Daniel Imberman > > Sent: Sunday, October 14, 2018 8:41:58 AM > > To: dev@airflow.incubator.apache.org > > Subje

Re: Question on Running Airflow 1.10 in Kubernetes

2018-10-14 Thread Michael Ghen
t; > I'm curious about your use cases that require multiple deployments in a > single day... > > Get Outlook for Android<https://aka.ms/ghei36> > > > From: Daniel Imberman > Sent: Sunday, October 14, 2018 8:41:58 AM > To: dev

Re: Question on Running Airflow 1.10 in Kubernetes

2018-10-14 Thread Jeff Payne
quire multiple deployments in a single day... Get Outlook for Android<https://aka.ms/ghei36> From: Daniel Imberman Sent: Sunday, October 14, 2018 8:41:58 AM To: dev@airflow.incubator.apache.org Subject: Re: Question on Running Airflow 1.10 in Kubernetes

Re: Question on Running Airflow 1.10 in Kubernetes

2018-10-14 Thread Daniel Imberman
8, 3:56 AM Pramiti Goel wrote: > Hi, > > We are trying to run airflow 1.10 in kubernetes. > 1) We are running our scheduler, worker and webserver service in individual > containers. > 2) We are using docker image which has airflow 1.10, python 3.x. We are > deploying our dags

Question on Running Airflow 1.10 in Kubernetes

2018-10-14 Thread Pramiti Goel
Hi, We are trying to run airflow 1.10 in kubernetes. 1) We are running our scheduler, worker and webserver service in individual containers. 2) We are using docker image which has airflow 1.10, python 3.x. We are deploying our dags in docker image. With above architecture of airflow setup in

Re: Airflow 1.10 Migration Duration

2018-09-28 Thread Kevin Yang
rflow.com/users/149428/taylor-edmiston> | Developer > > > Story > > > > <https://stackoverflow.com/story/taylor> > > > > > > > > > > > > On Tue, Sep 25, 2018 at 8:30 PM, Sid Anand > wrote: > > > > > > > > &

Re: Airflow 1.10 Migration Duration

2018-09-27 Thread Taylor Edmiston
; On Tue, Sep 25, 2018 at 8:30 PM, Sid Anand wrote: > > > > > > > I checked with our Ops guy and he mentioned that when he upgraded > from > > > > 1.8.x to 1.9.x, it took a few seconds. We had 3M rows in the > > > task_instance > > > > table

Re: Airflow 1.10 Migration Duration

2018-09-25 Thread Matt Davis
ps://stackoverflow.com/story/taylor> > > > > > > On Tue, Sep 25, 2018 at 8:30 PM, Sid Anand wrote: > > > > > I checked with our Ops guy and he mentioned that when he upgraded from > > > 1.8.x to 1.9.x, it took a few seconds. We had 3M rows in the > >

Re: Airflow 1.10 Migration Duration

2018-09-25 Thread Ruiqin Yang
We had 3M rows in the > task_instance > > table and run MySQL 5.7. > > > > -s > > > > On Tue, Sep 25, 2018 at 4:54 PM Matt Davis wrote: > > > > > Hi folks, > > > > > > Here at Clover we're excitedly migrating to Airflow 1.10 (

Re: Airflow 1.10 Migration Duration

2018-09-25 Thread Taylor Edmiston
e, Sep 25, 2018 at 4:54 PM Matt Davis wrote: > > > Hi folks, > > > > Here at Clover we're excitedly migrating to Airflow 1.10 (thanks for > > everyone's hard work on that!). We're finding that it's taking about 2 > > hours to apply all the migrations

Re: Airflow 1.10 Migration Duration

2018-09-25 Thread Sid Anand
I checked with our Ops guy and he mentioned that when he upgraded from 1.8.x to 1.9.x, it took a few seconds. We had 3M rows in the task_instance table and run MySQL 5.7. -s On Tue, Sep 25, 2018 at 4:54 PM Matt Davis wrote: > Hi folks, > > Here at Clover we're excitedly migrat

Airflow 1.10 Migration Duration

2018-09-25 Thread Matt Davis
Hi folks, Here at Clover we're excitedly migrating to Airflow 1.10 (thanks for everyone's hard work on that!). We're finding that it's taking about 2 hours to apply all the migrations to go from Airflow 1.8 to 1.10, largely driven by the 10 million rows in our task_instanc

Re: Regarding airflow 1.10

2018-08-22 Thread Daniel Imberman
Hi Hemanth, Fokko and Taylor have covered this question but just cc'ing myself in case you have any further questions about the k8s executor :). On Wed, Aug 22, 2018 at 12:10 PM Taylor Edmiston wrote: > Hemanth - > > To add to what Fokko mentioned... > > You can find the KubernetesExecutor imp

Re: Regarding airflow 1.10

2018-08-22 Thread Taylor Edmiston
Hemanth - To add to what Fokko mentioned... You can find the KubernetesExecutor implementation in KubernetesExecutor and AirflowKubernetesScheduler

Re: Regarding airflow 1.10

2018-08-21 Thread Driesprong, Fokko
Hi Hemanth, Thanks for the question. There is the KubernetesExecutor which just uses a container with Airflow to schedule a task on the kubernetes cluster. This container will then execute a single task, and then the Executor checks the exitcode of the container. There is also the KubernetesOpera

Regarding airflow 1.10

2018-08-21 Thread Musunuru, Hemanth
Hi Team, Does Kubernetes executor in 1.10 create new pod for every task ? (Which means won’t there be any active workers running except for new Tasks) We are trying to containerize our existing to airflow environment, your reply Will help us to understand it more. Thanks in Advance . Thanks

Re: Airflow 1.10 Release Date

2018-04-11 Thread Naik Kaxil
Op 11 apr. 2018 om 11:01 heeft Naik Kaxil het volgende geschreven: > > Hi, > > Me & my friend at going to present on Apache Airflow on Cloud at PyData London this month. > > Do we have a release data for Airflow 1.10 that I can

Re: Airflow 1.10 Release Date

2018-04-11 Thread Bolke de Bruin
Bolke Verstuurd vanaf mijn iPad > Op 11 apr. 2018 om 11:01 heeft Naik Kaxil het volgende > geschreven: > > Hi, > > Me & my friend at going to present on Apache Airflow on Cloud at PyData > London this month. > > Do we have a release data for Airflow 1.10 that

Airflow 1.10 Release Date

2018-04-11 Thread Naik Kaxil
Hi, Me & my friend at going to present on Apache Airflow on Cloud at PyData London this month. Do we have a release data for Airflow 1.10 that I can announce there and features it might include? Regards, Kaxil Kaxil Naik Data Reply 38 Grosvenor Gardens London SW1W 0EB - UK phone: +4

Starting the Airflow 1.10 release cycle

2018-04-06 Thread Bolke de Bruin
Hi All, Fokko and I are picking up the release management role for the 1.10 release. The aim of this release is to make sure we are, from the perspective of a release, graduation ready. So license compliancy is paramount and proper publishing of security vulnerabilities. It should also be our l

Re: Airflow 1.10

2018-03-28 Thread Naik Kaxil
gt; > a > > > > > > few places, but still there is some work to be done. For example, > > > don't > > > > > > allocate an executor slot to the sensors, but have a more > > > sophisticated > > > > > way > >

Re: Airflow 1.10

2018-03-27 Thread Feng Lu
et this in. At my current project we use > > sensors > > > > in > > > > > a > > > > > > few places, but still there is some work to be done. For example, > > > don't > > > > > > allocate an executor slot to the sensors, but have a mor

Re: Airflow 1.10

2018-03-26 Thread Milan van der Meer
> in > > > > a > > > > > few places, but still there is some work to be done. For example, > > don't > > > > > allocate an executor slot to the sensors, but have a more > > sophisticated > > > > way > > > > > of poking. > &g

Re: Airflow 1.10

2018-03-26 Thread Bolke de Bruin
gt;>>> few places, but still there is some work to be done. For example, >> don't >>>>> allocate an executor slot to the sensors, but have a more >> sophisticated >>>> way >>>>> of poking. >>>>> >>>>> Ch

Re: Airflow 1.10

2018-03-26 Thread Chris Riccomini
> > > > > > > > > > > > 2018-01-12 21:19 GMT+01:00 Chris Riccomini : > > > > > > > >> Just the operator (AIRFLOW-1517) > > > >> > > > >> On Fri, Jan 12, 2018 at 11:21 AM, Anirudh Ramanathan < > > > >> ramanath...

Re: Airflow 1.10

2018-01-16 Thread George Leslie-Waksman
t; > > >> ramanath...@google.com.invalid> wrote: > > >> > > >>> Sounds awesome. Is k8s support here referring to both the executor > and > > >> the > > >>> operator? > > >>> > > >>> Thanks, > >

Re: Airflow 1.10

2018-01-16 Thread Joy Gao
ds awesome. Is k8s support here referring to both the executor and > >> the > >>> operator? > >>> > >>> Thanks, > >>> > >>> > >>> On Jan 12, 2018 11:18 AM, "Sid Anand" wrote: > >>> > >>>>

Re: Airflow 1.10

2018-01-14 Thread Bolke de Bruin
>>>> +1 >>>> >>>> >>>> On Fri, Jan 12, 2018 at 10:56 AM, Chris Riccomini < >> criccom...@apache.org >>>> >>>> wrote: >>>> >>>>> Hey all, >>>>> >>>>> Aft

Re: Airflow 1.10

2018-01-14 Thread Driesprong, Fokko
; > > On Fri, Jan 12, 2018 at 10:56 AM, Chris Riccomini < > criccom...@apache.org > > > > > > wrote: > > > > > > > Hey all, > > > > > > > > After some past discussion on Airflow 1.10 vs 2.0, I think we've > > > conve

Re: Airflow 1.10

2018-01-12 Thread Chris Riccomini
quot;Sid Anand" wrote: > > > +1 > > > > > > On Fri, Jan 12, 2018 at 10:56 AM, Chris Riccomini > > > wrote: > > > > > Hey all, > > > > > > After some past discussion on Airflow 1.10 vs 2.0, I think we've > > con

Re: Airflow 1.10

2018-01-12 Thread Anirudh Ramanathan
Sounds awesome. Is k8s support here referring to both the executor and the operator? Thanks, On Jan 12, 2018 11:18 AM, "Sid Anand" wrote: > +1 > > > On Fri, Jan 12, 2018 at 10:56 AM, Chris Riccomini > wrote: > > > Hey all, > > > > After some p

Re: Airflow 1.10

2018-01-12 Thread Sid Anand
+1 On Fri, Jan 12, 2018 at 10:56 AM, Chris Riccomini wrote: > Hey all, > > After some past discussion on Airflow 1.10 vs 2.0, I think we've converged > on a 1.10 as the next step. 1.10 will include: > > * Timezone changes > * Kubernetes support > * New UI > >

Airflow 1.10

2018-01-12 Thread Chris Riccomini
Hey all, After some past discussion on Airflow 1.10 vs 2.0, I think we've converged on a 1.10 as the next step. 1.10 will include: * Timezone changes * Kubernetes support * New UI The first two have been merged in, as I saw Bolke just merged K8s (I saw a few follow-on patches coming, t