Re: [VOTE] Labelling scheme for the upcoming official docker images of Airflow

+1 (binding) On Thu, Jun 27, 2019 at 8:31 AM Kaxil Naik wrote: > > +1 (binding). Really appreciate all the effort you have been putting into > this Jarek. > > Regards, > Kaxil > > On Thu, Jun 27, 2019 at 11:38 AM Deng Xiaodong wrote: > > > +1 (binding) > > > > Thanks for initiating the discussio

Re: [VOTE] Labelling scheme for the upcoming official docker images of Airflow

+1 (binding). Really appreciate all the effort you have been putting into this Jarek. Regards, Kaxil On Thu, Jun 27, 2019 at 11:38 AM Deng Xiaodong wrote: > +1 (binding) > > Thanks for initiating the discussion and the voting. > > XD > > On Thu, Jun 27, 2019 at 12:57 Jarek Potiuk > wrote: > >

Re: [VOTE] Labelling scheme for the upcoming official docker images of Airflow

+1 (binding) Thanks for initiating the discussion and the voting. XD On Thu, Jun 27, 2019 at 12:57 Jarek Potiuk wrote: > Hello Airflow community, > > This email call for a vote on "Labelling scheme" for the docker images we > are going to publish at > https://cloud.docker.com/u/apache/reposito

Re: SLA semantics

On Wed, Jun 26, 2019 at 9:45 PM Gerard Toonstra wrote: > > That's not my experience of how SLA's work at the moment. I've observed > this to currrently work as: > > 1. An SLA is configured as the "time delta" after some dag execution > schedule. > 2. The SLA is configured at task level, so any ta

Re: [VOTE] Labelling scheme for the upcoming official docker images of Airflow

+1 (non-binding) Great to see we have production Airflow images. Many Airflow users will choice Puckel’s if in Docker environment, And will disappointed cause some bug in the docker image. Best Wish — Jiajie On Jun 27, 2019, at 13:03, Christian Lellmann mailto:christian.lellm...@googlemail.

Re: [VOTE] Labelling scheme for the upcoming official docker images of Airflow

+1 (non-binding) Regards Chris Jarek Potiuk schrieb am Do., 27. Juni 2019, 06:57: > Hello Airflow community, > > This email call for a vote on "Labelling scheme" for the docker images we > are going to publish at > https://cloud.docker.com/u/apache/repository/docker/apache/airflow. The > vote

[VOTE] Labelling scheme for the upcoming official docker images of Airflow

Hello Airflow community, This email call for a vote on "Labelling scheme" for the docker images we are going to publish at https://cloud.docker.com/u/apache/repository/docker/apache/airflow. The vote will lats for at least 1 week (July 4th, 7am CEST) and at least three +1 (binding) votes have been

Re: Travis builds in a queue for hours

Yep. That's what I suggested as the reason in the ticket - I guess INFRA are the only people who can do anything about it (increase concurrency ? pay more for Travis :)? ). On Wed, Jun 26, 2019 at 9:51 PM Ash Berlin-Taylor wrote: > I asked Travis on twitter and they said it was due to the Apache

Re: Travis builds in a queue for hours

I asked Travis on twitter and they said it was due to the Apache other projects build queues https://twitter.com/travisci/status/1143893051460526080 -ash On 26 June 2019 20:48:33 BST, Jarek Potiuk wrote: >Hello everyone, > >For the last few days the Travis builds for apache/airflow project are

Travis builds in a queue for hours

Hello everyone, For the last few days the Travis builds for apache/airflow project are waiting in a queue for hours. This is not a normal situation. I've opened INFRA ticket for that: https://issues.apache.org/jira/browse/INFRA-18657 J. -- Jarek Potiuk Polidea | Prin

Re: SLA semantics

That's not my experience of how SLA's work at the moment. I've observed this to currrently work as: 1. An SLA is configured as the "time delta" after some dag execution schedule. 2. The SLA is configured at task level, so any tasks still running or need to run after "time delta" will be aggregated

SLA semantics

Hi all, I'm looking to get some clarity on the intended behavior for SLAs. This has come up several times in the past, but as far as I can tell there hasn't been a definitive answer. As pointed out in https://issues.apache.org/jira/browse/AIRFLOW-249 (open for several years now): the SLA logi

Re: Kubernetes executor production readiness?

Once Daniel Imberman is back from his vacation, I do know he’s planning to jump into that JIRA issue to see what is going on. -Ry On Wednesday, June 26, 2019, Austin Weaver wrote: > We attempted to migrate our existing production airflow (celery executor) > to k8 executor, and saw many issues s

Re: Kubernetes executor production readiness?

We attempted to migrate our existing production airflow (celery executor) to k8 executor, and saw many issues scaling tasks ie. Couldn't run 100+ tasks concurrently, as described here https://issues.apache.org/jira/browse/AIRFLOW-4346 . Although there has not been much movement on the ticket, I am

Re: XCOM metadata behaviour in case of task retry

Hey, That's what is in airflow code, yes. Cheers, E On Wed, Jun 26, 2019 at 1:13 PM guptakumarta...@gmail.com < guptakumarta...@gmail.com> wrote: > Hi Team, > When any task goes into retry state, it is observed that the entries from > the xcom table are removed with respect to that task id. Is

XCOM metadata behaviour in case of task retry

Hi Team, When any task goes into retry state, it is observed that the entries from the xcom table are removed with respect to that task id. Is this the expected behaviour ? Thanks & Regards, Tanuj Gupta

Re: 1.10.4beta2 "snapshot" for available testing

This line is empty in V10 test and V10 stable as well :) not sure what's going on. I'll try to reinstall and check this again Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Wednesday, June 26, 2019 1:39 PM, Ash Berlin-Taylor wrote: > Are you sure you are on 1.10.4b2? The

Re: 1.10.4beta2 "snapshot" for available testing

Are you sure you are on 1.10.4b2? The stack trace points at line 113 which in 1.10.4b2 is an empty line https://github.com/apache/airflow/blob/1.10.4b2/airflow/www/views.py#L113 The stack trace matches 1.10.3 though h

Re: 1.10.4beta2 "snapshot" for available testing

http://localhost:8080/admin/dagmodel/ Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Wednesday, June 26, 2019 1:32 PM, Ash Berlin-Taylor wrote: > What URL does that end up on? I can't reproduce that. > > -a > > > On 26 Jun 2019, at 10:52, airflowuser airflowu...@protonm

Re: 1.10.4beta2 "snapshot" for available testing

What URL does that end up on? I can't reproduce that. -a > On 26 Jun 2019, at 10:52, airflowuser > wrote: > > Bug: > On the old UI: > 1. Click on edit DAG > 2. Enter Description > 3.Click Save > > You'll get error: > Traceback (most recent call last): > File "/usr/local/lib/python3.6/site-pa

Re: Field "owner" in DAG Model

That was broken in 1.10.3 - should be fixed in .4 > On 26 Jun 2019, at 11:00, airflowuser > wrote: > > If I may ask... > in the Old UI how do you filter DAGs by owner? > I'm running 1.10.3 and in the Search bar it searches only DAG it doesn't > search by owner. > > > Sent with ProtonMail Sec

Re: Field "owner" in DAG Model

Sure thing :) Please refer to https://airflow.readthedocs.io/en/1.10.3post1/security.html?highlight=filter_by_owner#multi-tenancy I didn’t use this feature myself though. XD On Wed, Jun 26, 2019 at 18:00 airflowuser wrote: > If I may ask... > in the Old UI how do you filter DAGs by owner? >

Re: Field "owner" in DAG Model

If I may ask... in the Old UI how do you filter DAGs by owner? I'm running 1.10.3 and in the Search bar it searches only DAG it doesn't search by owner. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Wednesday, June 26, 2019 12:27 PM, Deng Xiaodong wrote: > Hi folks, >

Re: 1.10.4beta2 "snapshot" for available testing

Bug: On the old UI: 1. Click on edit DAG 2. Enter Description 3.Click Save You'll get error: Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2311, in wsgi_app response = self.full_dispatch_request() packages/flask_admin/model/base.p

Re: Field "owner" in DAG Model

I would be in favor of keeping it and implementing the groups permissions. As we expand our use of airflow it would be very helpful to have this as an option. James Coder > On Jun 26, 2019, at 5:27 AM, Deng Xiaodong wrote: > > Hi folks, > > In DAG Model, we have "owner" field. In earlier Fl

Re: Kubernetes executor production readiness?

If tasks are short, the k8s startup/shutdown time can be a negative factor. The nice thing about k8s executor is that you can redeploy airflow without the need to drain the system to ensure work won’t be affected. Also, going all k8s gives you autoscaling. If you’d like to try our both, you can

Field "owner" in DAG Model

Hi folks, In DAG Model, we have "owner" field. In earlier Flask-Admin based UI (which is already removed in master branch), it was used by "filter_by_user" config item in [webserver] section to help control access. But now seems it's not used anywhere already (correct me if I'm wrong). I would li

Re: Tagging of the airflow images

You can start the vote, don't think only PMC needs to do this. On Wed, Jun 26, 2019, 12:10 Jarek Potiuk wrote: > Hello everyone (other committers especially) - do you think we need formal > voting on this? If yes, can I start it, or do we need someone from PMC :)? > > Since I am in the last stag

Re: 1.10.4beta2 "snapshot" for available testing

Because the list was autogenerated from the issue type in the jira ticket, and I didn't check all 200+ items in the change log :) -a > On 25 Jun 2019, at 18:59, airflowuser > wrote: > > Why [AIRFLOW-1684] - Branching based on XCom variable (Docs) (#4365) listed > under features? > > > Sen