Re: Another Milestone for Airflow & the community

Excellent news, really glad to hear this! 🎉 Thank you for sharing! Cheers, Kevin Y On Wed, Feb 1, 2023 at 11:03 PM Pierre Jeambrun wrote: > This is a very exciting news! > > On Thu 2 Feb 2023 at 06:31, Ping Zhang wrote: > >> Wow, that is great news! Thanks for sharing. >> >> Thanks, >> >> Pin

Re: [VOTE] AIP-52 Automatic setup and teardown tasks

+1 (binding) On Mon, Jan 9, 2023 at 1:41 PM Felix Uellendall wrote: > +1 (binding) > > -Felix > > > Sent from Proton Mail for iOS > > > On Mon, Jan 9, 2023 at 22:05, Pierre Jeambrun > wrote: > > +1 (binding) > > Le lun. 9 janv. 2023 à 21:12, Vikram Koka > a écrit : > >> +1 binding >> >> Vikram

Re: [VOTE] Release Airflow 2.3.0 from 2.3.0rc2

ready are) helping in explaining how they found the > Tree View missing and what they miss in the Grid View. > > From the current discussion it's really clear that two things are missing: > https://github.com/apache/airflow/discussions/23413: > > * expanding subtrees of groups

Re: [VOTE] Release Airflow 2.3.0 from 2.3.0rc2

gt;> >> The biggest problem with Tree view was that DAG versioning effort was (I >> believe) held back by it because it was next to impossible to think of >> "versioning" when Tree view was there. >> >> This is a bold move - but very much needed and even ove

Re: [VOTE] Release Airflow 2.3.0 from 2.3.0rc2

Hey guys sorry I saw this thread late--was on vacation last week. I noticed that we made a bold move replacing the tree view with the new grid view. I like the new grid view, but given the open discussion in this PR and popularity of tree view (in our

Re: [ANNOUNCEMENT] Support for MacOS M1 in Breeze

Excellent! Thanks to Jarek and everyone involved for working on getting us better support on ARM-based Linux. 🙏 Looking forward for the upcoming improvements 👍 On Thu, Mar 10, 2022 at 3:13 AM Jarek Potiuk wrote: > Hello Everyone, > > Another day, another announcement. I have just merged this PR

Re: [VOTE] AIP-45 Remove double dag parsing in airflow run

+1 (binding) On Mon, Mar 7, 2022 at 9:39 AM Ping Zhang wrote: > Hi friends, > > I would like to call for a vote on AIP-45 Remove double dag parsing in > airflow run: > > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-45+Remove+double+dag+parsing+in+airflow+run > > > *Materials from th

Re: [VOTE] AIP-43 DAG Processor separation

+1 (binding). This is a model proven to work and help make Airflow more scalable and secure--we run a similar setup in Airbnb and it works quite well for us. Thank you for driving this! Cheers, Kevin Y On Tue, Jan 25, 2022 at 12:16 PM Ping Zhang wrote: > +1. Thanks Mateusz for making the desi

Re: Increasing Astronomer's sponsorship of Airflow CI by 4x

Nice! Thanks Ash and Astronomer! On Thu, Jan 20, 2022 at 6:33 PM Ping Zhang wrote: > WOW, this is HUGE!! > > Thanks, > > Ping > > > On Thu, Jan 20, 2022 at 2:29 PM Tao Feng wrote: > >> Cool! >> >> On Thu, Jan 20, 2022 at 1:28 PM Jarek Potiuk wrote: >> >>> Super cool! >>> >>> czw., 20 sty 2022,

Re: New PMC Member: Jed Cunningham

Congrats Jed 👍 On Tue, Jan 4, 2022 at 10:38 AM Vikram Koka wrote: > Congratulations Jed, very well deserved indeed! > > > > On Tue, Jan 4, 2022 at 9:39 AM Ryan Hamilton > wrote: > >> Congrats Jed! >> >> On Tue, Jan 4, 2022 at 12:36 PM Xiaodong Deng wrote: >> >>> Congrats, Jed! >>> >>> On Tue,

Re: Apache Airflow newsletter - June & July 2021

Thanks Karalina, great work. The newsletter is very informative and amusing (thanks to Hektor)! Cheers, Kevin Y On Mon, Aug 9, 2021 at 12:56 AM Tobiasz Kędzierski < tobiaszkedzier...@gmail.com> wrote: > Thanks Karolina! Awsome newsletter and Hektor makes it even better! > > BR Tobiasz > > wt.,

Re: [DISCUSS] TaskGroup in Tree View

t; >>>> What do other people think? >>>> >>>> Best >>>> Bin >>>> >>>> On Sat, Mar 6, 2021 at 4:51 AM Kaxil Naik wrote: >>>> >>>>> Hi all, interesting discussion. I would love to hear about some more >

Re: New Committers: Qian Yu & Xinbin Huang

Congrats to both of you! Cheers, Kevin Y On Thu, Apr 8, 2021 at 8:13 PM Jiajie Zhong wrote: > Congrats to both of you! > > > Best Wish > — Jiajie > >

Re: [VOTE] AIP-39: Richer scheduler_interval

+1 binding On Fri, Mar 19, 2021 at 7:15 PM Daniel Imberman wrote: > +1 binding. Really proud of this work, James! > > On Fri, Mar 19, 2021 at 6:50 PM, Jarek Potiuk wrote: > > +1 binding. It will make it better for a class of uses. > > On Sat, Mar 20, 2021 at 2:44 AM Xinbin Huang > wrote: > >>

Re: [DISCUSS] TaskGroup in Tree View

re extending the TaskGroup concept to the backend? > > Best regards, > Vikram > > On Fri, Mar 5, 2021 at 1:31 AM Kevin Yang wrote: > >> Hi Nathan, >> >> Thanks a lot for your input and it is indeed a valid use case. This can >> be done either keeping TaskGrou

Re: [DISCUSS] TaskGroup in Tree View

; > I’ve already created a feature ticket to try to add some kind of TaskGroup > Sensor but perhaps this can also form part of the wider discussions here. > > > > https://github.com/apache/airflow/issues/14563 > > > > Cheers, > > > > Nathan > > > > *Fro

Re: [DISCUSS] TaskGroup in Tree View

esentation to also be in the Tree View. > - You are not proposing any "execution or scheduling" enhancements [again > for your use case] with TaskGroups > > Is the above correct? > > Vikram > > > > On Wed, Mar 3, 2021 at 9:21 PM Kevin Yang wrote: > >&g

[DISCUSS] TaskGroup in Tree View

Hi team, We are very glad to see the introduction of TaskGroup in Airflow 2.0 and really like it. Thanks to Yu Qian and everyone that contributed to it. To continue moving towards the goal of replacing SubDagOperator with TaskGroup, I'd like to kick off a discussion on bringing TaskGroup into Tree

Re: [DISCUSS][AIP-39] Richer (and pluggable) schedule_interval on DAGs

Hi Ash and James, This is an exciting move. What do you think about using this opportunity to extend Airflow's support to streaming like use cases? I.e. DAGs/tasks that want to run forever like a service. For such use cases, schedule interval might not be meaningful, then do we want to make the da

Re: New Airflow Committer: Vikram Koka

Congrats Vikram! On Tue, Jan 5, 2021 at 3:00 PM Leah Cole wrote: > Yay Vikram!! Congrats! > > On Tue, Jan 5, 2021 at 11:40 AM Vikram Koka wrote: > >> Thank you Felix and Elad! >> >> It has been such a pleasure working with all of you Apache Airflow over >> the last year and really looking forwa

Re: New PMC Member: Xiaodong Deng

Congratulations Xiaodong! Well deserved. Kevin Y On Mon, Jan 4, 2021 at 11:31 PM Jarek Potiuk wrote: > AH! very late for me :) Certainly well deserved :) > > J. > > > On Tue, Jan 5, 2021 at 8:17 AM Deng Xiaodong wrote: > >> Thanks, Tobiasz and Vikram! >> >> A big thanks to the community as we

Re: Apache Airflow 2.0.0 is released!

ps://airflow.apache.org/docs/apache-airflow/stable/executor/kubernetes.html?highlight=pod_override#pod-template-file> >>>> >>>> Docs on pod_override >>>> <https://airflow.apache.org/docs/apache-airflow/stable/executor/kubernetes.html?highlight=pod_overrid

Re: New Airflow Committer: Ryan Hamilton

Congrats Ryan! On Wed, Oct 28, 2020 at 11:02 AM Daniel Imberman wrote: > Congratulations Ryan! Well deserved and with high expectations! > > On Wed, Oct 28, 2020 at 11:01 AM, Deng Xiaodong > wrote: > > Congrats Ryan! > The UI improvements you kindly made are definitely a big progress of > Airfl

Re: [VOTE] AIP-8 Split Providers into Separate Packages for Airflow 2.0

+1 (binding) On Sun, Sep 13, 2020 at 1:29 PM Jarek Potiuk wrote: > Hello Everyone, > > Last week, at the Airflow 2.0 meeting the people involved made a decision > that we are releasing Airlow 2.0 as a set of separate "core" and > "providers" packages - similarly to the 1.10 "backport providers"

Re: [Meeting Notes] Airflow 2.0 Dev Call #2 - 24 Aug 2020

Thank you Kaxil, this is extremely helpful. We'll try to join at least the next meeting trying to see if we can provide more perspectives on SmartSensor and anything else we can help. Cheers, Kevin Y On Tue, Aug 25, 2020 at 4:28 PM Kaxil Naik wrote: > Hi all, > > I have created a document to s

Re: [VOTE] AIP-34 TaskGroup: A UI task grouping concept as an alternative to SubDagOperator

+1 (binding) On Tue, Aug 25, 2020 at 2:47 PM Kamil Breguła wrote: > +1 (binding) > > On Tue, Aug 25, 2020, 23:35 Daniel Imberman > wrote: > > > +1 binding > > > > via Newton Mail > > [ > > > https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.50&pv=10.15.5&source=email_footer_2 > > ] > > On Tue, A

Re: [VOTE] AIP-17: Consolidate and de-duplicate sensor tasks in airflow Smart Sensor

Kevin Yang Arthur Wiedmer Tomasz Urbaszek +1 (non-binding): Vikram Koka Damian Shaw Cheers, Kevin Y On Fri, Jul 24, 2020 at 4:09 PM Kaxil Naik wrote: > +1 binding > > On Sat, Jul 25, 2020 at 12:05 AM Kevin Yang wrote: > > > Thank you everyone! We'll keep the vote

Re: [VOTE] AIP-17: Consolidate and de-duplicate sensor tasks in airflow Smart Sensor

Thank you everyone! We'll keep the vote open until 2020-07-27T00:00Z. On Thu, Jul 23, 2020 at 4:16 AM Tomasz Urbaszek wrote: > +1 binding > > On Wed, Jul 22, 2020 at 4:57 PM Arthur Wiedmer > wrote: > > > +1 (binding) > > > > On Tue, Jul 21, 2020 at 11:12

Re: [VOTE] AIP-17: Consolidate and de-duplicate sensor tasks in airflow Smart Sensor

Thank you everyone for the feedback! I think the PR covered what is being proposed in the AIP and does not have big controversial change that cannot be revised easily. The feature itself has also been baked in Airbnb's large Airflow setup for 6+ months. So consider this my binding +1. I would love

Re: New PMCs: Daniel Imberman, Tomek Turbaszek & Kamil Breguła

😀👍🎉 On Tue, Jul 21, 2020 at 12:21 AM Michał Słowikowski < michal.slowikow...@polidea.com> wrote: > Congrats guys! > > On Mon, Jul 20, 2020 at 8:53 PM QP Hou wrote: > > > Congratulations to Daniel, Tomek and Kamil, well deserved! > > > > Thanks, > > QP Hou > > > > > > On Mon, Jul 20, 2020 at 7:21

Re: new committer: QP Hou

Congrats! Welcome QP. On Wed, Jul 15, 2020 at 10:35 AM Tomasz Urbaszek wrote: > Congrats QP! > > T. > > On Wed, Jul 15, 2020 at 7:31 PM Tao Feng wrote: > > > Welcome QP! > > > > On Tue, Jul 14, 2020 at 1:53 PM Jarek Potiuk > > wrote: > > > > > The Project Management Committee (PMC) for Apache

Re: [AIP-35] Add Signal Based Scheduling To Airflow

I'm in general supportive of this idea of supporting streaming jobs. We in Airbnb have historically ran stream jobs for years on Airflow, with some hacks of course. Yes the stream jobs might not be idempotent or so to fit in the Airflow paradigm. But I personally would love to see Airflow be expand

Re: Language matters - don't use the term "whitelist" in Apache Airflow

Thank you Ash. +1 and stamped. Cheers, Kevin Y On Sun, Jun 7, 2020 at 2:46 PM Deng Xiaodong wrote: > Appreciate your effort, Ash. And a +1 for this. > > > XD > > On Sun, Jun 7, 2020 at 23:31 Ash Berlin-Taylor wrote: > > > Hi everyone, > > > > I have just opened a PR to rename the few of the r

Re: Unsubscribe from dev-list

Take care Fokko! You have delivered essential impact on elevating Airflow to what it is now. I deeply apprecaite your mentorship and contributions to me and the project. And now it's time to enjoy the sunshine! Enjoy! Cheers, Kevin Y On Mon, Jun 1, 2020 at 11:36 AM Jarek Potiuk wrote: > Cheer

Re: Proposal: Change TI from having execution date to dag_run_id (in API at least)

+1 On Thu, May 14, 2020 at 1:18 PM Dan Davydov wrote: > +1 but in the future I think better would be > /dags/{dag_id}/dagRuns/{execution_date}/{run_number}. That would give an > automatic ordering between two runs, is a lot simpler than > "backfill_2020-03-16T00:00:00+00:00" and helps enable the

Re: [VOTE] Make conn_id unique in Airflow (a.k.a. Remove connection balancing HA )

+1 (binding) Thank you Ash for asking specifically!🙏 We do use it as a hacky load balancing mechanism in Airbnb but I agree that the right thing to is to make it unique, we will migrate to a proper solution. Cheers, Kevin Y On Fri, Apr 24, 2020 at 10:09 AM Daniel Standish wrote: > +1 (non-bi

Re: New committer: Jiajie Zhong

Congrats Jiajie! On Tue, Mar 31, 2020 at 10:39 AM Kaxil Naik wrote: > Congratulations Jiajie, welcome aboard. > > On Tue, Mar 31, 2020 at 6:31 PM Tomasz Urbaszek > wrote: > > > Congratulations Jiajie! Well deserved ;) > > > > T. > > > > On Tue, Mar 31, 2020 at 7:29 PM Jarek Potiuk > > wrote: >

Re: [VOTE] Switch from using Jira to Github Issues

+1 binding On Mon, Mar 16, 2020 at 9:34 AM Tao Feng wrote: > +1 > > On Mon, Mar 16, 2020 at 7:36 AM Ash Berlin-Taylor wrote: > > > The original discuss thread is here. > > > https://lists.apache.org/thread.html/r19014fac5aa2aa15a86890aa438940da263e430cb535ca451ef451a4%40%3Cdev.airflow.apache.or

Re: Stateless Webserver with DAG Serialization

This is thrilling news! Completely upleveling the experience using and maintaining the webserver. Thank you so much everyone who contributed to this initiative! Cheers, Kevin Y On Fri, Mar 13, 2020 at 6:24 PM Kaxil Naik wrote: > Hi all, > > Happy to tell you all that we have completed the firs

Re: Proposal: SIG-Kubernetes

+1 would love to join. On Wed, Feb 26, 2020 at 10:58 AM Xinbin Huang wrote: > +1 would love to join. > > On Wed, Feb 26, 2020 at 10:24 AM Dan Davydov > > wrote: > > > +1 would love to listen in on these > > > > On Wed, Feb 26, 2020 at 11:52 AM Tomasz Urbaszek < > > tomasz.urbas...@polidea.com>

Re: [VOTE] Add Probot Integrations to Airflow Github Repo

+1! Kaxil Naik 于2019年12月29日 周日下午6:21写道： > Thanks Max, > > I will raise a JIRA with the INFRA to add that too. > > Btw I had raised a request for the 4 integrations > https://issues.apache.org/jira/browse/INFRA-19607 but looks like it hasn't > gained any traction from them yet. > > > > On Fri, Dec

Re: Airflow user survey

Extremely useful, thank you so much Tomasz! On Wed, Dec 18, 2019 at 1:42 PM Ry Walker wrote: > Nice work on this everyone :) > > -Ry > > On Wed, Dec 18, 2019 at 12:01 PM, Tomasz Urbaszek < > tomasz.urbas...@polidea.com > wrote: > > > > > > > > > Hi all, > > > > > > > > > > We’ve just published r

Re: [VOTE] Allow using asserts in Airflow code

-1 (binding). People in the old thread has spoken for me. Specifically in Python, the confusion introduced by using asserts IMO can defeat all the benefits mentioned easily. Kevin Y On Thu, Dec 5, 2019 at 8:27 AM Tomasz Urbaszek wrote: > -1 (non-binding) > > T. > > > On Thu, Dec 5, 2019 at 4:

Re: Apache Airflow welcomes Kengo Seki (sekikn) as committer & PMC member

Congrats Kengo! Cheers, Kevin Y On Sun, Dec 1, 2019 at 5:49 PM hotmail wrote: > Congrats Kengo! > > > Best Wish > — Jiajie > >

Re: Improving the Airflow UI

Thank you for starting this discussion/initiative! Love it. Not a frontend guy but react looks fine. +1 for Dan's point of making it more testable. Great opportunity to establish a better UI baseline. Cheers, Kevin Y On Wed, Nov 27, 2019 at 12:29 PM Jarek Potiuk wrote: > Indeed it's been re-li

Re: New PMC members: Aizhamal and Kevin

>> > > > >> > On Fri, Nov 22, 2019 at 11:43 AM Kaxil Naik > > wrote: > > >> > > > >> > > Congratulations Aizhamal & Kevin. > > >> > > > > >> > > Well deserved. Keep up the good work. > > &g

Re: [DISCUSS] Airflow Summits dates and locations

+1 for both date and location in NA. On Thu, Nov 21, 2019 at 2:36 AM Kaxil Naik wrote: > +1 for the proposed dates and location. > > On Thu, Nov 21, 2019 at 4:46 AM Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > > > +1, the proposed dates and location seems reasonable to me. > > > >

Re: [VOTE] Accept new Airflow website contribution

+1(binding) very excited about the new site. Thanks all. On Thu, Nov 21, 2019 at 12:19 PM Driesprong, Fokko wrote: > Big +1 (binding) from my side. The new site looks great :-) > > Op do 21 nov. 2019 om 18:54 schreef Aizhamal Nurmamat kyzy < > aizha...@apache.org>: > > > I will leave this vote o

Re: [VOTE] 2nd attempt for AIP-21 update for Airflow 1.10.* backportability

+1 (binding) Kaxil Naik 于2019年11月15日 周五上午10:02写道： > I am fine with it :) > > On Fri, Nov 15, 2019 at 5:25 PM Jarek Potiuk > wrote: > > > I think we should leave all the fundamentals (or 'core') operators hooks > > etc where they are. I have not even considered moving them. I am ok to > have > >

Re: Proposed roadmap for Airflow 2.0

che Spark project, and create a preview > release > > for Airflow 2.0. > > > > Cheers, Fokko > > > > Op di 22 okt. 2019 om 04:58 schreef Kevin Yang : > > > > > For sure Fokko! I'll go through the PRs after finishing reading the one > &

Re: [Discuss] Airflow Summits 2020

Happy to help with the NA event organization too, if it happens in Bay Area. Cheers, Kevin Y On Tue, Nov 12, 2019 at 1:12 PM Austin Bennett wrote: > As an airflow user, and based in Bay Area (proposed first location), with > experience organizing events -- happy to contribute. > > On Tue, Nov 1

Re: Drop Python 3.5 support?

+1 Ash Berlin-Taylor 于2019年11月12日 周二下午2:51写道： > +1 from me too, especially now Debian ships a newer version. > > And yes, doing it for 2.0 seems like the best time > > -ash > > On 12 November 2019 22:45:42 GMT, Felix Uellendall > wrote: > >Absolutely +1. In my opinion that should come together w

Re: [Final review] Airflow website + need your input to the web copy

Excellent work! I really like it, esp. the documentation part. Thank you everyone who contributed to it 👍 +1 for it is very subjective and I'm in favor of the moving elements. I'm not so sure how being a project for serious user would conflict with just showing some moving elements in the landing

Re: [PROPOSE] Ease future migration path to 2.0 by provider's operators/hook backporting to 1.10.*

Tomasz and Ash got good points about the overhead of having separate repos. But while we grow bigger and more mature, I would prefer to have what was described in AIP-8. It shouldn't be extremely hard for us to come up with good strategies to handle the overhead. AIP-8 already talked about how it c

Re: [RESULT] [VOTE] Release 1.10.6 from 1.10.6rc2

ring > both - week-ends and week-days ;) > > On Mon, Oct 28, 2019 at 10:51 PM Ash Berlin-Taylor wrote: > > > :) Given this was no-code-changes since RC1 I wasn't worrying about it on > > this specific vote, but yes point taken :) > > > > -ash > > > &g

Re: Apache Airflow 1.10.6 released!

Releases keeps getting smoothier 👍 And definitely looking forwards to have DAG serialization, or maybe even stateless webserver? Cheers, Kevin Y On Mon, Oct 28, 2019 at 2:49 PM Jarek Potiuk wrote: > Great! So now we can now target our efforts towards 1.10.7 with long > awaited DAG serialisatio

Re: [RESULT] [VOTE] Release 1.10.6 from 1.10.6rc2

🎉 Tested core, UI and breeze locally with 3.7.3, all good. +1 (non-binding) Maybe next time make the voting hours all in weekdays? :P Cheers, Kevin Y On Mon, Oct 28, 2019 at 2:18 PM Ash Berlin-Taylor wrote: > Hello, > > Apache Airflow 1.10.6 (based on RC2) has been accepted. > > 4 “+1” bindin

Re: [VOTE] AIP-24: Persisting serialized DAG in DB for webserver scalability

So you can always > trace what's being executed, but before execution, it might be incorrect > (because the xcom etc is not properly being populated). > > Cheers, Fokko > > > Op do 24 okt. 2019 om 11:51 schreef Kevin Yang : > > > Just saw the latest email from Ka

Re: [VOTE] AIP-24: Persisting serialized DAG in DB for webserver scalability

o the rebase pain. Cheers, Kevin Y On Thu, Oct 24, 2019 at 2:40 AM Kevin Yang wrote: > Oops, I guess I'm too slow reading the PRs to let me vote in time 😛 > Congrats Zhou, Kaxil and Ash for getting it accepted 🎉 > > I'm pretty excited about this long-anticipated change a

Re: [VOTE] AIP-24: Persisting serialized DAG in DB for webserver scalability

Oops, I guess I'm too slow reading the PRs to let me vote in time 😛 Congrats Zhou, Kaxil and Ash for getting it accepted 🎉 I'm pretty excited about this long-anticipated change and thank you guys for the great work. Though I do have a few comments. I agree with Dan's concern about maintaining two

Re: Proposed roadmap for Airflow 2.0

too long with the 2.0 release, > > and not try to cram everything in there. Right now we're already > > backporting a lot to 1.10 and the resolving of the conflicts is getting > > more tedious. This already broke the 1.10.4 release. The master branch > > already has a lot of n

Re: Proposed roadmap for Airflow 2.0

Thanks Ash for putting together the doc, somehow I cannot do anything on confluence so I'll put my comments here. +1 for using this opportunity to define how we want to do releases, e.g. frequency, compatibility rules, etc. If the DAG isolation is being worked on I would love to see it in 2.0.

Re: [VOTE] Release 1.10.6 from 1.10.6rc1

lt images for 3.7 so it will take a long time to build it for the > first time from the scratch. > > J. > > > > On Sat, Oct 19, 2019 at 12:43 AM Kevin Yang wrote: > > > Just started to play with it for a bit and seems like we got a small bit > > inconsistency be

Re: Announcing SIG-Knative/ The Monthly Knative Executor Meetup/Call-for-contributers

👍 Thank you. On Fri, Oct 18, 2019 at 3:41 PM Kaxil Naik wrote: > Either we would record or have meeting notes :) > > On Fri, Oct 18, 2019 at 11:27 PM Kevin Yang wrote: > > > Sry that I didn't notice the event is 8am-9am. Can we the events recorded > > by any c

Re: [VOTE] Release 1.10.6 from 1.10.6rc1

Just started to play with it for a bit and seems like we got a small bit inconsistency between what version of python breeze supports. The code would allow only `2.7, 3.5, 3.6` but we claim we support `3.5, 3.6, 3.7`

Re: Announcing SIG-Knative/ The Monthly Knative Executor Meetup/Call-for-contributers

Sry that I didn't notice the event is 8am-9am. Can we the events recorded by any chance? On Thu, Oct 17, 2019 at 11:10 PM Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > Nice! > > On Wed, Oct 16, 2019 at 1:19 PM Ash Berlin-Taylor > wrote: > > > We meant October 30th (the date in the cal

Re: New PMC Member: Jarek Potiuk

Congrats! Well deserved. Ash Berlin-Taylor 于2019年10月18日 周五下午2:08写道： > And we'll deserved too! Congrats. > > -a > > On 18 October 2019 22:05:33 BST, Kaxil Naik wrote: > >Hi Airflow community, > > > >The Project Management Committee (PMC) for Apache Airflow > >has invited Jarek Potiuk to become a

Re: Announcing SIG-Knative/ The Monthly Knative Executor Meetup/Call-for-contributers

Exciting! Daniel Imberman 于2019年10月16日 周三上午10:58写道： > Fixing the calendar link :) > > https://calendar.google.com/event?action=TEMPLATE&tmeid=b2I4Yml2MWdhcXRtdGRtZmkzcXNxdmY5cThfMjAxOTEwMzBUMTUwMDAwWiBqMHY2c25uYzZ1N2l1cGI1Y2dxYTNpaTU4Z0Bn&tmsrc=j0v6snnc6u7iupb5cgqa3ii58g%40group.calendar.google.c

Re: AIP-7 completed :)

Woohoo! Thank you Jarek! I definitely find it very useful and really appreciate the nice work. Looking forward to see more changes from you and work together more. Cheers, Kevin Y On Wed, Sep 18, 2019 at 4:41 PM Kaxil Naik wrote: > Awesome work Jarek :) > > On Wed, Sep 18, 2019 at 10:36 PM Jare

Re: Codecov Integration

Thanks Jerek :D I’m very into seeing coverage increase in each PR personally Driesprong, Fokko 于2019年9月10日 周二上午1:53写道： > Thanks Jarek for fixing this! > > Cheers, Fokko > > Op zo 8 sep. 2019 om 12:19 schreef Jarek Potiuk >: > > > I think the latest PR should fix it (keep fingers crossed). > > ht

Re: [ANNOUNCE] Please welcome new Airflow committer Felix Uellendall

Congrats man :D On Mon, Sep 9, 2019 at 4:00 PM Chao-Han Tsai wrote: > Congrats. > > On Mon, Sep 9, 2019 at 6:15 AM Philippe Gagnon > wrote: > > > Congratulations! > > > > On Mon, Sep 9, 2019, 09:10 Kaxil Naik wrote: > > > > > Congratulations Felix > > > > > > On Mon, Sep 9, 2019 at 2:03 PM Jar

Re: "Breeze" dev environment

iuk wrote: > Fantastic. Thanks. Do you recall which tests/packages they were? I'd love > to see whats's the root cause. We are using the same image in CI and the > tests pass, so I wonder what could be the reason for that. > > On Thu, Aug 29, 2019 at 4:13 AM Kevin Yang wrote

Re: "Breeze" dev environment

Was looking forward to this since last meet up in bay area. Tried both the docker solution and the virtualenv IDE( PyCharm) solution today and it was great. Great job Jarek and thank you so much. (One minor thing that may just be problem on my side: some tests complain about missing package so I d

Re: Working on a new Airflow website!

Very excited, looking forward to it. Thank you guys Cheers, Kevin Y On Mon, Aug 26, 2019 at 10:04 AM Kamil Breguła wrote: > Hello, > > I am very happy to be able to work on this site. From the beginning of > work on this project, I wanted the project to be available to more > people. I am sure

Re: [ANNOUNCE] Please welcome new Airflow committer Chao-han Tsai

Congrats Chao-Han! Welcome onboard! Tao Feng 于2019年8月16日 周五下午3:19写道： > All, > > I'm pleased to announce that the Airflow PMC has voted in Chao-han Tsai to > be > a committer. Chao-han has been actively involved in the project for some > time > now, contributing patches including backfill enhancem

Re: [DISCUSS] Tweaks to the Airflow logo

Lol totally a shuriken, coool~ On Tue, Aug 13, 2019 at 12:22 PM Ash Berlin-Taylor wrote: > Of course, vector versions will be available. > > We'll give everyone a bit more time for feedback and opinions (a day or > two?), possibly do a revision then open up a formal vote. > > On 13 August 2019 2

Re: Blog post: Upgrading & Scaling Airflow at Robinhood

Nice post Abhishek! Glad our discussion was helpful for you guys. To share more context with the community, Airbnb had task stuck in QUEUED state problem before too. Our issues were more on the executor side. Originally it was because message lost issue in early version celery, which Alex Guziel a

Re: Apache Airflow 1.10.4 released!

Nice! Thanks Ash and everyone who helped! Cheers, Kevin Y On Wed, Aug 7, 2019 at 2:08 AM Driesprong, Fokko wrote: > Awesome! Thanks Ash for all the effort of releasing 1.10.4! > > Cheers, Fokko > > Op di 6 aug. 2019 om 23:27 schreef Jarek Potiuk >: > > > Well done! > > > > J. > > > > Principal

Re: Airflow DAG Serialisation

s/dagbag import exception/dag import timeout exception/ On Wed, Jul 31, 2019 at 11:17 PM Kevin Yang wrote: > Hi Jonathan, for your problem, aside waiting for AIP-24 for the long term, > you can try set the dagbag_import_timeout > <https://github.com/apache/airflow/blob/ma

Re: Airflow DAG Serialisation

with some > > > > > > optimisations > > > > > > > (caching mentioned by Fokko) - it can solve the major pain > points > > > > and I > > > > > > > think relatively quick and is potentially portable to 1.10.5 if > > we > > > > have >

Re: Airflow DAG Serialisation

Hope we can get DB > persistence move faster. > > Zhou > > On Mon, Jul 29, 2019, 5:46 PM Kevin Yang wrote: > >> oops, s/consistent file/consistent file order/ >> >> On Mon, Jul 29, 2019 at 5:42 PM Kevin Yang wrote: >> >>> Hi Zhou, >>> >>>

Re: Airflow DAG Serialisation

oops, s/consistent file/consistent file order/ On Mon, Jul 29, 2019 at 5:42 PM Kevin Yang wrote: > Hi Zhou, > > Totally understood, thank you for that. Streaming logic does cover most > cases, tho we still have the worst cases where os.walk doesn't give us > consiste

Re: Airflow DAG Serialisation

#x27;t need to restart > webserver per 20min. > > > > Thanks, > Zhou > > > On Mon, Jul 29, 2019 at 3:14 PM Kevin Yang wrote: > >> Hi Zhou, >> >> Thank you for the pointer. This solves the issue gunicorn restart rate >> throttles webserver refresh rate bu

Re: Airflow DAG Serialisation

eing gradually freshed > with this feature. > > Thanks, > Zhou > > > On Sat, Jul 27, 2019 at 2:43 AM Kevin Yang wrote: > >> Nice job Zhou! >> >> Really excited, exactly what we wanted for the webserver scaling issue. >> Want to add another big

Re: Airflow DAG Serialisation

Nice job Zhou! Really excited, exactly what we wanted for the webserver scaling issue. Want to add another big drive for Airbnb to start think about this previously to support the effort: it can not only bring consistency between webservers but also bring consistency between webserver and schedule

Re: [Announce] Apache Airflow YouTube Channel

Thank you Aizhamal! Glad to see this as a heavy YouTube user :D Cheers, Kevin Y Aizhamal Nurmamat kyzy 于2019年7月23日 周二下午2:05写道： > Hello everyone, > > I thought it would be nice to have a central repo for talk recordings and > videos related to Apache Airflow, so I have created a YouTube Channel f

Re: I have joined Astronomer full-time !!

Yeah! Congras Kaxil! Cheers, Kevin Y On Fri, Jun 21, 2019 at 3:22 PM Feng Lu wrote: > Congrats Kaxil! > > On Fri, Jun 21, 2019 at 2:09 PM Philippe Gagnon > wrote: > > > Congratulations Kaxil! This is wonderful news! > > > > On Fri, Jun 21, 2019 at 4:37 PM Tao Feng wrote: > > > > > Awesome:) C

Re: [VOTE] AIP-16: CLI: Use nested commands instead of flags

+1 (binding) On Wed, Jun 12, 2019 at 11:39 PM Chao-Han Tsai wrote: > +1 (non-binding) > > On Tue, Jun 11, 2019 at 3:36 PM Andrii Soldatenko < > andrii.soldate...@gmail.com> wrote: > > > +1 (non-binding) > > > > On Tue, Jun 11, 2019 at 11:41 PM Maxime Beauchemin < > > maximebeauche...@gmail.com>

Re: [DISCUSS] Time for a user@ list?

+1 On Thu, May 23, 2019 at 11:10 PM Kamil Breguła wrote: > +1 > > On Fri, May 24, 2019 at 7:57 AM Jakob Homan wrote: > > > The dev list is quite crowded, which several members of the community > > have noted, and it's getting hard to separate the technical direction > > questions from the gener

Re: [ANNOUNCE] Please welcome new Airflow committer Joshua Carp

Congrats Joshua! Cheers, Kevin Y On Mon, Apr 29, 2019 at 4:40 PM Deng Xiaodong wrote: > Congrats Joshua! > > XD > > On Tue, Apr 30, 2019 at 05:56 Ash Berlin-Taylor wrote: > > > Nice to have you on the team > > > > And Bas, Jarek and anyone else I've missed recently! So many new > > committers.

Re: [ANNOUNCE] Please welcome new Airflow committer Bas Harenslak

Congras Bas~! Cheers, Kevin Y On Tue, Apr 23, 2019 at 10:47 PM Chao-Han Tsai wrote: > Congrats Bas! > > On Tue, Apr 23, 2019 at 8:00 AM Deng Xiaodong wrote: > > > Congrats Bas! > > > > XD > > > > > On 23 Apr 2019, at 6:26 PM, Kaxil Naik wrote: > > > > > > Congrats Bas :) > > > > > > On Tue, A

Re: New job: I'm joining Astronomer.io to work on Airflow

Wow! Oooh yeah, win win win for you, Astronomer and us the community! :D Congratulations! Shame on me not knowing the podcast before, now I know and will follow ;) Looking forward to discuss the roadmap with you esp. about DAG serialization effort. Cheers, Kevin Y On Thu, Apr 18, 2019 at 11:43

Re: Proposal: Directly tie Jira into Github for airflow

+1 Cheers, Kevin Y Bas Harenslak 于2019年4月17日 周三下午2:16写道： > Nice one +1 > > Bas > > On 17 Apr 2019, at 23:13, Daniel Imberman > wrote: > > Using the Github Jira integration > https://github.com/marketplace/jira-software-github we can make it easier > to ma

Re: Proposal: Automatically mark stale PRs in github

+1 Cheers, Kevin Daniel Imberman 于2019年4月17日 周三下午2:02写道： > As part of our effort to reduce the PR backlog I wanted to proposed that > we set the github stale action https://github.com/apps/stale. This will > allow us to temporarily close PRs/tickets that are not actively being > worked on. > >

Re: [ANNOUNCE] Please welcome new Airflow committer Jarek Potiuk

Congras Jarek :D Cheers! Kevin Y On Wed, Apr 17, 2019 at 11:12 AM Sid Anand wrote: > Welcome aboard Jarek! > -s > > On Wed, Apr 17, 2019 at 11:05 AM Daniel Imberman < > dimberman.opensou...@gmail.com> wrote: > > > Congratulations Jarek! > > > > On Wed, Apr 17, 2019 at 10:59 AM Arthur Wiedmer <

Re: [AIP-19] Making the webserver stateless

Thank you Julian, nice work. Good idea trying to put context info into TaskInstance. Overall I would be strongly preferring option 2a, for not upsetting owners of big DAGs and being more tractable, we can keep in mind we may 2b later when implementing 2a. Cheers, Kevin Y On Sat, Apr 13, 2019 at 8

Re: Database referral integrity

ect the database to be much cleaner because > we make sure that old rows are being cleaned up, by cascading deletions. > I'm curious which queries would be slow. I'll park this, for now, as I > don't have too much time to work on this anyway :'( > > Cheers, Fokko &g

Re: [2.0 spring cleaning] Require unique conn_id

Yup unfortunately we Airbnb are relaying on the "feature" for some load balanching and also something like sensing partitions from 2 clusters at the same time( yup it is ugly). And at the same time we got bitten by having duplicate connections while one has outdated info. I think it does make sens

Re: Airflow 1.10.3 has been released!

Great job Ash! Release cycle has definitely been improved a lot since 1.10 Cheers, Kevin Yang Driesprong, Fokko 于2019年4月10日 周三上午11:26写道： > Thanks a lot for being our release manager Ash, great job! > > Big thanks to the community for making this happen. > > Cheers, Fokko >

Re: [DISCUSS] Utility DAGs

Agree on having core airflow related stuff built into airflow( like schedule delay instrumentation) and leave the others to cluster maintainer to set up( like log retention). How people handle log retention might be quite different depends on the logging backend. E.g. we use ElasticSearch and we do

  1   2   >