Re: Docker/Helm releases

2020-09-09 Thread Daniel Imberman
Thank you Niclas, this will help us a lot in figuring out our helm situation via Newton Mail [https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.50&pv=10.15.6&source=email_footer_2] On Wed, Sep 9, 2020 at 5:36 AM, Kaxil Naik wrote: Credits to Jarek on that one, he is the one who is actually dra

Re: Import style in Airflow codebase

2020-09-09 Thread Daniel Imberman
+1 on my end! via Newton Mail [https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.50&pv=10.15.6&source=email_footer_2] On Wed, Sep 9, 2020 at 7:38 PM, Maxime Beauchemin wrote: +1 On Wed, Sep 9, 2020 at 10:36 AM Vikram Koka wrote: > +1 on absolute import. Honestly, a huge fan of doing it as a

Re: Import style in Airflow codebase

2020-09-09 Thread Maxime Beauchemin
+1 On Wed, Sep 9, 2020 at 10:36 AM Vikram Koka wrote: > +1 on absolute import. Honestly, a huge fan of doing it as an absolute vs. > relative. > > On Wed, Sep 9, 2020 at 4:09 AM Kaxil Naik wrote: > > > No strong opinion but absolute import seems better from a user's > > perspective. > > > > On

Canceled event with note: Tentative: Airflow + Hacktoberfest 2020 Call @ Fri Sep 11, 2020 11am - 11:15am (CDT) (dev@airflow.apache.org)

2020-09-09 Thread alma
BEGIN:VCALENDAR PRODID:-//Google Inc//Google Calendar 70.9054//EN VERSION:2.0 CALSCALE:GREGORIAN METHOD:CANCEL BEGIN:VEVENT DTSTART:20200911T16Z DTEND:20200911T161500Z DTSTAMP:20200909T181658Z ORGANIZER;CN=a...@sg.com.mx:mailto:a...@sg.com.mx UID:6j7ibpaafrcghrbfcpagu0e...@google.com ATTENDEE;C

Re: [Meeting Notes] Airflow 2.0 Dev Call #3 - 7 Sep 2020

2020-09-09 Thread Kaxil Naik
Thanks, Vikram for updating the planning page. On Wed, Sep 9, 2020 at 5:43 PM Vikram Koka wrote: > Thanks Kaxil, this looks right to me as well. > I updated the main Airflow 2.0 planning page as well to reflect the current > scope and milestones based on this meeting. > > *Doc Link*: > https://c

Re: Import style in Airflow codebase

2020-09-09 Thread Vikram Koka
+1 on absolute import. Honestly, a huge fan of doing it as an absolute vs. relative. On Wed, Sep 9, 2020 at 4:09 AM Kaxil Naik wrote: > No strong opinion but absolute import seems better from a user's > perspective. > > On Wed, Sep 9, 2020 at 12:07 PM Jarek Potiuk > wrote: > > > +1! > > > > On

Re: [Meeting Notes] Airflow 2.0 Dev Call #3 - 7 Sep 2020

2020-09-09 Thread Vikram Koka
Thanks Kaxil, this looks right to me as well. I updated the main Airflow 2.0 planning page as well to reflect the current scope and milestones based on this meeting. *Doc Link*: https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+2.0+-+Planning I also wanted to thank Kevin from the AirBnb

Invitation: Tentative: Airflow + Hacktoberfest 2020 Call @ Fri Sep 11, 2020 11am - 11:15am (CDT) (dev@airflow.apache.org)

2020-09-09 Thread alma
BEGIN:VCALENDAR PRODID:-//Google Inc//Google Calendar 70.9054//EN VERSION:2.0 CALSCALE:GREGORIAN METHOD:REQUEST BEGIN:VEVENT DTSTART:20200911T16Z DTEND:20200911T161500Z DTSTAMP:20200909T140544Z ORGANIZER;CN=a...@sg.com.mx:mailto:a...@sg.com.mx UID:6j7ibpaafrcghrbfcpagu0e...@google.com ATTENDEE;

Re: Airflow + Hacktoberfest 2020

2020-09-09 Thread Alma Rinasz
Morning! I just sent a tentative invite for a call this Friday at 11AMCST in hopes that this helps get things moving. Looking forward to learning more. Stay healthy and safe everyone! On Wed, Sep 9, 2020 at 7:06 AM Jarek Potiuk wrote: > Happy to have the call this week Mara! > > On Sat, Aug 29,

Re: Docker/Helm releases

2020-09-09 Thread Kaxil Naik
Credits to Jarek on that one, he is the one who is actually drafting the proposal. On Wed, Sep 9, 2020, 13:31 Niclas Hedhman wrote: > Corporate requirements are typically that they can build everything from > sources and have clear instructions (preferably scriptable) on how to do > that. > > Go

Re: Docker/Helm releases

2020-09-09 Thread Niclas Hedhman
Corporate requirements are typically that they can build everything from sources and have clear instructions (preferably scriptable) on how to do that. Good to hear that ComDev is in the loop and you are together working on draft proposals. It will be greatly appreciated. Niclas On Wed, Sep 9

Re: Docker/Helm releases

2020-09-09 Thread Kaxil Naik
Thanks for your thoughtful reply. It clarifies few things but we will also definitely wait until the lines/rules are more clearer and we (Airflow PMC) would be very happy to help shape the rules if needed. On Wed, Sep 9, 2020, 13:26 Niclas Hedhman wrote: > No changes have been decided on recentl

Re: Docker/Helm releases

2020-09-09 Thread Niclas Hedhman
No changes have been decided on recently. If you have external dependencies that can be viewed as System Requirements, then you don't need to provide your own build instructions. Think; "Windows is a (optional) System Requirement" and what is reasonable in that case. So Docker itself could be a S

Re: [DISCUSS] Weekly newsletter/summary of the things going on in the community

2020-09-09 Thread Tomasz Urbaszek
I think it's a really good idea! Few points / ideas: 1. What tool can we use and if paid who will cover it? Is it possible to create new mailing list like `n...@airflow.apache.org` that can be used for a newsletter? EDIT: devlist don't support html right? 2. Who should be responsible for prepari

Re: Airflow + Hacktoberfest 2020

2020-09-09 Thread Jarek Potiuk
Happy to have the call this week Mara! On Sat, Aug 29, 2020 at 4:51 PM Jarek Potiuk wrote: > I am not in this Friday (vacations) but I will join the next one :) > > On Sat, Aug 29, 2020 at 2:17 AM Mara Ruvalcaba wrote: > > > > Hi everyone, > > > > Sorry for my late answer, we just ended the Bea

Re: [DISCUSS] Weekly newsletter/summary of the things going on in the community

2020-09-09 Thread Jarek Potiuk
Strong +1. I reviewed and cleaned-up my inbox yesterday (and I woke up this morning to 50+ new emails - surely some of them are for PRs and other things I am engaged with, but I am working on Airflow pretty much full time (ir sometimes I think 150% ;), but I only imagine how people must feel if th

Re: Import style in Airflow codebase

2020-09-09 Thread Kaxil Naik
No strong opinion but absolute import seems better from a user's perspective. On Wed, Sep 9, 2020 at 12:07 PM Jarek Potiuk wrote: > +1! > > On Wed, Sep 9, 2020 at 11:11 AM Ash Berlin-Taylor wrote: > > > Get's my vote, certainly. > > > > Here's a PR to do it -- https://github.com/apache/airflow/

Re: Import style in Airflow codebase

2020-09-09 Thread Jarek Potiuk
+1! On Wed, Sep 9, 2020 at 11:11 AM Ash Berlin-Taylor wrote: > Get's my vote, certainly. > > Here's a PR to do it -- https://github.com/apache/airflow/pull/10825 > > If no one complains in 24 hours lets merge that. > > On Sep 9 2020, at 9:35 am, Tomasz Urbaszek > wrote: > > > Hi all, > > > > I

Re: Docker/Helm releases

2020-09-09 Thread Jarek Potiuk
Added Niclas to my response :). Responding to devlist when someone from outside of it sends a message is tricky :) On Wed, Sep 9, 2020 at 12:35 PM Jarek Potiuk wrote: > Hello Niclas, > > Thanks for that. > > I feel that this guidance already answers most of my questions. > > I volunteered to lea

Re: Docker/Helm releases

2020-09-09 Thread Jarek Potiuk
Thanks! A really useful one. It's definitely not about bit-by-bit reproducibility. That also came in our discussion about images with the Apache Security team. I am much more advocate of what we currently have in our Production images - they are designed in the way that you should be able to have

Re: Docker/Helm releases

2020-09-09 Thread Kaxil Naik
Hi Niclas, I am glad to hear that the term "Convenience Binaries" is under review. Few things we were not clear or have some difference of opinion are: 1) Licensing a) Should we check all the binaries used in Dockerfiles for license-compliance? b) For Dockerfiles used in Helm Chart, is checkin

Re: [Meeting Notes] Airflow 2.0 Dev Call #3 - 7 Sep 2020

2020-09-09 Thread Jarek Potiuk
LGTM! Thanks, Kaxil for putting this together. It is really helpful! On Wed, Sep 9, 2020 at 12:29 PM Kaxil Naik wrote: > Hi all, > > I have created a document to summarize the discussion from our third dev > call for Airflow 2.0. > > Thank you all who joined the call. > > *Doc Link*: > > https:/

Re: Docker/Helm releases

2020-09-09 Thread Ash Berlin-Taylor
Jarek, you might want to check out https://reproducible-builds.org/ -- it's come out of Debian with the aim to try and make binary builds of .deb/software installed via .deb binary reproducible. I think you have less strict goals in mind than bit-for-bit identical rebuilds of docker images? If so

Re: Docker/Helm releases

2020-09-09 Thread Jarek Potiuk
Hello Niclas, Thanks for that. I feel that this guidance already answers most of my questions. I volunteered to lead proposal discussion and preparation for the ASF Board on this subject (and I am sure other PMCs from Airflow will also be engaged a lot, so I hope we can work out some reasonable

[Meeting Notes] Airflow 2.0 Dev Call #3 - 7 Sep 2020

2020-09-09 Thread Kaxil Naik
Hi all, I have created a document to summarize the discussion from our third dev call for Airflow 2.0. Thank you all who joined the call. *Doc Link*: https://cwiki.apache.org/confluence/display/AIRFLOW/Meeting+Notes#MeetingNotes-#3:7Sep2020 To all those who attended, can you please double-check

Re: Import style in Airflow codebase

2020-09-09 Thread Ash Berlin-Taylor
Get's my vote, certainly. Here's a PR to do it -- https://github.com/apache/airflow/pull/10825 If no one complains in 24 hours lets merge that. On Sep 9 2020, at 9:35 am, Tomasz Urbaszek wrote: > Hi all, > > I would like to spark a (hopefully short) discussion about import > style in Airflow.

Splitting 2.0 into provider packages/dynamic provider discovery

2020-09-09 Thread Jarek Potiuk
This is a proposal on how we can address dynamic provider discovery in Airflow 2.0 and 1.10.13 as well. At the meeting on Monday, we agreed that Airflow 2.0 will be released using a mechanism based on what we have for backport packages. One of the problems to solve was the dynamic discovery of pac

Import style in Airflow codebase

2020-09-09 Thread Tomasz Urbaszek
Hi all, I would like to spark a (hopefully short) discussion about import style in Airflow. In short: absolute vs relative imports. Reason for this discussion: https://github.com/apache/airflow/pull/10729#discussion_r485419342 Personally I think we should enforce (using pre-commit hook) absolute