Re: [DISCUSSION] AIP-49 OpenTelemetry Support for Apache Airflow

2022-04-25 Thread Howard Yoo
Hi Elad, Will take a look at it and let you know! Thanks, Howard On Mon, Apr 25, 2022 at 6:35 AM Elad Kalif wrote: > Hi Howard, > Have you made progress with addressing the comments? > I think once points are addressed we can maybe start a vote? > > On Tue, Apr 12, 2022 at 12:39 PM Jarek Potiuk

Re: [DISCUSS] Deprecate core airflow k8s settings in KubernetesPodOperator

2022-04-25 Thread Jarek Potiuk
Sure :) lets discuss on slack; I am just about to cleanup some back-log of tasks and happy to talk about it :). On Mon, Apr 25, 2022 at 2:32 PM Daniel Standish wrote: > > Appreciate it, Jarek > > Re your last point > >> 5) Or MAYBE we should simply incorporate cncf.kubernetes provider >> entirel

Re: [DISCUSS] Approach for new providers of the community

2022-04-25 Thread Jarek Potiuk
I think this is a different story (and different discussion). And I think we should have good reasons to split the repo. I think we do have it but for different reasons many people think we will get there sooner rather than later - but I think we should not hijack the discussion for it. This discus

Re: [DISCUSS] Experimental Prod ARM64 image for 2.3.0 ?

2022-04-25 Thread Kaxil Naik
Yeah agreed with QP -- I think we should start with experimental support On Mon, 25 Apr 2022 at 16:53, Eloi Codina wrote: > I also think it would be great enabling experimental ARM images. In my > case, I could test Airflow on a small ARM-based server before installing it > on our production ser

Re: [DISCUSS] Approach for new providers of the community

2022-04-25 Thread Kaxil Naik
Hey all, Another alternative is separating out core providers from the Core Airflow Repo into a separate repo within the Apache Org itself, maybe: apache-airflow-providers. That will not decrease the maintenance from the Committers but the Core work and release will be completely separate and unt

Re: [DISCUSS] Approach for new providers of the community

2022-04-25 Thread Jarek Potiuk
> 1. https://registry.astronomer.io/ > 2. Using the new classifier > https://pypi.org/search/?o=&c=Framework+%3A%3A+Apache+Airflow+%3A%3A+Provider Yep. precisely what I thought to place at the top of the ecosystem page. > On 25 April 2022 18:08:49 BST, "Ferruzzi, Dennis" > wrote: >> >> I still

Re: [DISCUSS] Approach for new providers of the community

2022-04-25 Thread Ash Berlin-Taylor
Two fast/easy ways to find them 1. https://registry.astronomer.io/ 2. Using the new classifier https://pypi.org/search/?o=&c=Framework+%3A%3A+Apache+Airflow+%3A%3A+Provider On 25 April 2022 18:08:49 BST, "Ferruzzi, Dennis" wrote: >I still think that easy inclusion with a defined pruning proces

Re: [DISCUSS] Approach for new providers of the community

2022-04-25 Thread Ferruzzi, Dennis
I still think that easy inclusion with a defined pruning process is best, but it's looking like that is the minority opinion. In which case, IFF we are going to be keeping them separate then I definitely agree that there needs to be a fast/easy/convenient way to find them. ___

Re: New data privacy policy - may impact your websites

2022-04-25 Thread Jarek Potiuk
Kamil? can you help ? On Sat, Apr 23, 2022 at 10:32 PM Ross Turk wrote: > > > The contents of docs-archive seem to have the updated tag now, and > the data is already starting to come in! Looks great. > > But the changes in #577 do not seem to have taken effect yet. They > are present in the gh-p

RE: [DISCUSS] Experimental Prod ARM64 image for 2.3.0 ?

2022-04-25 Thread Eloi Codina
I also think it would be great enabling experimental ARM images. In my case, I could test Airflow on a small ARM-based server before installing it on our production server. On 2022/04/13 11:53:32 Jarek Potiuk wrote:> Hey everyone,> > We have a CI image multi-platform image (including ARM64) platfor

Re: [DISCUSS] Approach for new providers of the community

2022-04-25 Thread Jarek Potiuk
Just to come back to it (please everyone a little patience - I think some people have not chimed in yet due to 2.3.0 "focus" so this discussion might take a little more time. My current thinking on it so far: * I am not really in the camp of "lets not add any more providers at all" and also not i

Re: [DISCUSS] Deprecate core airflow k8s settings in KubernetesPodOperator

2022-04-25 Thread Daniel Standish
Appreciate it, Jarek Re your last point 5) Or MAYBE we should simply incorporate cncf.kubernetes provider > entirely in the core of Airlfow? Maybe there should be NO > "cncf.kubernetes" provider? My Answer: This is the point which is the real reason for me being > reluctant here. I see it as qui

Re: [DISCUSS] Experimental Prod ARM64 image for 2.3.0 ?

2022-04-25 Thread Jarek Potiuk
Anyone else has an opinion? Should we enable the ARM image as experimental? On Sun, Apr 17, 2022 at 10:52 PM QP Hou wrote: > > Would be great if we can get an official ARM image out soon. Our M1 > users can really benefit from this. Running Airflow in emulated mode > is simply unusable at the mom

Re: [DISCUSSION] AIP-49 OpenTelemetry Support for Apache Airflow

2022-04-25 Thread Elad Kalif
Hi Howard, Have you made progress with addressing the comments? I think once points are addressed we can maybe start a vote? On Tue, Apr 12, 2022 at 12:39 PM Jarek Potiuk wrote: > > 1. I would assume the path for StatsD (dogstatsd) would be for > deprecation - we will perhaps comment or mark it

Re: [DISCUSS] Deprecate core airflow k8s settings in KubernetesPodOperator

2022-04-25 Thread Jarek Potiuk
Just to give a bit of context - why I think it is important. It had never ever happened that we had to yank 4 versions of a provider because of incompatibilities we learned after the fact. And it's not anyone's fault - i't just learning that we should take into account. And Cncf.kubernetes is a ve