Re: [DISCUSS] Approach for new providers of the community

2022-04-04 Thread Ferruzzi, Dennis
I think I'd just +1 Elad's comments. I don't know if we (the community) really need to be gatekeeping which providers get first class status like that. In the end, the users of any given provider become responsible for maintaining it, so I feel it sorts itself out without added bureaucracy. P

Re: [DISCUSSION] AIP-49 OpenTelemetry Support for Apache Airflow

2022-04-04 Thread Howard Yoo
Thank you, Elad! Yeah, it might have been TL;DR; moment on the PDF, but the description of the work required such amount, unfortunately (thanks for bearing with it!). I may not have all the answers, but do have some opinions on your 6 questions/points, so if I may add a little of them: 1. I would

Re: [DISCUSS] Deprecate core airflow k8s settings in KubernetesPodOperator

2022-04-04 Thread Daniel Standish
Thanks Jarek I register a bit of a mixed message. Re this Maybe I am wrong but I think the separation should be solved together, > there is a good reason why some k8s settings are in core. > Are you saying I should not proceed with this plan *(add warnings in KPO that push users to start using

Re: [DISCUSS] Approach for new providers of the community

2022-04-04 Thread Elad Kalif
Interesting topic! I think the most important thing for us is that we are able to maintain the provider (in terms of not causing problems for Airflow core or other providers). Some of the maintained providers (Google for example) have open bugs for 2 years. So even if we have many provider mantine

[DISCUSS] Approach for new providers of the community

2022-04-04 Thread Jarek Potiuk
Hey all, We seem to have an influx of new providers coming our way: * Delta Sharing: https://lists.apache.org/thread/kny1f23noqf1ssh7l9ys607m5wk3ff8c * Flyte: https://lists.apache.org/thread/b55g3gydgmqmhow6f7xzzbm5t0gmhs2x * Versatile Data Kit: https://lists.apache.org/thread/t1k3d0518v4kxz1pqs

Fwd: Applications for Travel Assistance to ApacheCon NA 2022 now open

2022-04-04 Thread Jarek Potiuk
Hello Everyone, Apache Con is back on-site this year - October, and for folks who want to attend and cannot travel for financial reasons - they can apply for financial assistance (see below). Small reminder - I am chairing the "Data Engineering" track and CFP is open https://apachecon.com/acna202

Re: [ANNOUNCE] Apache Airflow 2.2.5 Released

2022-04-04 Thread Jarek Potiuk
Wooho! On Mon, Apr 4, 2022 at 11:08 AM Ephraim Anierobi wrote: > > Dear Airflow community, > > I'm happy to announce that Airflow 2.2.5 was just released. > > The released sources and packages can be downloaded via > https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-f

[ANNOUNCE] Apache Airflow 2.2.5 Released

2022-04-04 Thread Ephraim Anierobi
Dear Airflow community, I'm happy to announce that Airflow 2.2.5 was just released. The released sources and packages can be downloaded via https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-sources.html Other installation methods are described in https://airflow.

[GitHub] [airflow] eladkal commented on a diff in pull request #22710: Add autodetect arg in BQCreateExternalTable Operator

2022-04-04 Thread GitBox
eladkal commented on code in PR #22710: URL: https://github.com/apache/airflow/pull/22710#discussion_r841493922 ## airflow/providers/google/cloud/operators/bigquery.py: ## @@ -964,6 +964,9 @@ class BigQueryCreateExternalTableOperator(BaseOperator): :param schema_object: If

[GitHub] [airflow] eladkal commented on a diff in pull request #22710: Add autodetect arg in BQCreateExternalTable Operator

2022-04-04 Thread GitBox
eladkal commented on code in PR #22710: URL: https://github.com/apache/airflow/pull/22710#discussion_r841492420 ## airflow/providers/google/cloud/operators/bigquery.py: ## @@ -964,6 +964,9 @@ class BigQueryCreateExternalTableOperator(BaseOperator): :param schema_object: If

[GitHub] [airflow] eladkal commented on a diff in pull request #22710: Add autodetect arg in BQCreateExternalTable Operator

2022-04-04 Thread GitBox
eladkal commented on code in PR #22710: URL: https://github.com/apache/airflow/pull/22710#discussion_r841492420 ## airflow/providers/google/cloud/operators/bigquery.py: ## @@ -964,6 +964,9 @@ class BigQueryCreateExternalTableOperator(BaseOperator): :param schema_object: If

[GitHub] [airflow] potiuk commented on a diff in pull request #22695: Fix Breeze2 autocomplete

2022-04-04 Thread GitBox
potiuk commented on code in PR #22695: URL: https://github.com/apache/airflow/pull/22695#discussion_r841476559 ## dev/breeze/src/airflow_breeze/utils/run_utils.py: ## @@ -102,6 +99,9 @@ def check_package_installed(package_name: str) -> bool: def get_filesystem_type(filepath

[GitHub] [airflow] hankehly commented on a diff in pull request #22710: Add autodetect arg in BQCreateExternalTable Operator

2022-04-04 Thread GitBox
hankehly commented on code in PR #22710: URL: https://github.com/apache/airflow/pull/22710#discussion_r841455146 ## airflow/providers/google/cloud/operators/bigquery.py: ## @@ -964,6 +964,9 @@ class BigQueryCreateExternalTableOperator(BaseOperator): :param schema_object: If

[GitHub] [airflow] hankehly commented on a diff in pull request #22710: Add autodetect arg in BQCreateExternalTable Operator

2022-04-04 Thread GitBox
hankehly commented on code in PR #22710: URL: https://github.com/apache/airflow/pull/22710#discussion_r841455146 ## airflow/providers/google/cloud/operators/bigquery.py: ## @@ -964,6 +964,9 @@ class BigQueryCreateExternalTableOperator(BaseOperator): :param schema_object: If

[GitHub] [airflow] eladkal commented on a diff in pull request #22710: Add autodetect arg in BQCreateExternalTable Operator

2022-04-04 Thread GitBox
eladkal commented on code in PR #22710: URL: https://github.com/apache/airflow/pull/22710#discussion_r841435403 ## airflow/providers/google/cloud/operators/bigquery.py: ## @@ -964,6 +964,9 @@ class BigQueryCreateExternalTableOperator(BaseOperator): :param schema_object: If