[GitHub] [airflow] jedcunningham commented on a diff in pull request #32099: Support setting dependencies for tasks called outside setup/teardown context manager

2023-06-23 Thread via GitHub
jedcunningham commented on code in PR #32099: URL: https://github.com/apache/airflow/pull/32099#discussion_r1240592248 ## airflow/models/baseoperator.py: ## @@ -1001,6 +1001,14 @@ def __enter__(self): def __exit__(self, exc_type, exc_val, exc_tb):

[GitHub] [airflow] amoghrajesh commented on pull request #31376: Add valid spark on k8s airflow connection urls to tests

2023-06-23 Thread via GitHub
amoghrajesh commented on PR #31376: URL: https://github.com/apache/airflow/pull/31376#issuecomment-1605244487 @hussein-awala can we merge this PR too as the dependent PR has been merged recently? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #32112: deferrable mode for `SageMakerTuningOperator`

2023-06-23 Thread via GitHub
vandonr-amz commented on code in PR #32112: URL: https://github.com/apache/airflow/pull/32112#discussion_r1240552004 ## tests/system/providers/amazon/aws/example_sagemaker.py: ## @@ -159,12 +159,11 @@ def _build_and_upload_docker_image(preprocess_script, repository_uri):

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #32113: Fix KubernetesPodOperator validate xcom json and add retries

2023-06-23 Thread via GitHub
boring-cyborg[bot] commented on PR #32113: URL: https://github.com/apache/airflow/pull/32113#issuecomment-1605198948 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] aagateuip opened a new pull request, #32113: Fix KubernetesPodOperator validate xcom json and add retries

2023-06-23 Thread via GitHub
aagateuip opened a new pull request, #32113: URL: https://github.com/apache/airflow/pull/32113 --- * Added retries to extract_xcom to guard against intermittent network connectivity failures. * xcom json is validated to make sure entire json was retrieved. * xcom sidecar is

[GitHub] [airflow] vandonr-amz opened a new pull request, #32112: deferrable mode for `SageMakerTuningOperator`

2023-06-23 Thread via GitHub
vandonr-amz opened a new pull request, #32112: URL: https://github.com/apache/airflow/pull/32112 took the opportunity to fix tiny things in the system test, and also migrate to the trigger to the method that logs the status while it waits -- This is an automated message from the

[GitHub] [airflow] github-actions[bot] closed pull request #31040: Added kube-prometheus-stack servicemonitor to gather metrics from Airflow statsd exporter

2023-06-23 Thread via GitHub
github-actions[bot] closed pull request #31040: Added kube-prometheus-stack servicemonitor to gather metrics from Airflow statsd exporter URL: https://github.com/apache/airflow/pull/31040 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] github-actions[bot] commented on pull request #30581: Make pause DAG its own role separate from edit DAG

2023-06-23 Thread via GitHub
github-actions[bot] commented on PR #30581: URL: https://github.com/apache/airflow/pull/30581#issuecomment-1605178177 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

[airflow] branch constraints-main updated: Updating constraints. Github run id:5361110862

2023-06-23 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new b0a5ba8486 Updating

[GitHub] [airflow] ITJamie commented on a diff in pull request #30531: [WIP] Migrate core `EmailOperator` to SMTP provider

2023-06-23 Thread via GitHub
ITJamie commented on code in PR #30531: URL: https://github.com/apache/airflow/pull/30531#discussion_r1240510568 ## airflow/utils/email.py: ## @@ -19,22 +19,69 @@ import collections.abc import logging -import os import re -import smtplib import warnings -from

[GitHub] [airflow] getaaron commented on issue #31105: airflow-webserver crashes if log files are larger than webserver available RAM

2023-06-23 Thread via GitHub
getaaron commented on issue #31105: URL: https://github.com/apache/airflow/issues/31105#issuecomment-1605119355 > Yeah, I thought of a way where we could maintain separate log_pos for different streams. This way we do not have to pull the whole file into memory just to get the final few

[GitHub] [airflow] potiuk commented on pull request #32110: add deferrable mode for `AthenaOperator`

2023-06-23 Thread via GitHub
potiuk commented on PR #32110: URL: https://github.com/apache/airflow/pull/32110#issuecomment-1605084324 static checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] potiuk commented on pull request #31920: Deprecate the 2 non-official elasticsearch libraries

2023-06-23 Thread via GitHub
potiuk commented on PR #31920: URL: https://github.com/apache/airflow/pull/31920#issuecomment-1605076347 Woooho! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] potiuk merged pull request #31920: Deprecate the 2 non-official elasticsearch libraries

2023-06-23 Thread via GitHub
potiuk merged PR #31920: URL: https://github.com/apache/airflow/pull/31920 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: Deprecate the 2 non-official elasticsearch libraries (#31920)

2023-06-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new b28c90354f Deprecate the 2 non-official

[GitHub] [airflow] potiuk commented on pull request #22253: Add SparkKubernetesOperator crd implementation

2023-06-23 Thread via GitHub
potiuk commented on PR #22253: URL: https://github.com/apache/airflow/pull/22253#issuecomment-1605071290 errors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] dstandish commented on pull request #32053: Change `as_setup` and `as_teardown` to instance methods

2023-06-23 Thread via GitHub
dstandish commented on PR #32053: URL: https://github.com/apache/airflow/pull/32053#issuecomment-1605068228 OK @uranusjr @ephraimbuddy @jedcunningham the scope of this PR expanded a little bit. I wanted to add validation so that (1) a task can't be both setup and teardown, (2) once you

[GitHub] [airflow] potiuk commented on issue #32048: Allow providers to add default connections

2023-06-23 Thread via GitHub
potiuk commented on issue #32048: URL: https://github.com/apache/airflow/issues/32048#issuecomment-1605064459 But yes. I am all for clarifying the behaviour, getting `airflow db init` to be renamed and defining when we create default conneciotns (and whether they should be coming from

[GitHub] [airflow] potiuk commented on issue #32048: Allow providers to add default connections

2023-06-23 Thread via GitHub
potiuk commented on issue #32048: URL: https://github.com/apache/airflow/issues/32048#issuecomment-1605062163 > How do we decide which providers would have default connections out of the box or in the past, since I don't see it for all of them? were there any criteria to be met? > >

[airflow] branch main updated: Add an option to load the dags from db for command tasks run (#32038)

2023-06-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 05a67efe32 Add an option to load the dags from db

[GitHub] [airflow] potiuk merged pull request #32038: Add an option to load the dags from db for command tasks run

2023-06-23 Thread via GitHub
potiuk merged PR #32038: URL: https://github.com/apache/airflow/pull/32038 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk closed issue #32020: Airflow tasks run -m cli command giving 504 response

2023-06-23 Thread via GitHub
potiuk closed issue #32020: Airflow tasks run -m cli command giving 504 response URL: https://github.com/apache/airflow/issues/32020 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] potiuk commented on pull request #29055: [AIP-51] Executors vending CLI commands

2023-06-23 Thread via GitHub
potiuk commented on PR #29055: URL: https://github.com/apache/airflow/pull/29055#issuecomment-1605060078 > This is still blocked by #30727 waiting for a merge of that one Maybe good time to merge it. -- This is an automated message from the Apache Git Service. To respond to the

[airflow] branch constraints-main updated: Updating constraints. Github run id:5360683424

2023-06-23 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new a5923a2b97 Updating

[GitHub] [airflow] potiuk commented on issue #32111: KubernetesPodOperator job intermittently fails - unable to retrieve json from xcom sidecar container due to network connectivity issues

2023-06-23 Thread via GitHub
potiuk commented on issue #32111: URL: https://github.com/apache/airflow/issues/32111#issuecomment-1605058814 Assigned you. Feel free to provide PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk commented on pull request #32054: Upgrade Flask-AppBuilder

2023-06-23 Thread via GitHub
potiuk commented on PR #32054: URL: https://github.com/apache/airflow/pull/32054#issuecomment-1605048344 > I'm going to check other changes in appbuilder since 4.3.1 and create a summary. Cool. I think the only thing that matters are changes that might need to be ported to

[GitHub] [airflow] potiuk commented on pull request #32016: Fix an issue that crashes Airflow Webserver when passed invalid private key path to Snowflake

2023-06-23 Thread via GitHub
potiuk commented on PR #32016: URL: https://github.com/apache/airflow/pull/32016#issuecomment-1605038514 conflicts :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] boring-cyborg[bot] commented on issue #32111: KubernetesPodOperator job intermittently fails - unable to retrieve json from xcom sidecar container due to network connectivity issues

2023-06-23 Thread via GitHub
boring-cyborg[bot] commented on issue #32111: URL: https://github.com/apache/airflow/issues/32111#issuecomment-1605035075 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[GitHub] [airflow] aagateuip opened a new issue, #32111: KubernetesPodOperator job intermittently fails - unable to retrieve json from xcom sidecar container due to network connectivity issues

2023-06-23 Thread via GitHub
aagateuip opened a new issue, #32111: URL: https://github.com/apache/airflow/issues/32111 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened We have seen that KubernetesPodOperator sometimes fails to retrieve json from xcom

[GitHub] [airflow] killua1zoldyck commented on issue #31105: airflow-webserver crashes if log files are larger than webserver available RAM

2023-06-23 Thread via GitHub
killua1zoldyck commented on issue #31105: URL: https://github.com/apache/airflow/issues/31105#issuecomment-1605020397 Yeah, I thought of a way where we could maintain separate log_pos for different streams. This way we do not have to pull the whole file into memory just to get the final

[GitHub] [airflow] potiuk closed issue #31054: Can not get more logs of DAGS on UI when deploy on Kubernetes

2023-06-23 Thread via GitHub
potiuk closed issue #31054: Can not get more logs of DAGS on UI when deploy on Kubernetes URL: https://github.com/apache/airflow/issues/31054 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on issue #31054: Can not get more logs of DAGS on UI when deploy on Kubernetes

2023-06-23 Thread via GitHub
potiuk commented on issue #31054: URL: https://github.com/apache/airflow/issues/31054#issuecomment-1605004330 Can you please update to 2.6.2 - there were a few issues connected to logs in 2.6.1 and 2.6.2 and they seem to be related (see for "logs" in the release notes). I think they should

[GitHub] [airflow] guard13 commented on issue #15291: Airflow Docker tutorial: Unhealthy container

2023-06-23 Thread via GitHub
guard13 commented on issue #15291: URL: https://github.com/apache/airflow/issues/15291#issuecomment-1605003054 Good morning, I give you the link to my solution which works and allows me to continue Airflow either on the Datascientest VM (but it dropped me, while resetting) or on my

[airflow] branch main updated: bugfix: break down run+wait method in ECS operator (#32104)

2023-06-23 Thread onikolas
This is an automated email from the ASF dual-hosted git repository. onikolas pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d49fa999a9 bugfix: break down run+wait method in

[GitHub] [airflow] o-nikolas merged pull request #32104: bugfix: break down run+wait method in ECS operator

2023-06-23 Thread via GitHub
o-nikolas merged PR #32104: URL: https://github.com/apache/airflow/pull/32104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] vandonr-amz opened a new pull request, #32110: add deferrable mode for `AthenaOperator`

2023-06-23 Thread via GitHub
vandonr-amz opened a new pull request, #32110: URL: https://github.com/apache/airflow/pull/32110 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[airflow] branch main updated: Use a waiter in `AthenaHook` (#31942)

2023-06-23 Thread onikolas
This is an automated email from the ASF dual-hosted git repository. onikolas pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 72d09a677f Use a waiter in `AthenaHook` (#31942)

[GitHub] [airflow] o-nikolas merged pull request #31942: Use a waiter in `AthenaHook`

2023-06-23 Thread via GitHub
o-nikolas merged PR #31942: URL: https://github.com/apache/airflow/pull/31942 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: Refactor Eks Create Cluster Operator code (#31960)

2023-06-23 Thread onikolas
This is an automated email from the ASF dual-hosted git repository. onikolas pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5c887988b0 Refactor Eks Create Cluster Operator

[GitHub] [airflow] o-nikolas merged pull request #31960: Refactor Eks Create Cluster Operator code

2023-06-23 Thread via GitHub
o-nikolas merged PR #31960: URL: https://github.com/apache/airflow/pull/31960 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] killua1zoldyck commented on issue #31105: airflow-webserver crashes if log files are larger than webserver available RAM

2023-06-23 Thread via GitHub
killua1zoldyck commented on issue #31105: URL: https://github.com/apache/airflow/issues/31105#issuecomment-1604974487 @potiuk The current log-streaming logic pulls the whole file into memory before moving to the specific log_pos based on the metadata after combining all the logs and

[GitHub] [airflow] potiuk closed issue #32023: Container Image not trusting LDAP certifcates

2023-06-23 Thread via GitHub
potiuk closed issue #32023: Container Image not trusting LDAP certifcates URL: https://github.com/apache/airflow/issues/32023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] potiuk commented on a diff in pull request #31840: added difference between Deferrable and Non-Deferrable Operators

2023-06-23 Thread via GitHub
potiuk commented on code in PR #31840: URL: https://github.com/apache/airflow/pull/31840#discussion_r1240378757 ## docs/apache-airflow/authoring-and-scheduling/deferring.rst: ## @@ -166,3 +166,25 @@ Airflow tries to only run triggers in one place at once, and maintains a

[GitHub] [airflow] potiuk commented on a diff in pull request #31840: added difference between Deferrable and Non-Deferrable Operators

2023-06-23 Thread via GitHub
potiuk commented on code in PR #31840: URL: https://github.com/apache/airflow/pull/31840#discussion_r1240373057 ## docs/apache-airflow/authoring-and-scheduling/deferring.rst: ## @@ -166,3 +166,25 @@ Airflow tries to only run triggers in one place at once, and maintains a

[GitHub] [airflow] potiuk commented on pull request #32013: Introducing AirflowClusterPolicySkipDag exception

2023-06-23 Thread via GitHub
potiuk commented on PR #32013: URL: https://github.com/apache/airflow/pull/32013#issuecomment-1604933188 BTW. The description should be generic enough to be "transplantable", also it needs ``.. versionadded:: 2.7`` in the documentation as this exception will not be available for earlier

[GitHub] [airflow] potiuk commented on pull request #32013: Introducing AirflowClusterPolicySkipDag exception

2023-06-23 Thread via GitHub
potiuk commented on PR #32013: URL: https://github.com/apache/airflow/pull/32013#issuecomment-1604929908 I like the idea. BUT. This will only be useful and usable if you describe your use case in the documentation -

[GitHub] [airflow] vidyapanchalZS commented on issue #32071: All DAGs file are parsing while executing some "airflow tasks" cli command

2023-06-23 Thread via GitHub
vidyapanchalZS commented on issue #32071: URL: https://github.com/apache/airflow/issues/32071#issuecomment-1604914542 Hello Hussein, thank you for looking into this request. In this request , i also mentioned some other "airflow tasks" cli command which giving similar output as

[GitHub] [airflow] potiuk commented on pull request #29790: Provider Databricks add jobs create operator.

2023-06-23 Thread via GitHub
potiuk commented on PR #29790: URL: https://github.com/apache/airflow/pull/29790#issuecomment-1604888950 > @potiuk is it okay if someone else forks Kyle's repo and fix the static check to not lose @kyle-winkelman's credit/commits and also address the previous comments made by @alexott? I

[GitHub] [airflow] boring-cyborg[bot] commented on issue #32107: Improved error logging for failed Dataflow jobs

2023-06-23 Thread via GitHub
boring-cyborg[bot] commented on issue #32107: URL: https://github.com/apache/airflow/issues/32107#issuecomment-1604884866 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[GitHub] [airflow] Srabasti opened a new issue, #32107: Improved error logging for failed Dataflow jobs

2023-06-23 Thread via GitHub
Srabasti opened a new issue, #32107: URL: https://github.com/apache/airflow/issues/32107 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened When running Dataflow job in Cloud Composer composer-1.20.12-airflow-1.10.15 Airflow

[GitHub] [airflow] tomrutter commented on issue #32091: Triggerer intermittent failure when running many triggerers

2023-06-23 Thread via GitHub
tomrutter commented on issue #32091: URL: https://github.com/apache/airflow/issues/32091#issuecomment-1604881767 I think task instance is set just below on line 691. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] tomrutter commented on issue #32091: Triggerer intermittent failure when running many triggerers

2023-06-23 Thread via GitHub
tomrutter commented on issue #32091: URL: https://github.com/apache/airflow/issues/32091#issuecomment-1604832307 Will check, but wouldn't all triggers fail if that were the case? There are many (>100) identical triggers working and only one that fails (all using the same custom trigger). I

[GitHub] [airflow] stikkireddy commented on pull request #29790: Provider Databricks add jobs create operator.

2023-06-23 Thread via GitHub
stikkireddy commented on PR #29790: URL: https://github.com/apache/airflow/pull/29790#issuecomment-1604803451 @potiuk is it okay if someone else forks Kyle's repo and fix the static check to not lose @kyle-winkelman's credit/commits and also address the previous comments made by @alexott?

[GitHub] [airflow] raphaelauv commented on pull request #32004: bump debian to bookworm

2023-06-23 Thread via GitHub
raphaelauv commented on PR #32004: URL: https://github.com/apache/airflow/pull/32004#issuecomment-1604795966 thanks for the explanation I could build locally the CI image without mysql -> ```shell breeze ci-image build --force-build --python-image

[GitHub] [airflow] darkag commented on pull request #32089: Parameters for vertica

2023-06-23 Thread via GitHub
darkag commented on PR #32089: URL: https://github.com/apache/airflow/pull/32089#issuecomment-1604773094 I tried to simplify code. I added a better handling for log_level parameter (add possibility to pass log level as "warning" and not just 30). Correction for request_complex_types which

[GitHub] [airflow] hussein-awala commented on issue #32091: Triggerer intermittent failure when running many triggerers

2023-06-23 Thread via GitHub
hussein-awala commented on issue #32091: URL: https://github.com/apache/airflow/issues/32091#issuecomment-1604772168 > It seems that as triggers fire, the link between the trigger row and the associated task_instance for the trigger is removed before the trigger row is removed. This leaves

[GitHub] [airflow] pankajkoti commented on issue #32096: Status of testing of Apache Airflow Helm Chart 1.10.0rc1

2023-06-23 Thread via GitHub
pankajkoti commented on issue #32096: URL: https://github.com/apache/airflow/issues/32096#issuecomment-1604761320 Thank you for clarifying in detail @potiuk and the quick fix. Yes, was wondering intially if the image itself needed some fix, but all clear now. -- This is an automated

[GitHub] [airflow] bhagany commented on issue #32106: GCSToBigQueryOperator and BigQueryToGCSOperator do not respect their project_id arguments

2023-06-23 Thread via GitHub
bhagany commented on issue #32106: URL: https://github.com/apache/airflow/issues/32106#issuecomment-1604752241 I should mention that I am about to go on a 2-week vacation, so while I am willing to discuss and submit a PR, my responses will be delayed. I also think this is a serious enough

[GitHub] [airflow] boring-cyborg[bot] commented on issue #32106: GCSToBigQueryOperator and BigQueryToGCSOperator do not respect their project_id arguments

2023-06-23 Thread via GitHub
boring-cyborg[bot] commented on issue #32106: URL: https://github.com/apache/airflow/issues/32106#issuecomment-1604748574 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[GitHub] [airflow] bhagany opened a new issue, #32106: GCSToBigQueryOperator and BigQueryToGCSOperator do not respect their project_id arguments

2023-06-23 Thread via GitHub
bhagany opened a new issue, #32106: URL: https://github.com/apache/airflow/issues/32106 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened We experienced this issue Airflow 2.6.1, but the problem exists in the Google provider

[GitHub] [airflow] potiuk commented on issue #32096: Status of testing of Apache Airflow Helm Chart 1.10.0rc1

2023-06-23 Thread via GitHub
potiuk commented on issue #32096: URL: https://github.com/apache/airflow/issues/32096#issuecomment-1604742095 Looks good ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] bhagany commented on issue #32093: GCSToBQ operator does not respect `project_id` in deferrable mode with impersonation chain.

2023-06-23 Thread via GitHub
bhagany commented on issue #32093: URL: https://github.com/apache/airflow/issues/32093#issuecomment-1604715011 Thank you for submitting this -- I have been procrastinating filing an issue of my own about this inconsistent handling of project ids. I believe the problems with changes to

[GitHub] [airflow] potiuk commented on issue #32096: Status of testing of Apache Airflow Helm Chart 1.10.0rc1

2023-06-23 Thread via GitHub
potiuk commented on issue #32096: URL: https://github.com/apache/airflow/issues/32096#issuecomment-1604711224 BTW @pankajkoti clarifying comment: The reason why the PR to remove support was included in the list was this

[GitHub] [airflow] vijayasarathib commented on pull request #32105: Adding missing hyperlink to the tutorial documentation

2023-06-23 Thread via GitHub
vijayasarathib commented on PR #32105: URL: https://github.com/apache/airflow/pull/32105#issuecomment-1604703092 > sure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] dnskr commented on issue #32096: Status of testing of Apache Airflow Helm Chart 1.10.0rc1

2023-06-23 Thread via GitHub
dnskr commented on issue #32096: URL: https://github.com/apache/airflow/issues/32096#issuecomment-1604701317 - [Use template comments for the chart license header #30569](https://github.com/apache/airflow/pull/30569) Looks good, all license header replacements are in place. -

[GitHub] [airflow] potiuk commented on pull request #32105: Adding missing hyperlink to the tutorial documentation

2023-06-23 Thread via GitHub
potiuk commented on PR #32105: URL: https://github.com/apache/airflow/pull/32105#issuecomment-1604700861 Ah. Checked that the link does not work. There is a mistake with "alias" package for the BaseOperator. he right fix is to change it to `airflow.models.baseoperator.BaseOperator` in

[GitHub] [airflow] potiuk commented on pull request #32105: Adding missing hyperlink to the tutorial documentation

2023-06-23 Thread via GitHub
potiuk commented on PR #32105: URL: https://github.com/apache/airflow/pull/32105#issuecomment-1604696556 The link is there. Clicking on class should lead you there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] potiuk closed pull request #32105: Adding missing hyperlink to the tutorial documentation

2023-06-23 Thread via GitHub
potiuk closed pull request #32105: Adding missing hyperlink to the tutorial documentation URL: https://github.com/apache/airflow/pull/32105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] vijayasarathib opened a new pull request, #32105: Adding missing hyperlink to the tutorial documentation

2023-06-23 Thread via GitHub
vijayasarathib opened a new pull request, #32105: URL: https://github.com/apache/airflow/pull/32105 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for

[GitHub] [airflow] potiuk commented on a diff in pull request #32052: Disable allowing by default testing of connnections in UI

2023-06-23 Thread via GitHub
potiuk commented on code in PR #32052: URL: https://github.com/apache/airflow/pull/32052#discussion_r1240170213 ## airflow/www/static/js/connection_form.js: ## @@ -123,6 +125,16 @@ function applyFieldBehaviours(connection) { */ function handleTestConnection(connectionType,

[airflow] branch main updated: Remove unclear (referring to CI) paragraph from "support python" (#32103)

2023-06-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 24e3d6ce57 Remove unclear (referring to CI)

[GitHub] [airflow] potiuk merged pull request #32103: Remove unclear (referring to CI) paragraph from "support python"

2023-06-23 Thread via GitHub
potiuk merged PR #32103: URL: https://github.com/apache/airflow/pull/32103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #32096: Status of testing of Apache Airflow Helm Chart 1.10.0rc1

2023-06-23 Thread via GitHub
potiuk commented on issue #32096: URL: https://github.com/apache/airflow/issues/32096#issuecomment-1604683871 https://github.com/apache/airflow/pull/32103 -> to remove the confusing paragraph. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] potiuk commented on pull request #32103: Remove unclear (referring to CI) paragraph from "support python"

2023-06-23 Thread via GitHub
potiuk commented on PR #32103: URL: https://github.com/apache/airflow/pull/32103#issuecomment-1604683438 It's funny to read your own description that you don't understand or that is more confusing than lack of it. Especially in the transition period, the original "clarification"

[GitHub] [airflow] vandonr-amz opened a new pull request, #32104: bugfix: break down run+wait method in ECS operator

2023-06-23 Thread via GitHub
vandonr-amz opened a new pull request, #32104: URL: https://github.com/apache/airflow/pull/32104 This method is just causing trouble by handling several things, it's hiding the logic. A bug fixed in #31838 was reintroduced in #31881 because the check that was skipped on

[GitHub] [airflow] potiuk opened a new pull request, #32103: Remove unclear (referring to CI) paragraph from "support python"

2023-06-23 Thread via GitHub
potiuk opened a new pull request, #32103: URL: https://github.com/apache/airflow/pull/32103 The paragraph was supposed to clarify things but it makes it confusing - it attempted to explain the difference vs "default" and supported versions of Python, referring to CI. Unfortunately this

[GitHub] [airflow] dstandish commented on a diff in pull request #32053: Change `as_setup` and `as_teardown` to instance methods

2023-06-23 Thread via GitHub
dstandish commented on code in PR #32053: URL: https://github.com/apache/airflow/pull/32053#discussion_r1240156463 ## airflow/example_dags/example_setup_teardown_taskflow.py: ## @@ -47,12 +45,10 @@ def normal(): @task_group def section_1(): # You can also

[GitHub] [airflow] potiuk commented on issue #32096: Status of testing of Apache Airflow Helm Chart 1.10.0rc1

2023-06-23 Thread via GitHub
potiuk commented on issue #32096: URL: https://github.com/apache/airflow/issues/32096#issuecomment-1604673066 > I am wondering if we want to include #30963 here as the PR has an associated milestone of 2.7.0 Python version is connected with Airflow version. It's not "chart" property

[airflow] branch constraints-main updated: Updating constraints. Github run id:5358827021

2023-06-23 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 09e7891ea0 Updating

[GitHub] [airflow] pankajkoti commented on issue #32096: Status of testing of Apache Airflow Helm Chart 1.10.0rc1

2023-06-23 Thread via GitHub
pankajkoti commented on issue #32096: URL: https://github.com/apache/airflow/issues/32096#issuecomment-1604651640 I am wondering if we want to include https://github.com/apache/airflow/pull/30963 here as the PR has an associated milestone of 2.7.0 If we want to include it here, I

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #32053: Change `as_setup` and `as_teardown` to instance methods

2023-06-23 Thread via GitHub
ephraimbuddy commented on code in PR #32053: URL: https://github.com/apache/airflow/pull/32053#discussion_r1240114070 ## airflow/example_dags/example_setup_teardown_taskflow.py: ## @@ -47,12 +45,10 @@ def normal(): @task_group def section_1(): # You can also

[GitHub] [airflow] spire-mike commented on issue #29531: Dynamic task mapping does not always create mapped tasks

2023-06-23 Thread via GitHub
spire-mike commented on issue #29531: URL: https://github.com/apache/airflow/issues/29531#issuecomment-1604625983 Thanks @knab-analytics, that's too bad. I hope it gets resolved. As a workaround I set `depends_on_past=False` and am using `ExternalTaskSensor` to ensure the last task of the

[GitHub] [airflow] o-nikolas commented on a diff in pull request #32052: Disable allowing by default testing of connnections in UI

2023-06-23 Thread via GitHub
o-nikolas commented on code in PR #32052: URL: https://github.com/apache/airflow/pull/32052#discussion_r1240093747 ## airflow/www/static/js/connection_form.js: ## @@ -123,6 +125,16 @@ function applyFieldBehaviours(connection) { */ function

[GitHub] [airflow] potiuk commented on issue #31733: after upgrade 2.6.0 from 2.3.4, task can't show log in webui

2023-06-23 Thread via GitHub
potiuk commented on issue #31733: URL: https://github.com/apache/airflow/issues/31733#issuecomment-1604592441 Can you please update to 2.6.2 ? I believe this is manifestation of a known issue -> see release notes and look for 'logs'

[GitHub] [airflow] blcksrx commented on a diff in pull request #31798: fix spark-kubernetes-operator compatibility

2023-06-23 Thread via GitHub
blcksrx commented on code in PR #31798: URL: https://github.com/apache/airflow/pull/31798#discussion_r1240077652 ## pyproject.toml: ## @@ -75,7 +75,7 @@ extend-ignore = [ # * Disable `flaky` plugin for pytest. This plugin conflicts with `rerunfailures` because provide same

[GitHub] [airflow] potiuk commented on a diff in pull request #31798: fix spark-kubernetes-operator compatibility

2023-06-23 Thread via GitHub
potiuk commented on code in PR #31798: URL: https://github.com/apache/airflow/pull/31798#discussion_r1240072325 ## pyproject.toml: ## @@ -75,7 +75,7 @@ extend-ignore = [ # * Disable `flaky` plugin for pytest. This plugin conflicts with `rerunfailures` because provide same

[airflow] branch main updated: Deferrable mode for ECS operators (#31881)

2023-06-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 415e076761 Deferrable mode for ECS operators

[GitHub] [airflow] potiuk merged pull request #31881: Deferrable mode for ECS operators

2023-06-23 Thread via GitHub
potiuk merged PR #31881: URL: https://github.com/apache/airflow/pull/31881 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #32004: bump debian to bookworm

2023-06-23 Thread via GitHub
potiuk commented on PR #32004: URL: https://github.com/apache/airflow/pull/32004#issuecomment-1604563692 The way to solve it later is to make a PR from "airflow" repo (then image build happens in-ci-workflow and the workflow has enough permissions to be able to push such image to airflow

[GitHub] [airflow] AVMusorin opened a new pull request, #32102: Fix reload gunicorn workers

2023-06-23 Thread via GitHub
AVMusorin opened a new pull request, #32102: URL: https://github.com/apache/airflow/pull/32102 Since gunicorn can't reload a new code if starts with ``--preload`` setting, we need to check ``reload_on_plugin_change`` before set it up. Gunicorn can't reload a new code because the code

[GitHub] [airflow] potiuk commented on pull request #32004: bump debian to bookworm

2023-06-23 Thread via GitHub
potiuk commented on PR #32004: URL: https://github.com/apache/airflow/pull/32004#issuecomment-1604561636 Yes. Because for non-committers, "build-image" is run from "main" not from your PR and in main you have `PYTHON_BASE_IMAGE=python:3.8-slim-bullseye`. The way to solve it is to push your

[GitHub] [airflow] pankajastro commented on a diff in pull request #32029: Add deferrable mode in EMR operator and sensor

2023-06-23 Thread via GitHub
pankajastro commented on code in PR #32029: URL: https://github.com/apache/airflow/pull/32029#discussion_r1240051624 ## airflow/providers/amazon/aws/sensors/emr.py: ## @@ -424,12 +431,16 @@ def __init__( job_flow_id: str, target_states: Iterable[str] | None =

[airflow] branch main updated: Adapt Notifier for sla_miss_callback (#31887)

2023-06-23 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e4ca68818e Adapt Notifier for

[GitHub] [airflow] ephraimbuddy merged pull request #31887: Adapt Notifier for sla_miss_callback

2023-06-23 Thread via GitHub
ephraimbuddy merged PR #31887: URL: https://github.com/apache/airflow/pull/31887 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #32101: Fix backfill_job_runner to work with custom executors

2023-06-23 Thread via GitHub
boring-cyborg[bot] commented on PR #32101: URL: https://github.com/apache/airflow/pull/32101#issuecomment-1604522311 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] adh-wonolo opened a new pull request, #32101: Fix backfill_job_runner to work with custom executors

2023-06-23 Thread via GitHub
adh-wonolo opened a new pull request, #32101: URL: https://github.com/apache/airflow/pull/32101 Backfill Job Runner pulls in the class name of your executor but doesn't pull in the full path so if you aren't using a default core executor you get an error like: ``` File

[airflow] branch constraints-main updated: Updating constraints. Github run id:5357825290

2023-06-23 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 63a8e19d8e Updating

[GitHub] [airflow] eladkal closed issue #32030: Status of testing Providers that were prepared on June 20, 2023

2023-06-23 Thread via GitHub
eladkal closed issue #32030: Status of testing Providers that were prepared on June 20, 2023 URL: https://github.com/apache/airflow/issues/32030 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] eladkal commented on issue #32030: Status of testing Providers that were prepared on June 20, 2023

2023-06-23 Thread via GitHub
eladkal commented on issue #32030: URL: https://github.com/apache/airflow/issues/32030#issuecomment-1604465569 Thank you everyone. Providers are released I invite everyone to help improve providers for the next release, a list of open issues can be found

[GitHub] [airflow-site] eladkal merged pull request #819: Add documentation for packages - 2023-06-20

2023-06-23 Thread via GitHub
eladkal merged PR #819: URL: https://github.com/apache/airflow-site/pull/819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

  1   2   >