[GitHub] [airflow] Lee-W commented on a diff in pull request #34652: Fix `dag.test()` and Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
Lee-W commented on code in PR #34652: URL: https://github.com/apache/airflow/pull/34652#discussion_r1339602913 ## airflow/sensors/external_task.py: ## @@ -163,9 +163,11 @@ def __init__( total_states = set(self.allowed_states + self.skipped_states + self.failed_states)

[GitHub] [airflow] Lee-W commented on a diff in pull request #34652: Fix `dag.test()` and Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
Lee-W commented on code in PR #34652: URL: https://github.com/apache/airflow/pull/34652#discussion_r1339602707 ## airflow/sensors/external_task.py: ## @@ -163,9 +163,11 @@ def __init__( total_states = set(self.allowed_states + self.skipped_states + self.failed_states)

[GitHub] [airflow] utkarsharma2 commented on a diff in pull request #34652: Fix `dag.test()` and Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
utkarsharma2 commented on code in PR #34652: URL: https://github.com/apache/airflow/pull/34652#discussion_r1339594679 ## airflow/sensors/external_task.py: ## @@ -163,9 +163,11 @@ def __init__( total_states = set(self.allowed_states + self.skipped_states + self.failed_s

[GitHub] [airflow] utkarsharma2 commented on pull request #34652: Fix `dag.test()` and Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
utkarsharma2 commented on PR #34652: URL: https://github.com/apache/airflow/pull/34652#issuecomment-1738537373 > No need to add this patch in the core operators, we already fixed it, and for that we add the comment (TODO: remove this if block when min_airflow_version is set to higher than 2

[GitHub] [airflow] Lee-W commented on a diff in pull request #34652: Fix `dag.test()` and Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
Lee-W commented on code in PR #34652: URL: https://github.com/apache/airflow/pull/34652#discussion_r1339583073 ## airflow/sensors/external_task.py: ## @@ -163,9 +163,11 @@ def __init__( total_states = set(self.allowed_states + self.skipped_states + self.failed_states)

[GitHub] [airflow] utkarsharma2 commented on a diff in pull request #34652: Fix `dag.test()` and Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
utkarsharma2 commented on code in PR #34652: URL: https://github.com/apache/airflow/pull/34652#discussion_r1339576279 ## airflow/sensors/external_task.py: ## @@ -163,9 +163,11 @@ def __init__( total_states = set(self.allowed_states + self.skipped_states + self.failed_s

[GitHub] [airflow] utkarsharma2 commented on a diff in pull request #34652: Fix `dag.test()` and Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
utkarsharma2 commented on code in PR #34652: URL: https://github.com/apache/airflow/pull/34652#discussion_r1339576279 ## airflow/sensors/external_task.py: ## @@ -163,9 +163,11 @@ def __init__( total_states = set(self.allowed_states + self.skipped_states + self.failed_s

[GitHub] [airflow] utkarsharma2 commented on a diff in pull request #34652: Fix `dag.test()` and Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
utkarsharma2 commented on code in PR #34652: URL: https://github.com/apache/airflow/pull/34652#discussion_r1339572230 ## airflow/sensors/external_task.py: ## @@ -163,9 +163,11 @@ def __init__( total_states = set(self.allowed_states + self.skipped_states + self.failed_s

[GitHub] [airflow] utkarsharma2 commented on a diff in pull request #34652: Fix `dag.test()` and Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
utkarsharma2 commented on code in PR #34652: URL: https://github.com/apache/airflow/pull/34652#discussion_r1339572230 ## airflow/sensors/external_task.py: ## @@ -163,9 +163,11 @@ def __init__( total_states = set(self.allowed_states + self.skipped_states + self.failed_s

[GitHub] [airflow] jens-scheffler-bosch commented on issue #34634: Bulk "Select All" for Pause/Unpause/Trigger/Delete in DAG view of Airflow UI

2023-09-27 Thread via GitHub
jens-scheffler-bosch commented on issue #34634: URL: https://github.com/apache/airflow/issues/34634#issuecomment-1738511649 > Or perhaps you are saying "kill switch is dangerous but we should have it anyway"? I was referring to the "danger" that if suh kill switch would be implemented

[GitHub] [airflow] Joffreybvn commented on a diff in pull request #34606: Add pagination to SimpleHttpOperator

2023-09-27 Thread via GitHub
Joffreybvn commented on code in PR #34606: URL: https://github.com/apache/airflow/pull/34606#discussion_r1339558401 ## airflow/providers/http/operators/http.py: ## @@ -49,6 +50,12 @@ class SimpleHttpOperator(BaseOperator): :param data: The data to pass. POST-data in POST/PU

[GitHub] [airflow] eladkal commented on a diff in pull request #34514: Refactor DataFusionInstanceLink usage

2023-09-27 Thread via GitHub
eladkal commented on code in PR #34514: URL: https://github.com/apache/airflow/pull/34514#discussion_r1339558030 ## airflow/providers/google/cloud/operators/datafusion.py: ## @@ -34,21 +34,12 @@ from airflow.providers.google.cloud.operators.cloud_base import GoogleCloudBaseOpe

[airflow] branch constraints-main updated: Updating constraints. Github run id:6333173101

2023-09-27 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new e050c92354 Updating con

[airflow] branch main updated: Restrict `astroid` version < 3 (#34658)

2023-09-27 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 19450e03f5 Restrict `astroid` version < 3 (#

[GitHub] [airflow] jedcunningham merged pull request #34658: Restrict `astroid` version < 3

2023-09-27 Thread via GitHub
jedcunningham merged PR #34658: URL: https://github.com/apache/airflow/pull/34658 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[GitHub] [airflow] github-actions[bot] closed pull request #33192: Add marker for a test to run ti with test mode

2023-09-27 Thread via GitHub
github-actions[bot] closed pull request #33192: Add marker for a test to run ti with test mode URL: https://github.com/apache/airflow/pull/33192 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [airflow] o-nikolas commented on a diff in pull request #34317: Use auth manager `is_authorized_` APIs to check user permissions in Rest API

2023-09-27 Thread via GitHub
o-nikolas commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1339330087 ## airflow/api_connexion/security.py: ## @@ -55,3 +68,166 @@ def decorated(*args, **kwargs): return cast(T, decorated) return requires_access_decora

[GitHub] [airflow] jedcunningham opened a new issue, #34660: Status of testing of Apache Airflow Helm Chart 1.11.0rc1

2023-09-27 Thread via GitHub
jedcunningham opened a new issue, #34660: URL: https://github.com/apache/airflow/issues/34660 ### Body We are kindly requesting that contributors to [Apache Airflow Helm Chart 1.11.0rc1](https://dist.apache.org/repos/dist/dev/airflow/helm-chart/1.11.0rc1) help test the RC. Ple

[airflow] annotated tag helm-chart/1.11.0rc1 updated (ee2109ff4a -> 01426e4c76)

2023-09-27 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to annotated tag helm-chart/1.11.0rc1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag helm-chart/1.11.0rc1 was modified! *** from ee2109ff4a (commit) to 014

svn commit: r64231 - /dev/airflow/helm-chart/1.10.0rc1/

2023-09-27 Thread jedcunningham
Author: jedcunningham Date: Wed Sep 27 23:29:31 2023 New Revision: 64231 Log: Remove old Helm Chart release: 1.10.0rc1 Removed: dev/airflow/helm-chart/1.10.0rc1/

svn commit: r64230 - /dev/airflow/helm-chart/1.11.0rc1/

2023-09-27 Thread jedcunningham
Author: jedcunningham Date: Wed Sep 27 23:28:46 2023 New Revision: 64230 Log: Add artifacts for Helm Chart 1.11.0rc1 Added: dev/airflow/helm-chart/1.11.0rc1/ dev/airflow/helm-chart/1.11.0rc1/airflow-1.11.0.tgz (with props) dev/airflow/helm-chart/1.11.0rc1/airflow-1.11.0.tgz.asc

[airflow] branch main updated: Add helm chart 1.11.0 changelog for Artifacthub (#34655)

2023-09-27 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2abebe3a7b Add helm chart 1.11.0 changelog f

[GitHub] [airflow] jedcunningham merged pull request #34655: Add helm chart 1.11.0 changelog for Artifacthub

2023-09-27 Thread via GitHub
jedcunningham merged PR #34655: URL: https://github.com/apache/airflow/pull/34655 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[GitHub] [airflow] ferruzzi commented on a diff in pull request #34324: Executors doc update

2023-09-27 Thread via GitHub
ferruzzi commented on code in PR #34324: URL: https://github.com/apache/airflow/pull/34324#discussion_r1339321927 ## docs/apache-airflow/core-concepts/executor/index.rst: ## @@ -78,3 +71,134 @@ There are two types of executor - those that run tasks *locally* (inside the ``s ..

[GitHub] [airflow] o-nikolas commented on a diff in pull request #34324: Executors doc update

2023-09-27 Thread via GitHub
o-nikolas commented on code in PR #34324: URL: https://github.com/apache/airflow/pull/34324#discussion_r1339320655 ## docs/apache-airflow/core-concepts/executor/index.rst: ## @@ -78,3 +71,134 @@ There are two types of executor - those that run tasks *locally* (inside the ``s .

[GitHub] [airflow] Taragolis opened a new pull request, #34658: Restrict `astroid` version < 3

2023-09-27 Thread via GitHub
Taragolis opened a new pull request, #34658: URL: https://github.com/apache/airflow/pull/34658 Our `main` branch CI unable to build docs due to an error, see: https://github.com/readthedocs/sphinx-autoapi/issues/407 ```console apache-airflow-providers-jenkins

[GitHub] [airflow] dstandish commented on issue #34634: Bulk "Select All" for Pause/Unpause/Trigger/Delete in DAG view of Airflow UI

2023-09-27 Thread via GitHub
dstandish commented on issue #34634: URL: https://github.com/apache/airflow/issues/34634#issuecomment-1738136970 > Still is a valid feature having a "simple" administrative "halt" of scheduler. If somebody wants to contribute this is valid... You write `Having a single "kill switch" m

[GitHub] [airflow] jens-scheffler-bosch commented on issue #34634: Bulk "Select All" for Pause/Unpause/Trigger/Delete in DAG view of Airflow UI

2023-09-27 Thread via GitHub
jens-scheffler-bosch commented on issue #34634: URL: https://github.com/apache/airflow/issues/34634#issuecomment-1738125495 I feel "turning all off" is might be very simple but if you have a selective set enabled, then re-enabling still is a tedious job. If I would need to make this I proba

[GitHub] [airflow] dstandish opened a new pull request, #34656: Fix foreign key warning re ab_user.id

2023-09-27 Thread via GitHub
dstandish opened a new pull request, #34656: URL: https://github.com/apache/airflow/pull/34656 Introduced in #34120. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] jens-scheffler-bosch commented on issue #34641: Jinja NativeEnvironment causing unexpected argument type change

2023-09-27 Thread via GitHub
jens-scheffler-bosch commented on issue #34641: URL: https://github.com/apache/airflow/issues/34641#issuecomment-1738122440 Oh, yeah. You can easily help and this even with no complex dev env setup. Use the page https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/params.html

[GitHub] [airflow] tnyz commented on pull request #34654: Remove required emr serverless operator param

2023-09-27 Thread via GitHub
tnyz commented on PR #34654: URL: https://github.com/apache/airflow/pull/34654#issuecomment-1738115366 >Optional dictionary for arbitrary parameters to the boto API create_application call. so I think this statement is not accurate as you cannot put release_version in this dictionary

[GitHub] [airflow] dstandish commented on pull request #32572: Reduce config's default for max TIs per query & enforce <= parallelism

2023-09-27 Thread via GitHub
dstandish commented on PR #32572: URL: https://github.com/apache/airflow/pull/32572#issuecomment-1738114465 Hi @vincbeck Can you help me understand why max_tis_per_query should be no greater than parallelism? -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] tnyz commented on pull request #34654: Remove required emr serverless operator param

2023-09-27 Thread via GitHub
tnyz commented on PR #34654: URL: https://github.com/apache/airflow/pull/34654#issuecomment-1738107974 > Could you add a bit more details about this best practices? Unfortunately I can’t find any info, according to the [boto3 doc](https://boto3.amazonaws.com/v1/documentation/api/latest/refe

[GitHub] [airflow] dstandish commented on pull request #34120: Fix unfound ab_user table in the CLI session

2023-09-27 Thread via GitHub
dstandish commented on PR #34120: URL: https://github.com/apache/airflow/pull/34120#issuecomment-1738099095 @hussein-awala i see these warnings now: ``` /Users/dstandish/code/airflow/airflow/models/taskinstance.py:3432 SAWarning: Can't validate argument 'foreign_key'; can't locate any

[GitHub] [airflow] Taragolis commented on pull request #34654: Remove required emr serverless operator param

2023-09-27 Thread via GitHub
Taragolis commented on PR #34654: URL: https://github.com/apache/airflow/pull/34654#issuecomment-1738094904 Could you add a bit more details about this best practices? Unfortunately I can’t find any info, according to the [boto3 doc](https://boto3.amazonaws.com/v1/documentation/api/latest

[GitHub] [airflow] jedcunningham commented on pull request #34639: Warning when viewing docs for previous versions of Airflow

2023-09-27 Thread via GitHub
jedcunningham commented on PR #34639: URL: https://github.com/apache/airflow/pull/34639#issuecomment-1738093225 I really like this idea, but I suspect this will have to be done on [airflow-site](https://github.com/apache/airflow-site/). This repo only has the latest-and-greatest, and this w

[GitHub] [airflow] jedcunningham commented on a diff in pull request #34579: Fix KubernetesPodTrigger startup timeout

2023-09-27 Thread via GitHub
jedcunningham commented on code in PR #34579: URL: https://github.com/apache/airflow/pull/34579#discussion_r1339224522 ## tests/providers/cncf/kubernetes/triggers/test_pod.py: ## @@ -334,3 +335,37 @@ def test_define_container_state_should_execute_successfully( )

[GitHub] [airflow] ferruzzi merged pull request #34653: Clarify Amazon Lambda invocation and sensing

2023-09-27 Thread via GitHub
ferruzzi merged PR #34653: URL: https://github.com/apache/airflow/pull/34653 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[airflow] branch main updated (493431023f -> e333380077)

2023-09-27 Thread ferruzzi
This is an automated email from the ASF dual-hosted git repository. ferruzzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 493431023f Release notes for helm chart 1.11.0 (#34618) add e80077 Clarify Amazon Lambda invocation and sens

[GitHub] [airflow] hussein-awala commented on issue #34644: Invalid kube-config file. Expected key current-context in kube-config when using deferrable=True

2023-09-27 Thread via GitHub
hussein-awala commented on issue #34644: URL: https://github.com/apache/airflow/issues/34644#issuecomment-1738076701 > Expected key current-context in kube-config Deferrable and no deferrable operators use the exact same method to load the kube config file since 7.0.0, I'm surprised t

[GitHub] [airflow] jedcunningham opened a new pull request, #34655: Add helm chart 1.11.0 changelog for Artifacthub

2023-09-27 Thread via GitHub
jedcunningham opened a new pull request, #34655: URL: https://github.com/apache/airflow/pull/34655 🤦‍♂️ forgot to generate this for 1.11.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[airflow] branch main updated: Release notes for helm chart 1.11.0 (#34618)

2023-09-27 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 493431023f Release notes for helm chart 1.11

[GitHub] [airflow] jedcunningham merged pull request #34618: Release notes for helm chart 1.11.0

2023-09-27 Thread via GitHub
jedcunningham merged PR #34618: URL: https://github.com/apache/airflow/pull/34618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[GitHub] [airflow] Taragolis commented on a diff in pull request #34653: Clarify Amazon Lambda invocation and sensing

2023-09-27 Thread via GitHub
Taragolis commented on code in PR #34653: URL: https://github.com/apache/airflow/pull/34653#discussion_r1339197054 ## docs/apache-airflow-providers-amazon/operators/lambda.rst: ## @@ -68,10 +100,10 @@ Sensors .. _howto/sensor:LambdaFunctionStateSensor: -Wait on an Amazon La

[GitHub] [airflow] tnyz opened a new pull request, #34654: Remove required emr serverless operator param

2023-09-27 Thread via GitHub
tnyz opened a new pull request, #34654: URL: https://github.com/apache/airflow/pull/34654 as best practice users should be able to provision their serverless app using the config api without having to define/duplicate job type and release version in the airflow abstration

[GitHub] [airflow] hussein-awala merged pull request #34645: Use `airflow.models.dag.DAG` in Google system tests

2023-09-27 Thread via GitHub
hussein-awala merged PR #34645: URL: https://github.com/apache/airflow/pull/34645 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[GitHub] [airflow] ferruzzi commented on a diff in pull request #34653: Clarify Amazon Lambda invocation and sensing

2023-09-27 Thread via GitHub
ferruzzi commented on code in PR #34653: URL: https://github.com/apache/airflow/pull/34653#discussion_r1339187733 ## docs/apache-airflow-providers-amazon/operators/lambda.rst: ## @@ -57,6 +57,38 @@ Invoke an AWS Lambda function To invoke an AWS lambda function you can use :cl

[GitHub] [airflow] ferruzzi commented on a diff in pull request #34324: Executors doc update

2023-09-27 Thread via GitHub
ferruzzi commented on code in PR #34324: URL: https://github.com/apache/airflow/pull/34324#discussion_r1339180017 ## docs/apache-airflow/core-concepts/executor/index.rst: ## @@ -78,3 +71,134 @@ There are two types of executor - those that run tasks *locally* (inside the ``s ..

[GitHub] [airflow] ferruzzi commented on a diff in pull request #34324: Executors doc update

2023-09-27 Thread via GitHub
ferruzzi commented on code in PR #34324: URL: https://github.com/apache/airflow/pull/34324#discussion_r1339169842 ## docs/apache-airflow/core-concepts/executor/index.rst: ## @@ -78,3 +71,134 @@ There are two types of executor - those that run tasks *locally* (inside the ``s ..

[GitHub] [airflow] vincbeck commented on a diff in pull request #34324: Executors doc update

2023-09-27 Thread via GitHub
vincbeck commented on code in PR #34324: URL: https://github.com/apache/airflow/pull/34324#discussion_r1339165472 ## docs/apache-airflow/core-concepts/executor/index.rst: ## @@ -78,3 +71,134 @@ There are two types of executor - those that run tasks *locally* (inside the ``s ..

[GitHub] [airflow] JonnyWaffles commented on issue #31544: Possibly add a feature to "Run Tasks" from the UI

2023-09-27 Thread via GitHub
JonnyWaffles commented on issue #31544: URL: https://github.com/apache/airflow/issues/31544#issuecomment-1738010381 Hi team, apologies for the delayed response @potiuk! After reading through the security model, I don't really understand how force running a task is non-compliant, but t

[GitHub] [airflow] pierrejeambrun commented on pull request #34568: Fix: make dry run optional for patch task instance

2023-09-27 Thread via GitHub
pierrejeambrun commented on PR #34568: URL: https://github.com/apache/airflow/pull/34568#issuecomment-1738005108 Yep, what Hussein said, not a breaking change -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [airflow] vincbeck commented on a diff in pull request #34653: Clarify Amazon Lambda invocation and sensing

2023-09-27 Thread via GitHub
vincbeck commented on code in PR #34653: URL: https://github.com/apache/airflow/pull/34653#discussion_r1339135104 ## docs/apache-airflow-providers-amazon/operators/lambda.rst: ## @@ -57,6 +57,38 @@ Invoke an AWS Lambda function To invoke an AWS lambda function you can use :cl

[GitHub] [airflow] jedcunningham commented on a diff in pull request #34618: Release notes for helm chart 1.11.0

2023-09-27 Thread via GitHub
jedcunningham commented on code in PR #34618: URL: https://github.com/apache/airflow/pull/34618#discussion_r1339130019 ## chart/RELEASE_NOTES.rst: ## @@ -23,6 +23,111 @@ Run ``helm repo update`` before upgrading the chart to the latest version. .. towncrier release notes sta

[GitHub] [airflow] o-nikolas commented on a diff in pull request #34324: Executors doc update

2023-09-27 Thread via GitHub
o-nikolas commented on code in PR #34324: URL: https://github.com/apache/airflow/pull/34324#discussion_r1339125938 ## docs/apache-airflow/core-concepts/executor/debug.rst: ## @@ -15,8 +15,47 @@ specific language governing permissions and limitations under the License.

[GitHub] [airflow] o-nikolas commented on a diff in pull request #34324: Executors doc update

2023-09-27 Thread via GitHub
o-nikolas commented on code in PR #34324: URL: https://github.com/apache/airflow/pull/34324#discussion_r1339125938 ## docs/apache-airflow/core-concepts/executor/debug.rst: ## @@ -15,8 +15,47 @@ specific language governing permissions and limitations under the License.

[GitHub] [airflow] Taragolis opened a new pull request, #34653: Clarify Amazon Lambda invocation and sensing

2023-09-27 Thread via GitHub
Taragolis opened a new pull request, #34653: URL: https://github.com/apache/airflow/pull/34653 Follow up for troubleshooting in Slack - Add information how to deal with timeout error during sync Amazon Lambda invocation - Add information for workaround for sensing a

[GitHub] [airflow] uranusjr merged pull request #34637: Unify mapped task group lookup logic

2023-09-27 Thread via GitHub
uranusjr merged PR #34637: URL: https://github.com/apache/airflow/pull/34637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[airflow] branch main updated: Unify mapped task group lookup logic (#34637)

2023-09-27 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e5325cd2b7 Unify mapped task group lookup logic (

[GitHub] [airflow] hussein-awala commented on a diff in pull request #34652: Fix `dag.test()` and Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
hussein-awala commented on code in PR #34652: URL: https://github.com/apache/airflow/pull/34652#discussion_r1339084563 ## airflow/sensors/external_task.py: ## @@ -163,9 +163,11 @@ def __init__( total_states = set(self.allowed_states + self.skipped_states + self.failed_

[GitHub] [airflow] hussein-awala commented on a diff in pull request #34568: Fix: make dry run optional for patch task instance

2023-09-27 Thread via GitHub
hussein-awala commented on code in PR #34568: URL: https://github.com/apache/airflow/pull/34568#discussion_r1339079159 ## airflow/api_connexion/schemas/task_instance_schema.py: ## @@ -180,7 +180,7 @@ def validate_form(self, data, **kwargs): class SetSingleTaskInstanceStateFormS

[GitHub] [airflow] hussein-awala commented on pull request #34568: Fix: make dry run optional for patch task instance

2023-09-27 Thread via GitHub
hussein-awala commented on PR #34568: URL: https://github.com/apache/airflow/pull/34568#issuecomment-1737929509 > Also, are we considering this change to the default a breaking change to the API or a bugfix? > > CC @eladkal Based on this issue #34563, we cannot use the endpoint

[GitHub] [airflow] hussein-awala commented on a diff in pull request #34392: Mark tasks with `all_skipped` trigger rule as `skipped` if any task is in `upstream_failed` state

2023-09-27 Thread via GitHub
hussein-awala commented on code in PR #34392: URL: https://github.com/apache/airflow/pull/34392#discussion_r1339053143 ## airflow/ti_deps/deps/trigger_rule_dep.py: ## @@ -379,7 +379,7 @@ def _evaluate_direct_relatives() -> Iterator[TIDepStatus]: if skipped:

[GitHub] [airflow] TJaniF commented on pull request #34392: Mark tasks with `all_skipped` trigger rule as `skipped` if any task is in `upstream_failed` state

2023-09-27 Thread via GitHub
TJaniF commented on PR #34392: URL: https://github.com/apache/airflow/pull/34392#issuecomment-1737849365 Thank you! Hmm... good point! My thinking was that since currently a `failed` upstream task causes the all_skipped-task to end up in `skipped`, then `upstream_failed` should do the same

[GitHub] [airflow] jens-scheffler-bosch commented on pull request #34392: Mark tasks with `all_skipped` trigger rule as `skipped` if any task is in `upstream_failed` state

2023-09-27 Thread via GitHub
jens-scheffler-bosch commented on PR #34392: URL: https://github.com/apache/airflow/pull/34392#issuecomment-1737838802 PR from code and attempt looks good. But I would assume if upstream failed is the status of previous tasks, then the task under this fix should also rather be going into "u

[GitHub] [airflow] utkarsharma2 opened a new pull request, #34652: Respect 'soft_fail' argument when running ExternalTaskSensor

2023-09-27 Thread via GitHub
utkarsharma2 opened a new pull request, #34652: URL: https://github.com/apache/airflow/pull/34652 This PR intends to solve the issue that the soft_fail argument in ExternalTaskSensor is not respected. Also closes: https://github.com/apache/airflow/issues/34497 -- This is an aut

[GitHub] [airflow] Taragolis closed issue #34646: Connections and variables are not being created based on extraSecrets and extraConfigMaps

2023-09-27 Thread via GitHub
Taragolis closed issue #34646: Connections and variables are not being created based on extraSecrets and extraConfigMaps URL: https://github.com/apache/airflow/issues/34646 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] vincbeck commented on a diff in pull request #34349: Let auth managers provide their own API endpoints

2023-09-27 Thread via GitHub
vincbeck commented on code in PR #34349: URL: https://github.com/apache/airflow/pull/34349#discussion_r1338967863 ## airflow/auth/managers/base_auth_manager.py: ## @@ -50,6 +50,10 @@ def get_cli_commands() -> list[CLICommand]: """ return [] +def get_api_b

[GitHub] [airflow] Calder-Ty commented on pull request #34568: Fix: make dry run optional for patch task instance

2023-09-27 Thread via GitHub
Calder-Ty commented on PR #34568: URL: https://github.com/apache/airflow/pull/34568#issuecomment-1737802596 > Also, are we considering this change to the default a breaking change to the API or a bugfix? > > CC @eladkal Technically it is, however, I think we can be fairly sure

[GitHub] [airflow] o-nikolas commented on pull request #34568: Fix: make dry run optional for patch task instance

2023-09-27 Thread via GitHub
o-nikolas commented on PR #34568: URL: https://github.com/apache/airflow/pull/34568#issuecomment-1737793184 Also, are we considering this change to the default a breaking change to the API or a bugfix? CC @eladkal -- This is an automated message from the Apache Git Service. To res

[GitHub] [airflow] jedcunningham commented on a diff in pull request #34349: Let auth managers provide their own API endpoints

2023-09-27 Thread via GitHub
jedcunningham commented on code in PR #34349: URL: https://github.com/apache/airflow/pull/34349#discussion_r1338936861 ## airflow/auth/managers/base_auth_manager.py: ## @@ -50,6 +50,10 @@ def get_cli_commands() -> list[CLICommand]: """ return [] +def get_

[GitHub] [airflow] Taragolis commented on a diff in pull request #34627: Error handling for when Azure container log cannot be read in properly.

2023-09-27 Thread via GitHub
Taragolis commented on code in PR #34627: URL: https://github.com/apache/airflow/pull/34627#discussion_r1338907495 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -319,6 +319,9 @@ def _monitor_logging(self, resource_group: str, name: str) -> int:

[GitHub] [airflow] Taragolis commented on a diff in pull request #34627: Error handling for when Azure container log cannot be read in properly.

2023-09-27 Thread via GitHub
Taragolis commented on code in PR #34627: URL: https://github.com/apache/airflow/pull/34627#discussion_r1338907495 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -319,6 +319,9 @@ def _monitor_logging(self, resource_group: str, name: str) -> int:

[GitHub] [airflow] jedcunningham commented on pull request #34647: Move cli's Connection export and Variable export command print logic to a seprate function

2023-09-27 Thread via GitHub
jedcunningham commented on PR #34647: URL: https://github.com/apache/airflow/pull/34647#issuecomment-1737728724 @utkarsharma2, it lives here now: https://github.com/apache/airflow/blob/fb92ff8486f21b61a840ddc4414429c3a9adfc88/airflow/auth/managers/fab/cli_commands/user_command.py#L153 --

[GitHub] [airflow] jedcunningham closed issue #34482: KubernetesPodOperator shutting down istio sidecar but not deleting pod upon failure

2023-09-27 Thread via GitHub
jedcunningham closed issue #34482: KubernetesPodOperator shutting down istio sidecar but not deleting pod upon failure URL: https://github.com/apache/airflow/issues/34482 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[airflow] branch main updated: Fix Pod not being removed after istio-sidecar is removed (#34500)

2023-09-27 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new fb92ff8486 Fix Pod not being removed after i

[GitHub] [airflow] jedcunningham merged pull request #34500: Fix Pod not being removed after istio-sidecar is removed

2023-09-27 Thread via GitHub
jedcunningham merged PR #34500: URL: https://github.com/apache/airflow/pull/34500 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[airflow] branch main updated: Fail dag test if defer without triggerer (#34619)

2023-09-27 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e81bb48779 Fail dag test if defer without trigge

[GitHub] [airflow] dstandish merged pull request #34619: Fail dag test if defer without triggerer

2023-09-27 Thread via GitHub
dstandish merged PR #34619: URL: https://github.com/apache/airflow/pull/34619 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[GitHub] [airflow] krisfur commented on a diff in pull request #34627: Error handling for when Azure container log cannot be read in properly.

2023-09-27 Thread via GitHub
krisfur commented on code in PR #34627: URL: https://github.com/apache/airflow/pull/34627#discussion_r1338860691 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -319,6 +319,9 @@ def _monitor_logging(self, resource_group: str, name: str) -> int:

[GitHub] [airflow] krisfur commented on a diff in pull request #34627: Error handling for when Azure container log cannot be read in properly.

2023-09-27 Thread via GitHub
krisfur commented on code in PR #34627: URL: https://github.com/apache/airflow/pull/34627#discussion_r1338860691 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -319,6 +319,9 @@ def _monitor_logging(self, resource_group: str, name: str) -> int:

[GitHub] [airflow] SamWheating commented on pull request #34172: Add support for driver pool, instance flexibility policy, and min_num_instances for Dataproc

2023-09-27 Thread via GitHub
SamWheating commented on PR #34172: URL: https://github.com/apache/airflow/pull/34172#issuecomment-1737699234 I actually can't trigger the workflow - we'll have to wait for a committer to come along and do the final check + merge. -- This is an automated message from the Apache Git Servic

[GitHub] [airflow] Taragolis commented on a diff in pull request #34627: Error handling for when Azure container log cannot be read in properly.

2023-09-27 Thread via GitHub
Taragolis commented on code in PR #34627: URL: https://github.com/apache/airflow/pull/34627#discussion_r1338841760 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -319,6 +319,9 @@ def _monitor_logging(self, resource_group: str, name: str) -> int:

[GitHub] [airflow] krisfur commented on a diff in pull request #34627: Error handling for when Azure container log cannot be read in properly.

2023-09-27 Thread via GitHub
krisfur commented on code in PR #34627: URL: https://github.com/apache/airflow/pull/34627#discussion_r1338839397 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -319,6 +319,9 @@ def _monitor_logging(self, resource_group: str, name: str) -> int:

[GitHub] [airflow] krisfur commented on a diff in pull request #34627: Error handling for when Azure container log cannot be read in properly.

2023-09-27 Thread via GitHub
krisfur commented on code in PR #34627: URL: https://github.com/apache/airflow/pull/34627#discussion_r1338805545 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -319,6 +319,9 @@ def _monitor_logging(self, resource_group: str, name: str) -> int:

[GitHub] [airflow] Taragolis commented on a diff in pull request #34627: Error handling for when Azure container log cannot be read in properly.

2023-09-27 Thread via GitHub
Taragolis commented on code in PR #34627: URL: https://github.com/apache/airflow/pull/34627#discussion_r1338800572 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -319,6 +319,9 @@ def _monitor_logging(self, resource_group: str, name: str) -> int:

[GitHub] [airflow] Taragolis commented on a diff in pull request #34627: Error handling for when Azure container log cannot be read in properly.

2023-09-27 Thread via GitHub
Taragolis commented on code in PR #34627: URL: https://github.com/apache/airflow/pull/34627#discussion_r1338797309 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -319,6 +319,9 @@ def _monitor_logging(self, resource_group: str, name: str) -> int:

[airflow] branch main updated: Refactor dedent nested loops (#34409)

2023-09-27 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 07fe1d2a69 Refactor dedent nested loops (#34409)

[GitHub] [airflow] vincbeck merged pull request #34409: Refactor dedent nested loops

2023-09-27 Thread via GitHub
vincbeck merged PR #34409: URL: https://github.com/apache/airflow/pull/34409 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[GitHub] [airflow] vincbeck commented on pull request #34466: Add distinct function to MongoHook in apache-airflow-providers-mongo

2023-09-27 Thread via GitHub
vincbeck commented on PR #34466: URL: https://github.com/apache/airflow/pull/34466#issuecomment-1737631885 There are still some static checks failures. You should be able to fix them automatically by looking at [documentation](https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.r

[GitHub] [airflow] belaytzev commented on issue #34646: Connections and variables are not being created based on extraSecrets and extraConfigMaps

2023-09-27 Thread via GitHub
belaytzev commented on issue #34646: URL: https://github.com/apache/airflow/issues/34646#issuecomment-1737588388 > It not how it initially designed, airflow just lookup in configured storage during the execution, see [Secrets Backend](https://airflow.apache.org/docs/apache-airflow/stable/se

[GitHub] [airflow] vincbeck commented on a diff in pull request #34317: Use auth manager `is_authorized_` APIs to check user permissions in Rest API

2023-09-27 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1338731975 ## airflow/api_connexion/endpoints/dag_endpoint.py: ## @@ -180,7 +180,7 @@ def patch_dags(limit, session, offset=0, only_active=True, tags=None, dag_id_pat ret

[GitHub] [airflow] vincbeck commented on a diff in pull request #34317: Use auth manager `is_authorized_` APIs to check user permissions in Rest API

2023-09-27 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1338734563 ## airflow/api_connexion/endpoints/xcom_endpoint.py: ## @@ -39,14 +39,7 @@ from airflow.api_connexion.types import APIResponse -@security.requires_access( -

[GitHub] [airflow] vincbeck commented on a diff in pull request #34317: Use auth manager `is_authorized_` APIs to check user permissions in Rest API

2023-09-27 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1338731975 ## airflow/api_connexion/endpoints/dag_endpoint.py: ## @@ -180,7 +180,7 @@ def patch_dags(limit, session, offset=0, only_active=True, tags=None, dag_id_pat ret

[GitHub] [airflow] Taragolis commented on issue #34646: Connections and variables are not being created based on extraSecrets and extraConfigMaps

2023-09-27 Thread via GitHub
Taragolis commented on issue #34646: URL: https://github.com/apache/airflow/issues/34646#issuecomment-1737541634 It not how it initially designed, airflow just lookup in configured storage during the execution, see [Secrets Backend](https://airflow.apache.org/docs/apache-airflow/stable/secu

[GitHub] [airflow] vincbeck commented on a diff in pull request #34349: Let auth managers provide their own API endpoints

2023-09-27 Thread via GitHub
vincbeck commented on code in PR #34349: URL: https://github.com/apache/airflow/pull/34349#discussion_r1338719267 ## airflow/auth/managers/base_auth_manager.py: ## @@ -50,6 +50,10 @@ def get_cli_commands() -> list[CLICommand]: """ return [] +def get_api_b

[GitHub] [airflow] phanikumv commented on a diff in pull request #34643: DatabricksHook ClusterState

2023-09-27 Thread via GitHub
phanikumv commented on code in PR #34643: URL: https://github.com/apache/airflow/pull/34643#discussion_r1338695414 ## tests/providers/databricks/hooks/test_databricks.py: ## @@ -158,6 +162,11 @@ def repair_run_endpoint(host): """ return f"https://{host}/api/2.1/jobs/ru

[GitHub] [airflow] fuatcakici commented on issue #34604: AttributeError while using the function update_state

2023-09-27 Thread via GitHub
fuatcakici commented on issue #34604: URL: https://github.com/apache/airflow/issues/34604#issuecomment-1737453705 I have potentially also found another error which you may want to know. After manually setting the `simple_task` in the first DAG run to FAILED, if I manually set that whole DAG

[GitHub] [airflow] mnacharov commented on a diff in pull request #34342: Chart: added basic support git-sync v4.0.0+

2023-09-27 Thread via GitHub
mnacharov commented on code in PR #34342: URL: https://github.com/apache/airflow/pull/34342#discussion_r1338645269 ## airflow/providers/cncf/kubernetes/pod_template_file_examples/git_sync_v4_template.yaml: ## Review Comment: ok -- This is an automated message from the A

[GitHub] [airflow] mnacharov commented on a diff in pull request #34342: Chart: added basic support git-sync v4.0.0+

2023-09-27 Thread via GitHub
mnacharov commented on code in PR #34342: URL: https://github.com/apache/airflow/pull/34342#discussion_r1338644564 ## chart/templates/_helpers.yaml: ## @@ -197,6 +197,7 @@ If release name contains chart name it will be used as a full name. imagePullPolicy: {{ .Values.images.

  1   2   >