Re: [PR] Add indexes on dag_id column in refencing tables to speed up deletion of dag records [airflow]

2024-05-13 Thread via GitHub
pankajkoti commented on code in PR #39526: URL: https://github.com/apache/airflow/pull/39526#discussion_r1598002824 ## airflow/migrations/versions/0143_2_9_2_add_index_on_dag_id_in_referencing_tables.py: ## @@ -0,0 +1,69 @@ +# +# Licensed to the Apache Software Foundation (ASF)

Re: [I] `Rendered Template` link missing from `Details` tab [airflow]

2024-05-13 Thread via GitHub
nathadfield commented on issue #39527: URL: https://github.com/apache/airflow/issues/39527#issuecomment-2106873465 @bbovenzi Great thanks! I'll change this to a feature request then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] BigQuery task decorated functions failing in Airflow 2.9.1 [airflow]

2024-05-13 Thread via GitHub
nathadfield commented on issue #39541: URL: https://github.com/apache/airflow/issues/39541#issuecomment-2106915100 @pankajastro Interesting. Downgrading does work but I'm curious to know what issue is and why the hook method gives the error but the operator does not. -- This is an automa

Re: [PR] feat: enable mysql keda support for triggerer [airflow]

2024-05-13 Thread via GitHub
claneys commented on PR #37365: URL: https://github.com/apache/airflow/pull/37365#issuecomment-2106948231 Hi, Is there anything blocking this PR to be merge ? I might miss something. Thanks :pray: cc: @jedcunningham -- This is an automated message from the Apache Git Servi

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
kacpermuda commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598085567 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener fr

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
mobuchowski commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598114661 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener f

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
mobuchowski commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598122196 ## tests/providers/openlineage/plugins/test_openlineage.py: ## @@ -131,7 +131,7 @@ def teardown_method(self): ), ], ) -def test_plu

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598127409 ## tests/providers/openlineage/plugins/test_openlineage.py: ## @@ -131,7 +131,7 @@ def teardown_method(self): ), ], ) -def test_plugin_d

Re: [PR] Chart: Allow AWS ECS Executor [airflow]

2024-05-13 Thread via GitHub
LipuFei commented on code in PR #38524: URL: https://github.com/apache/airflow/pull/38524#discussion_r1598132532 ## chart/README.md: ## @@ -37,6 +37,9 @@ cluster using the [Helm](https://helm.sh) package manager. ## Features * Supported executors: ``LocalExecutor``, ``Celery

Re: [PR] Chart: Allow AWS ECS Executor [airflow]

2024-05-13 Thread via GitHub
LipuFei commented on PR #38524: URL: https://github.com/apache/airflow/pull/38524#issuecomment-2107044378 > Thanks @LipuFei. Can you add tests for this also? Might be worth running around looking at LocalExecutor cases, as that's the most similar from the charts perspective I think. >

Re: [PR] Chart: Allow AWS ECS Executor [airflow]

2024-05-13 Thread via GitHub
LipuFei commented on code in PR #38524: URL: https://github.com/apache/airflow/pull/38524#discussion_r1598132924 ## chart/templates/scheduler/scheduler-deployment.yaml: ## @@ -53,7 +53,7 @@ metadata: release: {{ .Release.Name }} chart: "{{ .Chart.Name }}-{{ .Chart.Vers

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598138877 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener from a

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598138877 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener from a

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
kacpermuda commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598144566 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener fr

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
kacpermuda commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598144566 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener fr

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598164584 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener from a

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598166848 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener from a

Re: [PR] add deferrable support to `DatabricksNotebookOperator` [airflow]

2024-05-13 Thread via GitHub
rawwar commented on code in PR #39295: URL: https://github.com/apache/airflow/pull/39295#discussion_r1598171177 ## tests/providers/databricks/operators/test_databricks.py: ## @@ -1865,6 +1865,46 @@ def test_execute_without_wait_for_termination(self): operator.launch_not

Re: [PR] Fix validation for BQ label values for BigQueryInsertJobOperator [airflow]

2024-05-13 Thread via GitHub
kisssam commented on PR #39568: URL: https://github.com/apache/airflow/pull/39568#issuecomment-2107122252 On testing on a Cloud Composer environment with `apache-airflow-providers-google==10.18.0rc2`, all other test cases except the scenario where a task starts with a hyphen fails.

Re: [I] Unknown error in KubernetesJobWatcher. Failing [airflow]

2024-05-13 Thread via GitHub
barun-mazumdar commented on issue #33066: URL: https://github.com/apache/airflow/issues/33066#issuecomment-2107124499 same issue here as well with kube 1.28.5 and airflow 2.8.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
kacpermuda commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598202292 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener fr

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
kacpermuda commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598215772 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener fr

[PR] Pin google-cloud-bigquery to < 3.21.0 [airflow]

2024-05-13 Thread via GitHub
pankajastro opened a new pull request, #39583: URL: https://github.com/apache/airflow/pull/39583 related-to: https://github.com/apache/airflow/issues/39541 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/

Re: [PR] Pin google-cloud-bigquery to < 3.21.0 [airflow]

2024-05-13 Thread via GitHub
pankajastro commented on code in PR #39583: URL: https://github.com/apache/airflow/pull/39583#discussion_r1598218115 ## airflow/providers/google/provider.yaml: ## @@ -113,7 +113,7 @@ dependencies: - google-auth-httplib2>=0.0.1 - google-cloud-aiplatform>=1.42.1 - google-

Re: [I] BigQuery task decorated functions failing in Airflow 2.9.1 [airflow]

2024-05-13 Thread via GitHub
pankajastro commented on issue #39541: URL: https://github.com/apache/airflow/issues/39541#issuecomment-2107167940 It is because the way we pass `nowait` param to `insert_job` method. When you are using @task then `nowait` is False but when you are using operator it is True https:/

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598221236 ## tests/providers/openlineage/plugins/test_listener.py: ## @@ -33,10 +33,20 @@ from airflow.providers.openlineage.plugins.listener import OpenLineageListener from a

Re: [I] BigQuery task decorated functions failing in Airflow 2.9.1 [airflow]

2024-05-13 Thread via GitHub
nathadfield commented on issue #39541: URL: https://github.com/apache/airflow/issues/39541#issuecomment-2107223643 I see. Thanks for the info. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Status of testing Providers that were prepared on May 12, 2024 [airflow]

2024-05-13 Thread via GitHub
dabla commented on issue #39578: URL: https://github.com/apache/airflow/issues/39578#issuecomment-2107229846 Hello, I tested following PR and they are all working a expected: - [38111](https://github.com/apache/airflow/pull/38111) - [39237](https://github.com/apache/airflow/pull/392

[I] ExternalPythonOperator Jinja template rendering exception when op_kwargs dictionary value contain special characters [airflow]

2024-05-13 Thread via GitHub
samodelkinas opened a new issue, #39584: URL: https://github.com/apache/airflow/issues/39584 ### Apache Airflow version 2.9.1 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? Airflow DAG throws an exception while calling

Re: [I] ExternalPythonOperator Jinja template rendering exception when op_kwargs dictionary value contain special characters [airflow]

2024-05-13 Thread via GitHub
boring-cyborg[bot] commented on issue #39584: URL: https://github.com/apache/airflow/issues/39584#issuecomment-2107245331 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[PR] fix(baseoperator): change start_trigger into start_trigger_cls and start_trigger_kwargs [airflow]

2024-05-13 Thread via GitHub
Lee-W opened a new pull request, #39585: URL: https://github.com/apache/airflow/pull/39585 … --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests

Re: [PR] remove extra return in `get_conn` of mssql provider [airflow]

2024-05-13 Thread via GitHub
Lee-W commented on PR #39582: URL: https://github.com/apache/airflow/pull/39582#issuecomment-2107263872 Kinda curious why this is not caught by linters 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] remove extra return in `get_conn` of mssql provider [airflow]

2024-05-13 Thread via GitHub
Lee-W merged PR #39582: URL: https://github.com/apache/airflow/pull/39582 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.ap

(airflow) branch main updated: remove extra return (#39582)

2024-05-13 Thread weilee
This is an automated email from the ASF dual-hosted git repository. weilee pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new dc61da2509 remove extra return (#39582) dc61da2509

Re: [PR] Pin google-cloud-bigquery to < 3.21.0 [airflow]

2024-05-13 Thread via GitHub
potiuk commented on PR #39583: URL: https://github.com/apache/airflow/pull/39583#issuecomment-2107275224 FYI. Whenever we put upper-binding, we should always make a comment and add link to the issue that should remain open - describing the problem. Ideally maybe even opening it in the offen

Re: [PR] Fix Mark Instance state buttons stay disabled if user lacks permission (#37451) [airflow]

2024-05-13 Thread via GitHub
RodrigoGanancia commented on PR #38732: URL: https://github.com/apache/airflow/pull/38732#issuecomment-2107277260 Hey, the code is now ready, since the fix is pretty simple I removed the test case which was not needed. I'm sorry for the latest commit, It's my first time contributing and I m

[I] Making `mssql` provider to support only `mssql+pymssql` dialect [airflow]

2024-05-13 Thread via GitHub
rawwar opened a new issue, #39586: URL: https://github.com/apache/airflow/issues/39586 ### Description Currently, we allow users to pass dialect via `sqlalchemy_scheme` extra field of the `mssql` connections. I am proposing to remove this field as it is not possible to support multip

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598335884 ## tests/providers/openlineage/plugins/test_openlineage.py: ## @@ -131,7 +131,7 @@ def teardown_method(self): ), ], ) -def test_plugin_d

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598335884 ## tests/providers/openlineage/plugins/test_openlineage.py: ## @@ -131,7 +131,7 @@ def teardown_method(self): ), ], ) -def test_plugin_d

Re: [I] ExternalPythonOperator Jinja template rendering exception when op_kwargs dictionary value contain special characters [airflow]

2024-05-13 Thread via GitHub
rawwar commented on issue #39584: URL: https://github.com/apache/airflow/issues/39584#issuecomment-2107431272 I was able to replicate the issue. But, I am not sure if this is a valid issue. I was also able to replicate this for a usual PythonOperator. Because, all jinja template variables

Re: [I] Status of testing Providers that were prepared on May 12, 2024 [airflow]

2024-05-13 Thread via GitHub
VladaZakharova commented on issue #39578: URL: https://github.com/apache/airflow/issues/39578#issuecomment-2107454520 Hi! https://github.com/apache/airflow/pull/39206 works as expected -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] add missing conn_id to string representation of ObjectStoragePath [airflow]

2024-05-13 Thread via GitHub
TJaniF commented on PR #39313: URL: https://github.com/apache/airflow/pull/39313#issuecomment-2107515331 Hello @eladkal 🙂 Very sorry for the late poke about this but is there a chance to include a 1.3.2 release of the common io provider with this bugfix in the current wave of releas

Re: [I] ExternalPythonOperator Jinja template rendering exception when op_kwargs dictionary value contain special characters [airflow]

2024-05-13 Thread via GitHub
rawwar commented on issue #39584: URL: https://github.com/apache/airflow/issues/39584#issuecomment-2107580559 I looked into this, and the following is my analysis. This error will happen whenever we are parsing values using `jinja_env.from_string(value)` . This happens when the string

Re: [I] ExternalPythonOperator Jinja template rendering exception when op_kwargs dictionary value contain special characters [airflow]

2024-05-13 Thread via GitHub
rawwar commented on issue #39584: URL: https://github.com/apache/airflow/issues/39584#issuecomment-2107594170 @potiuk , I think the solution for this is not a code fix. Rather, update the documentation to inform users that the values passed to `op_args` and `op_kwargs` should follow jinja s

Re: [PR] Chart: Enable templating in extraContainers and extraInitContainers [airflow]

2024-05-13 Thread via GitHub
mewa commented on PR #38507: URL: https://github.com/apache/airflow/pull/38507#issuecomment-2107640878 Rebased on main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[PR] Move Kubernetes cli to provider package [airflow]

2024-05-13 Thread via GitHub
romsharon98 opened a new pull request, #39587: URL: https://github.com/apache/airflow/pull/39587 closes: https://github.com/apache/airflow/issues/39199 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apac

Re: [PR] Fix Pod Number Exception in the sync mode if reattach_on_restart parameter is False [airflow]

2024-05-13 Thread via GitHub
e-galan commented on code in PR #39329: URL: https://github.com/apache/airflow/pull/39329#discussion_r1598601888 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -1129,6 +1142,36 @@ def dry_run(self) -> None: def execute_complete(self, context: Context, event: d

Re: [PR] Fix Pod Number Exception in the sync mode if reattach_on_restart parameter is False [airflow]

2024-05-13 Thread via GitHub
e-galan commented on code in PR #39329: URL: https://github.com/apache/airflow/pull/39329#discussion_r1598602503 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -1129,6 +1142,36 @@ def dry_run(self) -> None: def execute_complete(self, context: Context, event: d

Re: [I] ExternalPythonOperator Jinja template rendering exception when op_kwargs dictionary value contain special characters [airflow]

2024-05-13 Thread via GitHub
potiuk commented on issue #39584: URL: https://github.com/apache/airflow/issues/39584#issuecomment-2107822246 > @potiuk , I think the solution for this is not a code fix. Rather, update the documentation to inform users that the values passed to `op_args` and `op_kwargs` should follow jinja

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598615598 ## tests/providers/openlineage/plugins/test_openlineage.py: ## @@ -131,7 +131,7 @@ def teardown_method(self): ), ], ) -def test_plugin_d

Re: [PR] Fix custom actions in security manager `has_access` [airflow]

2024-05-13 Thread via GitHub
vincbeck commented on PR #39421: URL: https://github.com/apache/airflow/pull/39421#issuecomment-2107860287 My bad! Thanks for fixing it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598627080 ## tests/providers/openlineage/plugins/test_openlineage.py: ## @@ -131,7 +131,7 @@ def teardown_method(self): ), ], ) -def test_plugin_d

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598628805 ## tests/providers/smtp/notifications/test_smtp.py: ## @@ -129,7 +133,7 @@ def test_notifier_with_defaults(self, mock_smtphook_hook, create_task_instance):

Re: [PR] Fix `xmlsec` version pin [airflow]

2024-05-13 Thread via GitHub
vincbeck commented on PR #39528: URL: https://github.com/apache/airflow/pull/39528#issuecomment-2107990207 Sorry I was away for the week so I could not really help here. As mentioned by Jed, the only reason `xmlsec` was part of the image is because it is a dependency of `python3-saml`. We d

[PR] Add `parameters` as template field for SqlSensor [airflow]

2024-05-13 Thread via GitHub
josh-fell opened a new pull request, #39588: URL: https://github.com/apache/airflow/pull/39588 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[I] UI bug with `DAG Docs` display in `DAG Run` Triggering page [airflow]

2024-05-13 Thread via GitHub
pedro-cf opened a new issue, #39589: URL: https://github.com/apache/airflow/issues/39589 ### Apache Airflow version 2.9.1 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? When triggering a `DAG Run` of a `DAG` that inclu

Re: [I] Failure on editing a Variable [airflow]

2024-05-13 Thread via GitHub
raphaelauv commented on issue #39581: URL: https://github.com/apache/airflow/issues/39581#issuecomment-2108091448 I also seen this bug before 2.9.1 with LDAP -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on PR #39513: URL: https://github.com/apache/airflow/pull/39513#issuecomment-2108099753 All good and green here :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on PR #39513: URL: https://github.com/apache/airflow/pull/39513#issuecomment-2108101776 ![image](https://github.com/apache/airflow/assets/595491/0b199e24-82ee-46fd-8404-4cfd7e8032c8) -- This is an automated message from the Apache Git Service. To respond to the messag

(airflow) branch main updated (dc61da2509 -> ce4e8477f1)

2024-05-13 Thread joshfell
This is an automated email from the ASF dual-hosted git repository. joshfell pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from dc61da2509 remove extra return (#39582) add ce4e8477f1 Add `parameters` as template field for SqlSensor (#39588)

Re: [PR] Add `parameters` as template field for SqlSensor [airflow]

2024-05-13 Thread via GitHub
josh-fell merged PR #39588: URL: https://github.com/apache/airflow/pull/39588 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [I] Status of testing Providers that were prepared on May 12, 2024 [airflow]

2024-05-13 Thread via GitHub
wlinamchurch commented on issue #39578: URL: https://github.com/apache/airflow/issues/39578#issuecomment-2108122956 https://github.com/apache/airflow/pull/38911 works as expected -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
o-nikolas commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598727430 ## tests/providers/amazon/aws/executors/ecs/test_ecs_executor.py: ## @@ -367,6 +368,7 @@ def test_stopped_tasks(self): class TestAwsEcsExecutor: """Tests the

[PR] Handle task adoption for batch executor [airflow]

2024-05-13 Thread via GitHub
vincbeck opened a new pull request, #39590: URL: https://github.com/apache/airflow/pull/39590 Add the feature to adopt abandoned tasks in the Batch executor. Similar to #37786 but with Batch. --- **^ Add meaningful description above** Read the **[Pull

Re: [PR] [OpenLineage] Add more debug logs to facilitate debugging [airflow]

2024-05-13 Thread via GitHub
mobuchowski merged PR #39136: URL: https://github.com/apache/airflow/pull/39136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airf

(airflow) branch main updated: chore: Add more OpenLineage logs to facilitate debugging (#39136)

2024-05-13 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8bc6c32366 chore: Add more OpenLineage logs to

Re: [I] mypy errors - Argument 1 has incompatible type "XComArg" [airflow]

2024-05-13 Thread via GitHub
jonathan-ostrander commented on issue #39514: URL: https://github.com/apache/airflow/issues/39514#issuecomment-2108281294 This was solved by https://github.com/apache/airflow/pull/26023 but that code was moved under the `dev` directory by https://github.com/apache/airflow/pull/28498. AFAIK

Re: [PR] Update example AzureContainerInstancesOperator [airflow]

2024-05-13 Thread via GitHub
Kylea650 commented on PR #39466: URL: https://github.com/apache/airflow/pull/39466#issuecomment-2108288124 > Can you fix static checks? Should be good now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Amazon Bedrock - Retrieve and RetrieveAndGenerate [airflow]

2024-05-13 Thread via GitHub
ferruzzi commented on code in PR #39500: URL: https://github.com/apache/airflow/pull/39500#discussion_r1598804403 ## tests/system/providers/amazon/aws/example_bedrock_knowledge_base.py: ## @@ -480,6 +483,24 @@ def delete_opensearch_policies(collection_name: str): ) # [

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-13 Thread via GitHub
potiuk commented on code in PR #39513: URL: https://github.com/apache/airflow/pull/39513#discussion_r1598811009 ## tests/providers/amazon/aws/executors/ecs/test_ecs_executor.py: ## @@ -367,6 +368,7 @@ def test_stopped_tasks(self): class TestAwsEcsExecutor: """Tests the AWS

Re: [PR] Amazon Bedrock - Retrieve and RetrieveAndGenerate [airflow]

2024-05-13 Thread via GitHub
o-nikolas commented on code in PR #39500: URL: https://github.com/apache/airflow/pull/39500#discussion_r1598821701 ## tests/system/providers/amazon/aws/example_bedrock_knowledge_base.py: ## @@ -480,6 +483,24 @@ def delete_opensearch_policies(collection_name: str): ) #

Re: [PR] Move Kubernetes cli to provider package [airflow]

2024-05-13 Thread via GitHub
shahar1 commented on code in PR #39587: URL: https://github.com/apache/airflow/pull/39587#discussion_r1598825622 ## airflow/cli/commands/kubernetes_command.py: ## @@ -36,6 +37,12 @@ from airflow.utils.cli import get_dag from airflow.utils.providers_configuration_loader import

Re: [PR] Amazon Bedrock - Retrieve and RetrieveAndGenerate [airflow]

2024-05-13 Thread via GitHub
ferruzzi commented on code in PR #39500: URL: https://github.com/apache/airflow/pull/39500#discussion_r1598842673 ## tests/system/providers/amazon/aws/example_bedrock_knowledge_base.py: ## @@ -480,6 +483,24 @@ def delete_opensearch_policies(collection_name: str): ) # [

Re: [PR] Amazon Bedrock - Retrieve and RetrieveAndGenerate [airflow]

2024-05-13 Thread via GitHub
o-nikolas commented on code in PR #39500: URL: https://github.com/apache/airflow/pull/39500#discussion_r1598853590 ## tests/system/providers/amazon/aws/example_bedrock_knowledge_base.py: ## @@ -480,6 +483,24 @@ def delete_opensearch_policies(collection_name: str): ) #

(airflow) branch main updated: Task adoption for hybrid executors (#39531)

2024-05-13 Thread onikolas
This is an automated email from the ASF dual-hosted git repository. onikolas pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 3e229d8df5 Task adoption for hybrid executors (#3

Re: [PR] AIP-61 - Support task adoption for hybrid executors [airflow]

2024-05-13 Thread via GitHub
o-nikolas merged PR #39531: URL: https://github.com/apache/airflow/pull/39531 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[PR] Remove compat code for 2.7.0 - its now the min Airflow version [airflow]

2024-05-13 Thread via GitHub
jedcunningham opened a new pull request, #39591: URL: https://github.com/apache/airflow/pull/39591 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[PR] Introduce Amazon Comprehend Service [airflow]

2024-05-13 Thread via GitHub
gopidesupavan opened a new pull request, #39592: URL: https://github.com/apache/airflow/pull/39592 Added Amazon Comprehend Start Pii Entities Detection Job Operator Doc, Hook, Operator, Sensor, Trigger, Waiter, Unit Test, System Test. At present it supports only Pii

Re: [PR] Chart: Allow AWS ECS Executor [airflow]

2024-05-13 Thread via GitHub
jedcunningham commented on PR #38524: URL: https://github.com/apache/airflow/pull/38524#issuecomment-2108682949 For rbac I think thinking of [this](https://github.com/apache/airflow/blob/ce4e8477f1972400dda228f802e8f501de5dbe49/chart/templates/rbac/pod-launcher-rolebinding.yaml#L24-L25), bu

Re: [PR] Introduce Amazon Comprehend Service [airflow]

2024-05-13 Thread via GitHub
vincbeck commented on code in PR #39592: URL: https://github.com/apache/airflow/pull/39592#discussion_r1599010536 ## tests/system/providers/amazon/aws/example_comprehend.py: ## @@ -0,0 +1,140 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [I] Status of testing Providers that were prepared on May 12, 2024 [airflow]

2024-05-13 Thread via GitHub
dondaum commented on issue #39578: URL: https://github.com/apache/airflow/issues/39578#issuecomment-2108697474 #38961 is good ✅ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Introduce Amazon Comprehend Service [airflow]

2024-05-13 Thread via GitHub
gopidesupavan commented on code in PR #39592: URL: https://github.com/apache/airflow/pull/39592#discussion_r1599023840 ## tests/system/providers/amazon/aws/example_comprehend.py: ## @@ -0,0 +1,140 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contri

Re: [PR] Allow FTPHook to change port number [airflow]

2024-05-13 Thread via GitHub
vincbeck commented on code in PR #39465: URL: https://github.com/apache/airflow/pull/39465#discussion_r1599058827 ## airflow/providers/ftp/hooks/ftp.py: ## @@ -58,7 +58,11 @@ def get_conn(self) -> ftplib.FTP: if self.conn is None: params = self.get_connecti

Re: [PR] Handle task adoption for batch executor [airflow]

2024-05-13 Thread via GitHub
hussein-awala commented on code in PR #39590: URL: https://github.com/apache/airflow/pull/39590#discussion_r1599119239 ## airflow/providers/amazon/aws/executors/batch/batch_executor.py: ## @@ -306,14 +307,22 @@ def attempt_submit_jobs(self): self.pending_job

[PR] Fix spelling satiny -> sanity [airflow]

2024-05-13 Thread via GitHub
dstandish opened a new pull request, #39594: URL: https://github.com/apache/airflow/pull/39594 That is all -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[PR] Skip get_current_task_instance_session if use_internal_api [airflow]

2024-05-13 Thread via GitHub
dstandish opened a new pull request, #39595: URL: https://github.com/apache/airflow/pull/39595 We don't need a session when configured to use the internal API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Remove compat code for 2.7.0 - its now the min Airflow version [airflow]

2024-05-13 Thread via GitHub
jedcunningham merged PR #39591: URL: https://github.com/apache/airflow/pull/39591 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated: Remove compat code for 2.7.0 - its now the min Airflow version (#39591)

2024-05-13 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e3897dcbed Remove compat code for 2.7.0 - it

[PR] Fix stacklevel for _log_state helper [airflow]

2024-05-13 Thread via GitHub
dstandish opened a new pull request, #39596: URL: https://github.com/apache/airflow/pull/39596 This is necessary to show what line the helper was invoked from. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Fix spelling satiny -> sanity [airflow]

2024-05-13 Thread via GitHub
jedcunningham merged PR #39594: URL: https://github.com/apache/airflow/pull/39594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated: Fix spelling satiny -> sanity (#39594)

2024-05-13 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4a59276187 Fix spelling satiny -> sanity (#3

[PR] Fix typo in Auth Manager docs [airflow]

2024-05-13 Thread via GitHub
tomakita opened a new pull request, #39598: URL: https://github.com/apache/airflow/pull/39598 Fixed a minor typo in the `apache-airflow` docs for Auth Managers. This is my first commit -- let me know if I did something wrong, and I'll fix it ASAP :) --- **^ Add

Re: [PR] Fix typo in Auth Manager docs [airflow]

2024-05-13 Thread via GitHub
boring-cyborg[bot] commented on PR #39598: URL: https://github.com/apache/airflow/pull/39598#issuecomment-2108968633 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

Re: [PR] Introduce Amazon Comprehend Service [airflow]

2024-05-13 Thread via GitHub
o-nikolas commented on code in PR #39592: URL: https://github.com/apache/airflow/pull/39592#discussion_r1599187322 ## airflow/providers/amazon/aws/operators/comprehend.py: ## @@ -0,0 +1,195 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor l

Re: [I] Dynamic mapped tasks group arguments are interpreted as MappedArgument when provided to classic operators [airflow]

2024-05-13 Thread via GitHub
joshtree41 commented on issue #39222: URL: https://github.com/apache/airflow/issues/39222#issuecomment-2109022868 Noticing this issue as well, tested a task group with task flow API and the arguments get picked up properly, but when passed to traditional operators they don't get picked up p

Re: [I] sla_miss_callback is not working with 5 input parameters as official document, only dag_id 1 input is allowed, no enough context for alert [airflow]

2024-05-13 Thread via GitHub
github-actions[bot] commented on issue #38931: URL: https://github.com/apache/airflow/issues/38931#issuecomment-2109035884 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [PR] Improve logging readability with DockerOperator [airflow]

2024-05-13 Thread via GitHub
github-actions[bot] closed pull request #38008: Improve logging readability with DockerOperator URL: https://github.com/apache/airflow/pull/38008 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Update example AzureContainerInstancesOperator [airflow]

2024-05-13 Thread via GitHub
josh-fell merged PR #39466: URL: https://github.com/apache/airflow/pull/39466 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated: Update example AzureContainerInstancesOperator (#39466)

2024-05-13 Thread joshfell
This is an automated email from the ASF dual-hosted git repository. joshfell pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ecf6e57e27 Update example AzureContainerInstances

Re: [PR] Fix typo in Auth Manager docs [airflow]

2024-05-13 Thread via GitHub
josh-fell merged PR #39598: URL: https://github.com/apache/airflow/pull/39598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

  1   2   >