Re: [I] Type hinting in `DatabricksSubmitRunOperator` is not compatible with parameter passing possibilities [airflow]

2024-08-06 Thread via GitHub
github-actions[bot] commented on issue #29931: URL: https://github.com/apache/airflow/issues/29931#issuecomment-2270529678 This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activit

Re: [PR] Fix DateTimeOperator tests in Database Isolation Mode [airflow]

2024-08-06 Thread via GitHub
potiuk merged PR #41271: URL: https://github.com/apache/airflow/pull/41271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Fix DateTimeOperator tests in Database Isolation Mode (#41271)

2024-08-06 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6229c1c98a Fix DateTimeOperator tests in Database I

Re: [PR] Don't reserialize DAG if no revision's applied [airflow]

2024-08-06 Thread via GitHub
potiuk commented on code in PR #41207: URL: https://github.com/apache/airflow/pull/41207#discussion_r1705037213 ## airflow/utils/db.py: ## @@ -1683,8 +1686,10 @@ def upgradedb( os.environ["AIRFLOW__DATABASE__SQL_ALCHEMY_MAX_SIZE"] = val settings.rec

Re: [PR] Swallow exception when mini-scheduling raises an exception [airflow]

2024-08-06 Thread via GitHub
potiuk commented on code in PR #41260: URL: https://github.com/apache/airflow/pull/41260#discussion_r1705044857 ## airflow/models/taskinstance.py: ## @@ -3892,9 +3892,17 @@ def schedule_downstream_tasks(self, session: Session = NEW_SESSION, max_tis_per_ :meta: privat

[PR] Change the format of inserted airflow version in "update-migration-references" command from airflow_version='...' to airflow_version="..." [airflow]

2024-08-06 Thread via GitHub
Lee-W opened a new pull request, #41275: URL: https://github.com/apache/airflow/pull/41275 the original one won't pass the ruff linter --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/ai

[PR] Move Jens to PMC members [airflow-site]

2024-08-06 Thread via GitHub
potiuk opened a new pull request, #1051: URL: https://github.com/apache/airflow-site/pull/1051 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

(airflow-site) branch add-jens-to-pmc-members created (now 5a79f092f9)

2024-08-06 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-jens-to-pmc-members in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 5a79f092f9 Move Jens to PMC members No new revisions were added by this update.

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-08-06 Thread via GitHub
trlopes1974 commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2270628845 just got one... Dagrun Running | Task instance's dagrun was not in the 'running' state but in the state 'failed'. -- | -- Task Instance State | Task is

Re: [I] [BUG] nested task group info unavailable in task OL event [airflow]

2024-08-06 Thread via GitHub
rahul-madaan closed issue #41261: [BUG] nested task group info unavailable in task OL event URL: https://github.com/apache/airflow/issues/41261 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [I] [BUG] nested task group info unavailable in task OL event [airflow]

2024-08-06 Thread via GitHub
rahul-madaan commented on issue #41261: URL: https://github.com/apache/airflow/issues/41261#issuecomment-2270637112 Thanks for the explanation @kacpermuda, closing this issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-08-06 Thread via GitHub
trlopes1974 commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2270649074 #41276 to take care of the "2" cenario has the "1" has a solution now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[I] Executor reports task instance finished (failed) although the task says it's queued. (Info: None) Was the task killed externally [airflow]

2024-08-06 Thread via GitHub
trlopes1974 opened a new issue, #41276: URL: https://github.com/apache/airflow/issues/41276 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.9.2 ### What happened? Case 2 of iss

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-08-06 Thread via GitHub
trlopes1974 commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2270651176 @potiuk , if you guide-me, I can deploy the fix to our production env and see if "1" goes away -- This is an automated message from the Apache Git Service. To respond to the

[PR] Fix tests/operators/test_bash.py and branch_operator.py for Database Isolation Mode [airflow]

2024-08-06 Thread via GitHub
jscheffl opened a new pull request, #41277: URL: https://github.com/apache/airflow/pull/41277 Related: https://github.com/apache/airflow/pull/41067 Next one: Fixing test_bash.py and test_branch_operator.py -- This is an automated message from the Apache Git Service. To respond to

[I] Clearing a Mapped Task (with Downstream included) clears all Downstream tasks instead of mapped downstream tasks [airflow]

2024-08-06 Thread via GitHub
gavinhonl opened a new issue, #41278: URL: https://github.com/apache/airflow/issues/41278 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.9.2 ### What happened? If a mapped tas

Re: [I] The remote task tracked by the trigger failed due to the reassignment of the trigger task [airflow]

2024-08-06 Thread via GitHub
TakawaAkirayo commented on issue #41231: URL: https://github.com/apache/airflow/issues/41231#issuecomment-2270708754 @tirkarthi class RemoteJob: RUNNING_STATUS = "running" FAILED_STATUS = "failed" SUCCESS_STATUS = "success" def __init__(self, status):

[PR] Fix dataset page cannot correctly load triggered dag runs due to lack of dagId [airflow]

2024-08-06 Thread via GitHub
Lee-W opened a new pull request, #41279: URL: https://github.com/apache/airflow/pull/41279 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.

Re: [I] The remote task tracked by the trigger failed due to the reassignment of the trigger task [airflow]

2024-08-06 Thread via GitHub
tirkarthi commented on issue #41231: URL: https://github.com/apache/airflow/issues/41231#issuecomment-2270799382 cleanup method is called when trigger is cancelled due to triggerer restart, reassignment of trigger etc. You need to implement something like safe_to_cancel method in the linked

Re: [PR] Fix tests/operators/test_bash.py and branch_operator.py for Database Isolation Mode [airflow]

2024-08-06 Thread via GitHub
uranusjr commented on code in PR #41277: URL: https://github.com/apache/airflow/pull/41277#discussion_r1705212652 ## tests/operators/test_branch_operator.py: ## @@ -127,113 +120,152 @@ def test_branch_list_without_dag_run(self): else: raise

Re: [PR] Add DatasetDagRunQueue to all the consuming DAGs of a dataset alias [airflow]

2024-08-06 Thread via GitHub
uranusjr commented on code in PR #41264: URL: https://github.com/apache/airflow/pull/41264#discussion_r1705267718 ## airflow/datasets/__init__.py: ## @@ -336,6 +342,14 @@ def iter_datasets(self) -> Iterator[tuple[str, Dataset]]: yield k, v seen.

Re: [PR] Add `CloudRunServiceHook` and `CloudRunCreateServiceOperator` [airflow]

2024-08-06 Thread via GitHub
jx2lee commented on code in PR #40008: URL: https://github.com/apache/airflow/pull/40008#discussion_r1705307429 ## tests/system/providers/google/cloud/cloud_run/example_cloud_run_service.py: ## @@ -0,0 +1,106 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

Re: [PR] Add DatasetDagRunQueue to all the consuming DAGs of a dataset alias [airflow]

2024-08-06 Thread via GitHub
Lee-W commented on code in PR #41264: URL: https://github.com/apache/airflow/pull/41264#discussion_r1705308958 ## airflow/datasets/__init__.py: ## @@ -336,6 +342,14 @@ def iter_datasets(self) -> Iterator[tuple[str, Dataset]]: yield k, v seen.add

Re: [PR] kubernetes executor cleanup_stuck_queued_tasks optimization [airflow]

2024-08-06 Thread via GitHub
dirrao commented on PR #41220: URL: https://github.com/apache/airflow/pull/41220#issuecomment-2270991658 @jedcunningham / @hussein-awala Can you review it whenever you are free? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] K8s Executor: failing the task in case the watcher receives an event with the reason ProviderFailed [airflow]

2024-08-06 Thread via GitHub
dirrao commented on PR #41186: URL: https://github.com/apache/airflow/pull/41186#issuecomment-2270992198 @jedcunningham / @hussein-awala Can you review it whenever you are free? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Add DatasetDagRunQueue to all the consuming DAGs of a dataset alias [airflow]

2024-08-06 Thread via GitHub
uranusjr commented on code in PR #41264: URL: https://github.com/apache/airflow/pull/41264#discussion_r1705338968 ## airflow/datasets/__init__.py: ## @@ -294,6 +297,9 @@ def as_expression(self) -> Any: def iter_datasets(self) -> Iterator[tuple[str, Dataset]]: yield

Re: [PR] Add DatasetDagRunQueue to all the consuming DAGs of a dataset alias [airflow]

2024-08-06 Thread via GitHub
uranusjr commented on code in PR #41264: URL: https://github.com/apache/airflow/pull/41264#discussion_r1705338968 ## airflow/datasets/__init__.py: ## @@ -294,6 +297,9 @@ def as_expression(self) -> Any: def iter_datasets(self) -> Iterator[tuple[str, Dataset]]: yield

Re: [PR] Add DatasetDagRunQueue to all the consuming DAGs of a dataset alias [airflow]

2024-08-06 Thread via GitHub
uranusjr commented on code in PR #41264: URL: https://github.com/apache/airflow/pull/41264#discussion_r1705340564 ## airflow/datasets/__init__.py: ## @@ -294,6 +297,9 @@ def as_expression(self) -> Any: def iter_datasets(self) -> Iterator[tuple[str, Dataset]]: yield

Re: [PR] Upgrade package gcloud-aio-auth>=5.2.0 [airflow]

2024-08-06 Thread via GitHub
moiseenkov commented on PR #41262: URL: https://github.com/apache/airflow/pull/41262#issuecomment-2271049094 Hi @potiuk , @ashb , I made some changes in the scripts/docker/entrypoint_ci.sh to fix the CI. PTAL. -- This is an automated message from the Apache Git Service. To respond to t

(airflow) branch v2-10-stable updated (002a0753a2 -> d61b5bb8b9)

2024-08-06 Thread utkarsharma
This is an automated email from the ASF dual-hosted git repository. utkarsharma pushed a change to branch v2-10-stable in repository https://gitbox.apache.org/repos/asf/airflow.git omit 002a0753a2 Regenerate command images omit 95cdc05be6 Update RELEASE_NOTES.rst omit 56b7cf2522 Upda

(airflow) 01/01: Update default branches for 2-10

2024-08-06 Thread utkarsharma
This is an automated email from the ASF dual-hosted git repository. utkarsharma pushed a commit to branch v2-10-stable in repository https://gitbox.apache.org/repos/asf/airflow.git commit d61b5bb8b949e5af20aa2ff4fcfcd09c058b92b3 Author: utkarsh sharma AuthorDate: Tue Aug 6 15:29:00 2024 +0530

(airflow) 02/03: Update RELEASE_NOTES.rst

2024-08-06 Thread utkarsharma
This is an automated email from the ASF dual-hosted git repository. utkarsharma pushed a commit to branch v2-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 6db1ba8afd9bd35dba7ea555e5d8257f136845b2 Author: utkarsh sharma AuthorDate: Tue Aug 6 17:20:30 2024 +0530

(airflow) 01/03: Update version to 2.10.0

2024-08-06 Thread utkarsharma
This is an automated email from the ASF dual-hosted git repository. utkarsharma pushed a commit to branch v2-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 05ba78f8cf2ab3dc826b5aed3b1c632cbaf50345 Author: utkarsh sharma AuthorDate: Tue Aug 6 16:52:38 2024 +0530

(airflow) branch v2-10-test updated (002a0753a2 -> e2e9cfabd2)

2024-08-06 Thread utkarsharma
This is an automated email from the ASF dual-hosted git repository. utkarsharma pushed a change to branch v2-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git omit 002a0753a2 Regenerate command images omit 95cdc05be6 Update RELEASE_NOTES.rst omit 56b7cf2522 Update

[PR] Sync v2-10-stable with v2-10-test to release 2.10.0rc1 [airflow]

2024-08-06 Thread via GitHub
utkarsharma2 opened a new pull request, #41280: URL: https://github.com/apache/airflow/pull/41280 Time for 2.10.0rc1! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] Provider Airbyte: breaking change, update provider to use Airbyte API Python SDK [airflow]

2024-08-06 Thread via GitHub
Imbruced commented on code in PR #41122: URL: https://github.com/apache/airflow/pull/41122#discussion_r1705417934 ## airflow/providers/airbyte/hooks/airbyte.py: ## @@ -155,105 +130,59 @@ def wait_for_job(self, job_id: str | int, wait_seconds: float = 3, timeout: floa

Re: [PR] Provider Airbyte: breaking change, update provider to use Airbyte API Python SDK [airflow]

2024-08-06 Thread via GitHub
Imbruced commented on code in PR #41122: URL: https://github.com/apache/airflow/pull/41122#discussion_r1705418519 ## airflow/providers/airbyte/hooks/airbyte.py: ## @@ -155,105 +130,59 @@ def wait_for_job(self, job_id: str | int, wait_seconds: float = 3, timeout: floa

(airflow-site) branch main updated: Move Jens to PMC members (#1051)

2024-08-06 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new 4f08e0363b Move Jens to PMC members (#1051) 4f

Re: [PR] Move Jens to PMC members [airflow-site]

2024-08-06 Thread via GitHub
potiuk merged PR #1051: URL: https://github.com/apache/airflow-site/pull/1051 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Fix tests/operators/test_bash.py and branch_operator.py for Database Isolation Mode [airflow]

2024-08-06 Thread via GitHub
potiuk commented on code in PR #41277: URL: https://github.com/apache/airflow/pull/41277#discussion_r1705427423 ## tests/operators/test_branch_operator.py: ## @@ -127,113 +120,152 @@ def test_branch_list_without_dag_run(self): else: raise Ex

Re: [PR] Provider Airbyte: breaking change, update provider to use Airbyte API Python SDK [airflow]

2024-08-06 Thread via GitHub
Imbruced commented on code in PR #41122: URL: https://github.com/apache/airflow/pull/41122#discussion_r1705427834 ## airflow/providers/airbyte/operators/airbyte.py: ## @@ -73,47 +74,39 @@ def __init__( self.connection_id = connection_id self.timeout = timeout

Re: [I] Executor reports task instance finished (failed) although the task says it's queued. (Info: None) Was the task killed externally [airflow]

2024-08-06 Thread via GitHub
scaoupgrade commented on issue #41276: URL: https://github.com/apache/airflow/issues/41276#issuecomment-2271143465 That 10 minutes gap could be because of this config: https://github.com/apache/airflow/blob/6229c1c98a2a78d00255d6b6c5b70d032cff9b80/airflow/config_templates/config.yml#L2600

Re: [PR] Upgrade package gcloud-aio-auth>=5.2.0 [airflow]

2024-08-06 Thread via GitHub
potiuk merged PR #41262: URL: https://github.com/apache/airflow/pull/41262 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (6229c1c98a -> 16abb911e8)

2024-08-06 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 6229c1c98a Fix DateTimeOperator tests in Database Isolation Mode (#41271) add 16abb911e8 Upgrade package gcloud-a

(airflow-site) branch gh-pages updated (cb6e9770e1 -> 0acb03850b)

2024-08-06 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard cb6e9770e1 Rewritten history to remove past gh-pages deployments new 0acb03850b Rewritten history to

(airflow) branch v2-10-test updated (e2e9cfabd2 -> 9048b75903)

2024-08-06 Thread utkarsharma
This is an automated email from the ASF dual-hosted git repository. utkarsharma pushed a change to branch v2-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git discard e2e9cfabd2 Regenerate command images discard 6db1ba8afd Update RELEASE_NOTES.rst add 542620ba9e Update

[PR] Make remaining provider tests pass in DB isolation mode [airflow]

2024-08-06 Thread via GitHub
potiuk opened a new pull request, #41282: URL: https://github.com/apache/airflow/pull/41282 All provider's tests should pass now in DB isolation mode * some tests are fixed * some modules are excluded as it makes no sense * some modules are excluded and note about adding informat

Re: [I] New dag run / task state: disposed [airflow]

2024-08-06 Thread via GitHub
billcrook commented on issue #12199: URL: https://github.com/apache/airflow/issues/12199#issuecomment-2271238725 Standardized text in a free-form text field is a classic anti-pattern. When end users start doing this it is an indication there is a missing feature. -- This is an automated m

[I] Airflow REST GET API call for TaskInstances returns unordered items, making paged retrieval impossible [airflow]

2024-08-06 Thread via GitHub
DFINITYManu opened a new issue, #41283: URL: https://github.com/apache/airflow/issues/41283 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.9.1 ### What happened? When using pa

Re: [I] Airflow REST GET API call for TaskInstances returns unordered items, making paged retrieval impossible [airflow]

2024-08-06 Thread via GitHub
boring-cyborg[bot] commented on issue #41283: URL: https://github.com/apache/airflow/issues/41283#issuecomment-2271245456 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Sync v2-10-stable with v2-10-test to release 2.10.0b2 [airflow]

2024-08-06 Thread via GitHub
utkarsharma2 commented on PR #41184: URL: https://github.com/apache/airflow/pull/41184#issuecomment-2271248014 Hey @jedcunningham, thanks for the review. I have updated the new sync [PR](https://github.com/apache/airflow/pull/41280) according to your suggestions. PTAL. -- This is an auto

Re: [PR] Sync v2-10-stable with v2-10-test to release 2.10.0rc1 [airflow]

2024-08-06 Thread via GitHub
utkarsharma2 commented on code in PR #41280: URL: https://github.com/apache/airflow/pull/41280#discussion_r1705508594 ## RELEASE_NOTES.rst: ## @@ -21,6 +21,333 @@ .. towncrier release notes start +Airflow 2.10.0 (2024-08-06) +--- + +Features +^^^

Re: [PR] Sync v2-10-stable with v2-10-test to release 2.10.0rc1 [airflow]

2024-08-06 Thread via GitHub
utkarsharma2 commented on code in PR #41280: URL: https://github.com/apache/airflow/pull/41280#discussion_r1705510883 ## RELEASE_NOTES.rst: ## @@ -21,6 +21,333 @@ .. towncrier release notes start +Airflow 2.10.0 (2024-08-06) +--- + +Features +^^^

Re: [PR] Add DatasetDagRunQueue to all the consuming DAGs of a dataset alias [airflow]

2024-08-06 Thread via GitHub
Lee-W commented on code in PR #41264: URL: https://github.com/apache/airflow/pull/41264#discussion_r1705522857 ## airflow/datasets/__init__.py: ## @@ -294,6 +297,9 @@ def as_expression(self) -> Any: def iter_datasets(self) -> Iterator[tuple[str, Dataset]]: yield se

Re: [PR] Sync v2-10-stable with v2-10-test to release 2.10.0rc1 [airflow]

2024-08-06 Thread via GitHub
utkarsharma2 commented on code in PR #41280: URL: https://github.com/apache/airflow/pull/41280#discussion_r1705508594 ## RELEASE_NOTES.rst: ## @@ -21,6 +21,333 @@ .. towncrier release notes start +Airflow 2.10.0 (2024-08-06) +--- + +Features +^^^

[PR] chore: remove OpenLineage deprecation warnings [airflow]

2024-08-06 Thread via GitHub
kacpermuda opened a new pull request, #41284: URL: https://github.com/apache/airflow/pull/41284 We still intend to remove these facets and they are still deprecated, but the current warnings are being shown to users, which could be misleading since users are not involved with th

Re: [PR] Add `CloudRunServiceHook` and `CloudRunCreateServiceOperator` [airflow]

2024-08-06 Thread via GitHub
jx2lee commented on code in PR #40008: URL: https://github.com/apache/airflow/pull/40008#discussion_r1705579800 ## tests/system/providers/google/cloud/cloud_run/example_cloud_run_service.py: ## @@ -0,0 +1,101 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

Re: [PR] Add `CloudRunServiceHook` and `CloudRunCreateServiceOperator` [airflow]

2024-08-06 Thread via GitHub
jx2lee commented on code in PR #40008: URL: https://github.com/apache/airflow/pull/40008#discussion_r1705580781 ## tests/system/providers/google/cloud/cloud_run/example_cloud_run_service.py: ## @@ -0,0 +1,101 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

Re: [PR] Add `CloudRunServiceHook` and `CloudRunCreateServiceOperator` [airflow]

2024-08-06 Thread via GitHub
jx2lee commented on code in PR #40008: URL: https://github.com/apache/airflow/pull/40008#discussion_r1705580302 ## tests/system/providers/google/cloud/cloud_run/example_cloud_run_service.py: ## @@ -0,0 +1,101 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

Re: [PR] Change inserted airflow version of "update-migration-references" command from airflow_version='...' to airflow_version="..." [airflow]

2024-08-06 Thread via GitHub
Lee-W merged PR #41275: URL: https://github.com/apache/airflow/pull/41275 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.ap

(airflow) branch main updated (16abb911e8 -> 1544b08523)

2024-08-06 Thread weilee
This is an automated email from the ASF dual-hosted git repository. weilee pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 16abb911e8 Upgrade package gcloud-aio-auth>=5.2.0 (#41262) add 1544b08523 Change inserted airflow version of "upd

Re: [I] Support named mapping for task group [airflow]

2024-08-06 Thread via GitHub
rzilkha commented on issue #40799: URL: https://github.com/apache/airflow/issues/40799#issuecomment-2271352591 @uranusjr we basically would like to have the same view as we have in dynamic tasks. where instead of indexes we would see labels which will generated as a function of the d

Re: [PR] kubernetes executor cleanup_stuck_queued_tasks optimization [airflow]

2024-08-06 Thread via GitHub
romsharon98 commented on code in PR #41220: URL: https://github.com/apache/airflow/pull/41220#discussion_r1705611703 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py: ## @@ -601,34 +601,31 @@ def cleanup_stuck_queued_tasks(self, tis: list[TaskInstance]) ->

Re: [PR] Add `CloudRunServiceHook` and `CloudRunCreateServiceOperator` [airflow]

2024-08-06 Thread via GitHub
jx2lee commented on code in PR #40008: URL: https://github.com/apache/airflow/pull/40008#discussion_r1705620209 ## airflow/providers/google/cloud/operators/cloud_run.py: ## @@ -353,3 +354,129 @@ def _wait_for_operation(self, operation: operation.Operation): except Exce

(airflow) branch constraints-main updated: Updating constraints. Github run id:10267532310

2024-08-06 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new c9d2cdefb2 Updating con

Re: [PR] Add `CloudRunServiceHook` and `CloudRunCreateServiceOperator` [airflow]

2024-08-06 Thread via GitHub
jx2lee commented on code in PR #40008: URL: https://github.com/apache/airflow/pull/40008#discussion_r1705623664 ## tests/providers/google/cloud/operators/test_cloud_run.py: ## @@ -387,3 +394,67 @@ def test_execute_with_invalid_limit(self, hook_mock): limit = -1

Re: [PR] Make remaining provider tests pass in DB isolation mode [airflow]

2024-08-06 Thread via GitHub
vincbeck commented on code in PR #41282: URL: https://github.com/apache/airflow/pull/41282#discussion_r1705634620 ## airflow/api_internal/endpoints/rpc_api_endpoint.py: ## @@ -121,7 +121,7 @@ def initialize_method_map() -> dict[str, Callable]: MetastoreBackend._fetch_va

Re: [PR] Add `CloudRunServiceHook` and `CloudRunCreateServiceOperator` [airflow]

2024-08-06 Thread via GitHub
jx2lee commented on code in PR #40008: URL: https://github.com/apache/airflow/pull/40008#discussion_r1705634809 ## tests/system/providers/google/cloud/cloud_run/example_cloud_run_service.py: ## @@ -0,0 +1,101 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

Re: [PR] Fix RedshiftDataOperator not running in deferred mode when it should [airflow]

2024-08-06 Thread via GitHub
vincbeck commented on PR #41206: URL: https://github.com/apache/airflow/pull/41206#issuecomment-2271430140 > I realize now that this fix will change the operator's behavior: for folks who have been using it with `deferrable = True` and `wait_for_completion = False` to make it go into deferr

(airflow) branch main updated (1544b08523 -> 968b8f2cce)

2024-08-06 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 1544b08523 Change inserted airflow version of "update-migration-references" command from airflow_version='...' to a

Re: [PR] Fix dataset page cannot correctly load triggered dag runs due to lack of dagId [airflow]

2024-08-06 Thread via GitHub
bbovenzi merged PR #41279: URL: https://github.com/apache/airflow/pull/41279 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] K8s Executor: failing the task in case the watcher receives an event with the reason ProviderFailed [airflow]

2024-08-06 Thread via GitHub
jedcunningham commented on code in PR #41186: URL: https://github.com/apache/airflow/pull/41186#discussion_r1705642762 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor_utils.py: ## @@ -220,6 +220,11 @@ def process_status( # However, need to free th

Re: [I] Status of testing Providers that were prepared on August 03, 2024 [airflow]

2024-08-06 Thread via GitHub
uzhastik commented on issue #41237: URL: https://github.com/apache/airflow/issues/41237#issuecomment-2271471544 ydb provider works as expected, checked https://github.com/apache/airflow/pull/40631 thank you! -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]

2024-08-06 Thread via GitHub
joaopamaral commented on code in PR #40703: URL: https://github.com/apache/airflow/pull/40703#discussion_r1705704846 ## tests/dag_processing/test_processor.py: ## @@ -1035,7 +1035,7 @@ def test_counter_for_last_num_of_db_queries(self): with create_session() as session

Re: [I] Unable to fetch logs from amazon s3 [airflow]

2024-08-06 Thread via GitHub
vincbeck commented on issue #41171: URL: https://github.com/apache/airflow/issues/41171#issuecomment-2271532214 I tested it using the same configuration with breeze (against main and against Airflow 2.9.2) and it works properly. @KanaSukita, @burito111, did it stop working when you upgraded

Re: [I] Executor reports task instance finished (failed) although the task says it's queued. (Info: None) Was the task killed externally [airflow]

2024-08-06 Thread via GitHub
trlopes1974 commented on issue #41276: URL: https://github.com/apache/airflow/issues/41276#issuecomment-2271569597 Maybe, it makes some sense as we do not have that setting in the configuration. But, what is bothering me is WHY? why did it timeout after being queued? We have no exhaustio

[PR] Added Datapipes to the list of companies using Apache Airflow [airflow]

2024-08-06 Thread via GitHub
rishabh-cldcvr opened a new pull request, #41285: URL: https://github.com/apache/airflow/pull/41285 Added Datapipes to INTHEWILD.md -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Added Datapipes to the list of companies using Apache Airflow [airflow]

2024-08-06 Thread via GitHub
boring-cyborg[bot] commented on PR #41285: URL: https://github.com/apache/airflow/pull/41285#issuecomment-2271586757 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

(airflow) branch constraints-2-10 updated: Updating constraints. Github run id:10267005253

2024-08-06 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-10 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-10 by this push: new ccc6b82586 Updating con

Re: [PR] Skip highlighting lines that contain log group markers. [airflow]

2024-08-06 Thread via GitHub
tirkarthi commented on code in PR #41222: URL: https://github.com/apache/airflow/pull/41222#discussion_r1705773792 ## airflow/www/static/js/utils/index.ts: ## @@ -190,6 +192,15 @@ const highlightByKeywords = ( errorKeywords: string[], warningKeywords: string[] ): string =

Re: [PR] Skip highlighting lines that contain log group markers. [airflow]

2024-08-06 Thread via GitHub
tirkarthi commented on code in PR #41222: URL: https://github.com/apache/airflow/pull/41222#discussion_r1705773013 ## airflow/www/static/js/utils/index.ts: ## @@ -25,6 +25,8 @@ import useOffsetTop from "./useOffsetTop"; // Delay in ms for various hover actions const hoverDelay

[I] taskflow does not understand * (argument unpack operator): ValueError: The key 'how' in args is a part of kwargs and therefore reserved. [airflow]

2024-08-06 Thread via GitHub
jabbera opened a new issue, #41286: URL: https://github.com/apache/airflow/issues/41286 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.9.2 ### What happened? Executing the att

Re: [PR] Fix RedshiftDataOperator not running in deferred mode when it should [airflow]

2024-08-06 Thread via GitHub
eladkal commented on PR #41206: URL: https://github.com/apache/airflow/pull/41206#issuecomment-2271625537 Can you add to the provider changelog a block of ``` Main .. .. warning:: ``` to the changelog where you explain how to mitigate the change. It needs

Re: [I] Add legend to grid view [airflow]

2024-08-06 Thread via GitHub
bbovenzi commented on issue #28737: URL: https://github.com/apache/airflow/issues/28737#issuecomment-2271642733 looking good! we may need to adjust the statuswith notes margins but otherwise its great! -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Scheduled DAGs with cron expression [airflow]

2024-08-06 Thread via GitHub
tirkarthi commented on issue #41270: URL: https://github.com/apache/airflow/issues/41270#issuecomment-2271649681 Can you please add a sample dag to reproduce this locally? By don't work do you mean the dag is triggered on incorrect dates or not triggered at all? I tried a dag with sch

Re: [PR] chore: remove OpenLineage deprecation warnings [airflow]

2024-08-06 Thread via GitHub
eladkal commented on code in PR #41284: URL: https://github.com/apache/airflow/pull/41284#discussion_r1705836969 ## airflow/providers/openlineage/utils/utils.py: ## @@ -647,20 +653,6 @@ def get_filtered_unknown_operator_keys(operator: BaseOperator) -> dict: return {attr: v

[PR] Fix sagemaker experiments cleanup [airflow]

2024-08-06 Thread via GitHub
o-nikolas opened a new pull request, #41287: URL: https://github.com/apache/airflow/pull/41287 We had some experiment cleanup logic, for cleaning up the experiment created by the create experiment task, but more experiments are created as side-effects of other tasks (such as creating pipeli

(airflow) branch main updated (968b8f2cce -> d855294a80)

2024-08-06 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 968b8f2cce fix(TriggeredDagRuns.tsx): dataset page cannot correctly load triggered dag runs due to lack of dagId (#412

Re: [PR] Make remaining provider tests pass in DB isolation mode [airflow]

2024-08-06 Thread via GitHub
potiuk merged PR #41282: URL: https://github.com/apache/airflow/pull/41282 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Make remaining provider tests pass in DB isolation mode [airflow]

2024-08-06 Thread via GitHub
potiuk commented on code in PR #41282: URL: https://github.com/apache/airflow/pull/41282#discussion_r1705851720 ## airflow/api_internal/endpoints/rpc_api_endpoint.py: ## @@ -121,7 +121,7 @@ def initialize_method_map() -> dict[str, Callable]: MetastoreBackend._fetch_vari

(airflow) branch main updated: Swallow exception when mini-scheduling raises an exception (#41260)

2024-08-06 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 377e8cdcc5 Swallow exception when mini-scheduling r

Re: [PR] Swallow exception when mini-scheduling raises an exception [airflow]

2024-08-06 Thread via GitHub
potiuk merged PR #41260: URL: https://github.com/apache/airflow/pull/41260 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-08-06 Thread via GitHub
potiuk closed issue #39717: Executor reports task instance (...) finished (failed) although the task says it's queued URL: https://github.com/apache/airflow/issues/39717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-08-06 Thread via GitHub
potiuk closed issue #39717: Executor reports task instance (...) finished (failed) although the task says it's queued URL: https://github.com/apache/airflow/issues/39717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-08-06 Thread via GitHub
potiuk commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2271745848 @trlopes1974 -> I see you opened a new issue (cool) - for testing just apply the patch from #41260 to your installation - it might be building your own image with the change applie

(airflow) branch main updated: Don't reserialize DAG if no revision's applied (#41207)

2024-08-06 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7fa80f9b64 Don't reserialize DAG if no revision's a

Re: [PR] Don't reserialize DAG if no revision's applied [airflow]

2024-08-06 Thread via GitHub
potiuk merged PR #41207: URL: https://github.com/apache/airflow/pull/41207 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] duplicate key value violates unique constraint serialized_dag_pkey for dynamic dag generation [airflow]

2024-08-06 Thread via GitHub
potiuk closed issue #40082: duplicate key value violates unique constraint serialized_dag_pkey for dynamic dag generation URL: https://github.com/apache/airflow/issues/40082 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Fix tests/operators/test_bash.py and branch_operator.py for Database Isolation Mode [airflow]

2024-08-06 Thread via GitHub
jscheffl commented on code in PR #41277: URL: https://github.com/apache/airflow/pull/41277#discussion_r1705868958 ## tests/operators/test_branch_operator.py: ## @@ -127,113 +120,152 @@ def test_branch_list_without_dag_run(self): else: raise

Re: [PR] Chart: add a specific internal IP address for the ClusterIP service [airflow]

2024-08-06 Thread via GitHub
jedcunningham commented on code in PR #40912: URL: https://github.com/apache/airflow/pull/40912#discussion_r1705874694 ## chart/values.yaml: ## @@ -2144,6 +2144,7 @@ pgbouncer: service: extraAnnotations: {} +ip: ~ Review Comment: What do you think about `cluste

  1   2   >