[PR] Dedup optional build args [airflow]

2024-02-27 Thread via GitHub
drajguru opened a new pull request, #37765: URL: https://github.com/apache/airflow/pull/37765 Remove the construction of some duplicated arguments from the ci image build script. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

(airflow) branch main updated: Better fix ownership behaviour for Breeze (#37760)

2024-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new dc4df38e7f Better fix ownership behaviour for

Re: [PR] Better fix ownership behaviour for Breeze [airflow]

2024-02-27 Thread via GitHub
potiuk merged PR #37760: URL: https://github.com/apache/airflow/pull/37760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Hashicorp Vault: VAULT_CAPATH & VAULT_CACERT broken by pre-creating session to pass along [airflow]

2024-02-27 Thread via GitHub
tungbq commented on issue #37611: URL: https://github.com/apache/airflow/issues/37611#issuecomment-1968016358 > @tungbq maybe you can look into this issue ? Sure, I will take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Migrate XCom model queries to sqlalchemy v2 [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on code in PR #37761: URL: https://github.com/apache/airflow/pull/37761#discussion_r1505178664 ## airflow/models/xcom.py: ## @@ -509,40 +513,40 @@ def get_many( message = "Passing 'execution_date' to 'XCom.get_many()' is deprecated. Use

Re: [PR] Update GCS hook to get crc32c hash for CMEK-protected objects [airflow]

2024-02-27 Thread via GitHub
github-actions[bot] commented on PR #34982: URL: https://github.com/apache/airflow/pull/34982#issuecomment-1967947102 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

[PR] D105 Check on Amazon [airflow]

2024-02-27 Thread via GitHub
Satoshi-Sh opened a new pull request, #37764: URL: https://github.com/apache/airflow/pull/37764 ## Related Issue #37523 ## Checks - [x] airflow/providers/amazon/aws/exceptions.py - [x] airflow/providers/amazon/aws/executors/ecs/utils.py - [x]

(airflow) branch main updated: Possibility to set labels specific to the migrateDatabaseJob objects and pods (#37490)

2024-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 051720401d Possibility to set labels specific to

Re: [PR] Possibility to set labels specific to the migrateDatabaseJob objects and pods [airflow]

2024-02-27 Thread via GitHub
boring-cyborg[bot] commented on PR #37490: URL: https://github.com/apache/airflow/pull/37490#issuecomment-1967931789 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional

Re: [PR] Possibility to set labels specific to the migrateDatabaseJob objects and pods [airflow]

2024-02-27 Thread via GitHub
potiuk merged PR #37490: URL: https://github.com/apache/airflow/pull/37490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Fix invalid deprecation of `DataFusionPipelineLinkHelper` [airflow]

2024-02-27 Thread via GitHub
potiuk merged PR #37755: URL: https://github.com/apache/airflow/pull/37755 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Fix invalid deprecation of `DataFusionPipelineLinkHelper` (#37755)

2024-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f001576a94 Fix invalid deprecation of

Re: [I] Policies do not run at the startup of each worker [airflow]

2024-02-27 Thread via GitHub
potiuk closed issue #37621: Policies do not run at the startup of each worker URL: https://github.com/apache/airflow/issues/37621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Policies do not run at the startup of each worker [airflow]

2024-02-27 Thread via GitHub
potiuk commented on issue #37621: URL: https://github.com/apache/airflow/issues/37621#issuecomment-1967929973 Policies need to be run everywhere when task is parsed - because they potentially modify task instance. Ths is intended and the way how it could vary per execution is nicely

Re: [PR] Readme Update [airflow]

2024-02-27 Thread via GitHub
potiuk commented on code in PR #37727: URL: https://github.com/apache/airflow/pull/37727#discussion_r1505155715 ## README.md: ## @@ -180,15 +180,13 @@ them to the appropriate format and workflow that your tool requires. ```bash -pip install 'apache-airflow==2.8.2' \ -

Re: [PR] Migrate XCom model queries to sqlalchemy v2 [airflow]

2024-02-27 Thread via GitHub
hussein-awala commented on PR #37761: URL: https://github.com/apache/airflow/pull/37761#issuecomment-1967921798 I will try to update it to make the tests pass without updating them (to ensure it's b/c) -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] Relax "Setup tasks must be followed with trigger rule ALL_SUCCESS" constraint [airflow]

2024-02-27 Thread via GitHub
potiuk closed issue #37730: Relax "Setup tasks must be followed with trigger rule ALL_SUCCESS" constraint URL: https://github.com/apache/airflow/issues/37730 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Immediately fail the task in case of worker pod having a fatal container state [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37670: URL: https://github.com/apache/airflow/pull/37670#issuecomment-1967915713 But also would love to get someone else to take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] Migrate XCom model queries to sqlalchemy v2 [airflow]

2024-02-27 Thread via GitHub
hussein-awala opened a new pull request, #37761: URL: https://github.com/apache/airflow/pull/37761 related: #28723 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Immediately fail the task in case of worker pod having a fatal container state [airflow]

2024-02-27 Thread via GitHub
potiuk commented on code in PR #37670: URL: https://github.com/apache/airflow/pull/37670#discussion_r1505146539 ## airflow/providers/cncf/kubernetes/provider.yaml: ## @@ -224,6 +224,16 @@ config: type: string example: ~ default: "False" +

(airflow) branch main updated: Exclude some folders from licence check (#37759)

2024-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 019cdfc730 Exclude some folders from licence check

Re: [PR] Exclude some folders from licence check [airflow]

2024-02-27 Thread via GitHub
potiuk merged PR #37759: URL: https://github.com/apache/airflow/pull/37759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] Better fix ownership behaviour for Breeze [airflow]

2024-02-27 Thread via GitHub
potiuk opened a new pull request, #37760: URL: https://github.com/apache/airflow/pull/37760 Fix ownership with docker is fixing the whole airflow directory now and is less verbose when nothing happens. --- **^ Add meaningful description above** Read

Re: [I] Support for Flask-sesssion 0.6.0+ [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on issue #36897: URL: https://github.com/apache/airflow/issues/36897#issuecomment-1967904433 I tried to figure out how to adopt the changes, `rc0` or released version, but have no a clue right now. -- This is an automated message from the Apache Git Service. To

[PR] Exclude some folders from licence check [airflow]

2024-02-27 Thread via GitHub
potiuk opened a new pull request, #37759: URL: https://github.com/apache/airflow/pull/37759 Those folders should not need to have files with licence headers. --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on code in PR #37757: URL: https://github.com/apache/airflow/pull/37757#discussion_r1505128250 ## .pre-commit-config.yaml: ## @@ -855,7 +855,7 @@ repos: files: ^dev/breeze/.*$ pass_filenames: false require_serial: true -

(airflow) branch main updated (bca4ac3493 -> f418589e77)

2024-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from bca4ac3493 Fix execution data validation error in `/get_logs_with_metadata` endpoint (#37756) add f418589e77

Re: [PR] Update to latest installer versions. [airflow]

2024-02-27 Thread via GitHub
potiuk merged PR #37754: URL: https://github.com/apache/airflow/pull/37754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on code in PR #37757: URL: https://github.com/apache/airflow/pull/37757#discussion_r1505123675 ## .pre-commit-config.yaml: ## @@ -855,7 +855,7 @@ repos: files: ^dev/breeze/.*$ pass_filenames: false require_serial: true -

Re: [PR] Set job labels for traceability [airflow]

2024-02-27 Thread via GitHub
stiak commented on code in PR #37736: URL: https://github.com/apache/airflow/pull/37736#discussion_r1505112975 ## airflow/providers/google/cloud/operators/bigquery.py: ## @@ -2773,6 +2773,14 @@ def _submit_job( hook: BigQueryHook, job_id: str, ) ->

Re: [PR] Set job labels for traceability [airflow]

2024-02-27 Thread via GitHub
stiak commented on code in PR #37736: URL: https://github.com/apache/airflow/pull/37736#discussion_r1505111633 ## airflow/providers/google/cloud/operators/bigquery.py: ## @@ -2773,6 +2773,14 @@ def _submit_job( hook: BigQueryHook, job_id: str, ) ->

Re: [PR] AWS Batch Executor [airflow]

2024-02-27 Thread via GitHub
syedahsn commented on code in PR #37618: URL: https://github.com/apache/airflow/pull/37618#discussion_r1505100344 ## airflow/providers/amazon/aws/executors/batch/batch_executor.py: ## @@ -0,0 +1,386 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

(airflow) branch constraints-main updated: Updating constraints. Github run id:8071789952

2024-02-27 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 9e20d406b6 Updating

Re: [PR] Set job labels for traceability [airflow]

2024-02-27 Thread via GitHub
stiak commented on code in PR #37736: URL: https://github.com/apache/airflow/pull/37736#discussion_r1505079476 ## airflow/providers/google/cloud/operators/bigquery.py: ## @@ -2773,6 +2773,14 @@ def _submit_job( hook: BigQueryHook, job_id: str, ) ->

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on code in PR #37757: URL: https://github.com/apache/airflow/pull/37757#discussion_r1505078924 ## .pre-commit-config.yaml: ## @@ -855,7 +855,7 @@ repos: files: ^dev/breeze/.*$ pass_filenames: false require_serial: true -

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on code in PR #37757: URL: https://github.com/apache/airflow/pull/37757#discussion_r1505077750 ## .pre-commit-config.yaml: ## @@ -855,7 +855,7 @@ repos: files: ^dev/breeze/.*$ pass_filenames: false require_serial: true -

Re: [PR] Adding run_id column to log table [airflow]

2024-02-27 Thread via GitHub
SamWheating commented on PR #37731: URL: https://github.com/apache/airflow/pull/37731#issuecomment-1967712398 > Just a watch-out - we have two migrations in-progress Thanks for the heads up - I'll keep a close  on the other two and try to keep this PR in a mergeable state. --

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-02-27 Thread via GitHub
potiuk commented on code in PR #37757: URL: https://github.com/apache/airflow/pull/37757#discussion_r1505064063 ## .pre-commit-config.yaml: ## @@ -855,7 +855,7 @@ repos: files: ^dev/breeze/.*$ pass_filenames: false require_serial: true -

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-02-27 Thread via GitHub
potiuk commented on code in PR #37757: URL: https://github.com/apache/airflow/pull/37757#discussion_r1505062170 ## .pre-commit-config.yaml: ## @@ -855,7 +855,7 @@ repos: files: ^dev/breeze/.*$ pass_filenames: false require_serial: true -

Re: [PR] Limit `pandas` to `<2.2` [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on PR #37748: URL: https://github.com/apache/airflow/pull/37748#issuecomment-1967671404 @potiuk yeah we could have some integration tests for that, e.g. for MySQL and Postgres. Let me create a task for that, just for avoid situation that we discuss and finally forget to

(airflow) branch main updated (090a7f0707 -> bca4ac3493)

2024-02-27 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 090a7f0707 DateTimeTrigger typing and tests (#37694) add bca4ac3493 Fix execution data validation error in

Re: [PR] Fix execution data validation error in `/get_logs_with_metadata` endpoint [airflow]

2024-02-27 Thread via GitHub
Taragolis merged PR #37756: URL: https://github.com/apache/airflow/pull/37756 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] AWS Batch Executor [airflow]

2024-02-27 Thread via GitHub
o-nikolas commented on code in PR #37618: URL: https://github.com/apache/airflow/pull/37618#discussion_r1505023213 ## docs/apache-airflow-providers-amazon/executors/general.rst: ## @@ -0,0 +1,331 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more

Re: [PR] Attempt to fix deprecation warnings in SQLA 2.0 [airflow]

2024-02-27 Thread via GitHub
hussein-awala closed pull request #30036: Attempt to fix deprecation warnings in SQLA 2.0 URL: https://github.com/apache/airflow/pull/30036 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Attempt to fix deprecation warnings in SQLA 2.0 [airflow]

2024-02-27 Thread via GitHub
hussein-awala commented on PR #30036: URL: https://github.com/apache/airflow/pull/30036#issuecomment-1967660130 I am closing this PR because it does not fix the mentioned issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on code in PR #37757: URL: https://github.com/apache/airflow/pull/37757#discussion_r1505037795 ## .pre-commit-config.yaml: ## @@ -855,7 +855,7 @@ repos: files: ^dev/breeze/.*$ pass_filenames: false require_serial: true -

[PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-02-27 Thread via GitHub
Taragolis opened a new pull request, #37757: URL: https://github.com/apache/airflow/pull/37757 This is attempt to prevent use `functools.lru_cache` or `functools.cache` on methods which lead to memory leak, e.g. object can't be garbage collected if it use methods decorated

Re: [PR] feat: enable mysql keda support for triggerer [airflow]

2024-02-27 Thread via GitHub
jedcunningham commented on code in PR #37365: URL: https://github.com/apache/airflow/pull/37365#discussion_r1505029734 ## helm_tests/airflow_core/test_triggerer.py: ## Review Comment: Should we roll these checks into [this

Re: [PR] Adding run_id column to log table [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37731: URL: https://github.com/apache/airflow/pull/37731#issuecomment-1967615883 > We actually have 3 in flight : #37734          -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Update statsd-mappings.yml [airflow]

2024-02-27 Thread via GitHub
jedcunningham closed pull request #37480: Update statsd-mappings.yml URL: https://github.com/apache/airflow/pull/37480 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] ECS Executor - add support to adopt orphaned tasks. [airflow]

2024-02-27 Thread via GitHub
ferruzzi commented on PR #36803: URL: https://github.com/apache/airflow/pull/36803#issuecomment-1967596864 Unless anyone here feels strongly otherwise, I think I'm going to close this one and open a new PR. It's been a while and it needs a rebase, plus there are now a few commits which

Re: [PR] Adding run_id column to log table [airflow]

2024-02-27 Thread via GitHub
jedcunningham commented on PR #37731: URL: https://github.com/apache/airflow/pull/37731#issuecomment-1967559142 We actually have 3 in flight :sweat_smile:: #37734 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

(airflow) branch main updated: DateTimeTrigger typing and tests (#37694)

2024-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 090a7f0707 DateTimeTrigger typing and tests

Re: [PR] DateTimeTrigger typing and tests [airflow]

2024-02-27 Thread via GitHub
potiuk merged PR #37694: URL: https://github.com/apache/airflow/pull/37694 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] subclassed ObjectStoragePath of new upath has some quirks [airflow]

2024-02-27 Thread via GitHub
potiuk commented on issue #37568: URL: https://github.com/apache/airflow/issues/37568#issuecomment-1967544274 Do we have a plan to take on that one? What needs to be done here to get it in (as blocker removal for Python 3.12) ? -- This is an automated message from the Apache Git Service.

Re: [PR] AWS Batch Executor [airflow]

2024-02-27 Thread via GitHub
o-nikolas commented on code in PR #37618: URL: https://github.com/apache/airflow/pull/37618#discussion_r1504838251 ## airflow/providers/amazon/aws/executors/batch/batch_executor.py: ## @@ -0,0 +1,386 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[PR] Fix execution data validation error in `/get_logs_with_metadata` endpoint [airflow]

2024-02-27 Thread via GitHub
Taragolis opened a new pull request, #37756: URL: https://github.com/apache/airflow/pull/37756 `execution_date` referenced before assigned so in case of invalid date (rare scenario) it return 500 error instead of 400 error --- **^ Add meaningful description

Re: [PR] AWS Batch Executor [airflow]

2024-02-27 Thread via GitHub
syedahsn commented on PR #37618: URL: https://github.com/apache/airflow/pull/37618#issuecomment-1967477064 @Lee-W @eladkal I'd like to hear your thoughts on this feature. If you have some time to take a look, I'd really appreciate it! -- This is an automated message from the Apache Git

Re: [I] Cant get Kubernetes task's log when using CeleryKubernetesExecutor [airflow]

2024-02-27 Thread via GitHub
potiuk commented on issue #21548: URL: https://github.com/apache/airflow/issues/21548#issuecomment-1967474576 > Ran into an old problem again in version 2.8 that we thought was fixed back in 2.6.3. We're having trouble getting logs from Kubernetes tasks when using the

Re: [I] Support for Flask-sesssion 0.6.0+ [airflow]

2024-02-27 Thread via GitHub
potiuk commented on issue #36897: URL: https://github.com/apache/airflow/issues/36897#issuecomment-1967469169 Not sure if we will get it .. rc notes published 2 weeks ago, and the author does not realise that once you released 0.6.0 you cannot just delete it from PyPI (even if I explained

Re: [PR] Adding run_id column to log table [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37731: URL: https://github.com/apache/airflow/pull/37731#issuecomment-1967460355 Just a watch-out - we have two migrations in-progress. I think we have a protection that will make the secon one not mergable if the first is but there will be some re-work needed once

Re: [PR] Add migration for rendered_map_index column in task_instance table. [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37708: URL: https://github.com/apache/airflow/pull/37708#issuecomment-1967459390 Just a watch-out - we have two migrations in-progress. I think we have a protection that will make the secon one not mergable if the first is but there will be some re-work needed once

Re: [PR] Limit `pandas` to `<2.2` [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37748: URL: https://github.com/apache/airflow/pull/37748#issuecomment-1967446535 BTW, @Taragolis - I wonder if we could add a test to detect this issue? I think it might be good to add it for the future - this will also give us more confidence whe they will fix it

Re: [I] SQLAlchemy / Pandas incompatibility [airflow]

2024-02-27 Thread via GitHub
potiuk commented on issue #37742: URL: https://github.com/apache/airflow/issues/37742#issuecomment-1967445035 We updated constraints, and re-built the images. When users will be using reference image or install airflow 2.8.2 following our recommendation (with constraints

Re: [PR] Limit `pandas` to `<2.2` [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37748: URL: https://github.com/apache/airflow/pull/37748#issuecomment-1967442320 Images are rebuilt (verified it). Announced at dev@ and users@, also marked it for 2.8.3 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Migrate executor docs to respective providers [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37728: URL: https://github.com/apache/airflow/pull/37728#issuecomment-1967440970 Well... Let's see in main and then when we release the docs :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

(airflow) branch main updated (cd33c2a6a7 -> 9e4bdc9e45)

2024-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from cd33c2a6a7 Migrate executor docs to respective providers (#37728) add 9e4bdc9e45 Limit `pandas` to `<2.2`

Re: [I] SQLAlchemy / Pandas incompatibility [airflow]

2024-02-27 Thread via GitHub
potiuk closed issue #37742: SQLAlchemy / Pandas incompatibility URL: https://github.com/apache/airflow/issues/37742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Limit `pandas` to `<2.2` [airflow]

2024-02-27 Thread via GitHub
potiuk merged PR #37748: URL: https://github.com/apache/airflow/pull/37748 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Migrate executor docs to respective providers [airflow]

2024-02-27 Thread via GitHub
RNHTTR commented on PR #37728: URL: https://github.com/apache/airflow/pull/37728#issuecomment-1967410707 omg finally!! 拾 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Migrate executor docs to respective providers [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37728: URL: https://github.com/apache/airflow/pull/37728#issuecomment-1967405775 Wooo ohoo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

(airflow) branch main updated: Migrate executor docs to respective providers (#37728)

2024-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new cd33c2a6a7 Migrate executor docs to respective

Re: [PR] Migrate executor docs to respective providers [airflow]

2024-02-27 Thread via GitHub
potiuk merged PR #37728: URL: https://github.com/apache/airflow/pull/37728 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] Fix invalid deprecation of `DataFusionPipelineLinkHelper` [airflow]

2024-02-27 Thread via GitHub
Taragolis opened a new pull request, #37755: URL: https://github.com/apache/airflow/pull/37755 `get_project_id` just raise an error instead of warning, so this method is unreachable. This part broken since Google Provider v10.10.0 --- **^ Add meaningful

Re: [PR] Migrate to connexion v3 [airflow]

2024-02-27 Thread via GitHub
Satoshi-Sh commented on code in PR #37638: URL: https://github.com/apache/airflow/pull/37638#discussion_r1504752066 ## pyproject.toml: ## @@ -83,7 +83,7 @@ dependencies = [ # The usage was added in #30596, seemingly only to override and improve the default error message.

(airflow) branch update-installers-automatically created (now 9003505fdf)

2024-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch update-installers-automatically in repository https://gitbox.apache.org/repos/asf/airflow.git at 9003505fdf Update to latest installer versions. No new revisions were added by this

[PR] Update to latest installer versions. [airflow]

2024-02-27 Thread via GitHub
potiuk opened a new pull request, #37754: URL: https://github.com/apache/airflow/pull/37754 We have now two installers - pip and uv and we want to upgrade them manually and get notified about it, but also we want to do it as easy as possible. This change builds on our canary build

Re: [PR] Update to latest installer versions. [airflow]

2024-02-27 Thread via GitHub
potiuk closed pull request #37753: Update to latest installer versions. URL: https://github.com/apache/airflow/pull/37753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

(airflow) branch main updated: Improve eager-upgrade additional requirements and constraint generation (#37745)

2024-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5d1b2715f7 Improve eager-upgrade additional

[PR] Update to latest installer versions. [airflow]

2024-02-27 Thread via GitHub
potiuk opened a new pull request, #37753: URL: https://github.com/apache/airflow/pull/37753 We have now two installers - pip and uv and we want to upgrade them manually and get notified about it, but also we want to do it as easy as possible. This change builds on our canary build

Re: [PR] Improve eager-upgrade additional requirements and constraint generation [airflow]

2024-02-27 Thread via GitHub
potiuk merged PR #37745: URL: https://github.com/apache/airflow/pull/37745 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[I] [AIP-49] Airflow support for OTEL traces [airflow]

2024-02-27 Thread via GitHub
howardyoo opened a new issue, #37752: URL: https://github.com/apache/airflow/issues/37752 ### Description We have had a successful release of OTEL support for Airflow that covered `metrics` part. Now, the time is right to move into the next chapter, where we'd be adding the new

Re: [PR] Fix credentials error for S3ToGCSOperator trigger [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on code in PR #37518: URL: https://github.com/apache/airflow/pull/37518#discussion_r1504722068 ## airflow/providers/google/cloud/transfers/s3_to_gcs.py: ## @@ -269,6 +269,7 @@ def transfer_files_async(self, files: list[str], gcs_hook: GCSHook, s3_hook: S3H

Re: [I] Policies do not run at the startup of each worker [airflow]

2024-02-27 Thread via GitHub
SamWheating commented on issue #37621: URL: https://github.com/apache/airflow/issues/37621#issuecomment-1967305008 From my understanding, the worker process will sometimes (depending on dag serialization, etc) need to reprocess DAG files. This means that the task and dag policies can be

Re: [PR] Use sys.version_info to get the python version [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37750: URL: https://github.com/apache/airflow/pull/37750#issuecomment-1967270620 https://github.com/apache/airflow/blob/main/scripts/docker/common.sh#L102 -> this is where it should be modified. -- This is an automated message from the Apache Git Service. To

Re: [PR] Use sys.version_info to get the python version [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37750: URL: https://github.com/apache/airflow/pull/37750#issuecomment-1967268136 While the change is good, his is wrong place where to update it. The dockerfile scripts are inlined automatically by pre-commit from `scripts/docker/*.sh` - and you should modify them

Re: [I] Make Kubernetes Scheduler Logs Uniform with Celery Executor when reporting TaskInstance Finished [airflow]

2024-02-27 Thread via GitHub
jedcunningham closed issue #28814: Make Kubernetes Scheduler Logs Uniform with Celery Executor when reporting TaskInstance Finished URL: https://github.com/apache/airflow/issues/28814 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] Make Kubernetes Scheduler Logs Uniform with Celery Executor when reporting TaskInstance Finished [airflow]

2024-02-27 Thread via GitHub
jedcunningham commented on issue #28814: URL: https://github.com/apache/airflow/issues/28814#issuecomment-1967265956 This was done in #28871. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(airflow-site) branch gh-pages updated (c3225425f3 -> 258fc58410)

2024-02-27 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard c3225425f3 Rewritten history to remove past gh-pages deployments new 258fc58410 Rewritten history to

Re: [PR] Use sys.version_info to get the python version [airflow]

2024-02-27 Thread via GitHub
boring-cyborg[bot] commented on PR #37750: URL: https://github.com/apache/airflow/pull/37750#issuecomment-1967249862 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[PR] Use sys.version_info to get the python version [airflow]

2024-02-27 Thread via GitHub
joankaradimov opened a new pull request, #37750: URL: https://github.com/apache/airflow/pull/37750 I have a customized image that uses `continuumio/miniconda3:4.12.0` as its `PYTHON_BASE_IMAGE`. The `python --version` in the said Conda distribution outputs: ``` Python 3.9.18 |

Re: [I] SparkKubernetesOperator not retrieves logs from the driver pod and displays them in the Airflow UI. [airflow]

2024-02-27 Thread via GitHub
SamWheating commented on issue #37681: URL: https://github.com/apache/airflow/issues/37681#issuecomment-1967186556 Hi there, could you please confirm which version of `apache-airflow-providers-cncf-kubernetes` you are using? -- This is an automated message from the Apache Git Service. To

Re: [PR] Update set-up-database in correspondance with the currrent configurat… [airflow]

2024-02-27 Thread via GitHub
potiuk commented on PR #37749: URL: https://github.com/apache/airflow/pull/37749#issuecomment-1967150140 Closing - as this was misunderstanding/typo mistake. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Update set-up-database in correspondance with the currrent configurat… [airflow]

2024-02-27 Thread via GitHub
potiuk closed pull request #37749: Update set-up-database in correspondance with the currrent configurat… URL: https://github.com/apache/airflow/pull/37749 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Update set-up-database in correspondance with the currrent configurat… [airflow]

2024-02-27 Thread via GitHub
potiuk commented on code in PR #37749: URL: https://github.com/apache/airflow/pull/37749#discussion_r1504635413 ## docs/apache-airflow/howto/set-up-database.rst: ## @@ -198,12 +198,6 @@ in the Postgres documentation to learn more. Details in the `SQLAlchemy Changelog

Re: [PR] Update set-up-database in correspondance with the currrent configurat… [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on code in PR #37749: URL: https://github.com/apache/airflow/pull/37749#discussion_r1504622839 ## docs/apache-airflow/howto/set-up-database.rst: ## @@ -198,12 +198,6 @@ in the Postgres documentation to learn more. Details in the `SQLAlchemy Changelog

Re: [PR] Update set-up-database in correspondance with the currrent configurat… [airflow]

2024-02-27 Thread via GitHub
Taragolis commented on code in PR #37749: URL: https://github.com/apache/airflow/pull/37749#discussion_r150468 ## docs/apache-airflow/howto/set-up-database.rst: ## @@ -198,12 +198,6 @@ in the Postgres documentation to learn more. Details in the `SQLAlchemy Changelog

Re: [PR] Update providers metadata 2024-02-27 [airflow]

2024-02-27 Thread via GitHub
eladkal commented on PR #37737: URL: https://github.com/apache/airflow/pull/37737#issuecomment-1967094881 Closed in favor of https://github.com/apache/airflow/pull/37741 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Update providers metadata 2024-02-27 [airflow]

2024-02-27 Thread via GitHub
eladkal closed pull request #37737: Update providers metadata 2024-02-27 URL: https://github.com/apache/airflow/pull/37737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Update set-up-database in correspondance with the currrent configurat… [airflow]

2024-02-27 Thread via GitHub
boring-cyborg[bot] commented on PR #37749: URL: https://github.com/apache/airflow/pull/37749#issuecomment-1967078238 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

  1   2   >