Re: [I] Celery tasks stuck queued. Clearing task fails rescheduling [airflow]

2023-12-01 Thread via GitHub
hterik commented on issue #35843: URL: https://github.com/apache/airflow/issues/35843#issuecomment-1835658027 Anecdotally, I've personally seen it happen at least three times in 2 weeks. I don't have any other detailed analysis of the actual frequency. -- This is an automated message from

[I] Scheduler executes tasks for temporary DagRuns created by airflow tasks test [airflow]

2023-12-01 Thread via GitHub
smphhh opened a new issue, #35994: URL: https://github.com/apache/airflow/issues/35994 ### Apache Airflow version 2.7.3 ### What happened When a temporary DagRun is created by the `airflow tasks test` command the scheduler will start executing all tasks for that DagRun,

Re: [PR] Fix `enable_logging=True` not working in `DockerSwarmOperator` [airflow]

2023-12-01 Thread via GitHub
eladkal commented on PR #35677: URL: https://github.com/apache/airflow/pull/35677#issuecomment-1835752455 Hi @stavdav143 please note that we are waiting for tests (or if code can't be tested easily then explanation why) We rarely accept code changes without tests -- This is an automat

Re: [PR] Cleanup code for elasticsearch<8 [airflow]

2023-12-01 Thread via GitHub
Taragolis commented on code in PR #35707: URL: https://github.com/apache/airflow/pull/35707#discussion_r1411827713 ## tests/providers/elasticsearch/log/test_es_task_handler.py: ## @@ -671,14 +671,11 @@ def test_retrieve_config_keys(): """ with conf_vars( { -

Re: [PR] New breeze command to clean up previous provider artifacts [airflow]

2023-12-01 Thread via GitHub
potiuk commented on code in PR #35970: URL: https://github.com/apache/airflow/pull/35970#discussion_r1411830974 ## dev/breeze/src/airflow_breeze/commands/release_management_commands.py: ## @@ -1815,3 +1871,26 @@ def update_constraints( if confirm_modifications(const

Re: [PR] Cleanup code for elasticsearch<8 [airflow]

2023-12-01 Thread via GitHub
eladkal merged PR #35707: URL: https://github.com/apache/airflow/pull/35707 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [PR] Cleanup code for elasticsearch<8 [airflow]

2023-12-01 Thread via GitHub
eladkal commented on code in PR #35707: URL: https://github.com/apache/airflow/pull/35707#discussion_r1411831635 ## tests/providers/elasticsearch/log/test_es_task_handler.py: ## @@ -671,14 +671,11 @@ def test_retrieve_config_keys(): """ with conf_vars( { -

(airflow) branch main updated: Cleanup code for elasticsearch<8 (#35707)

2023-12-01 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 396b1ba34e Cleanup code for elasticsearch<8 (#3570

Re: [PR] Move fs scheme definition from Python to YAML [airflow]

2023-12-01 Thread via GitHub
potiuk commented on PR #35972: URL: https://github.com/apache/airflow/pull/35972#issuecomment-1835759611 > It’s totally pratical for the plugin manager to just support the old format since it is very trivial, even indefinitely, as long as we don’t put it in the schema. Since we have not rel

Re: [PR] New breeze command to clean up previous provider artifacts [airflow]

2023-12-01 Thread via GitHub
eladkal commented on code in PR #35970: URL: https://github.com/apache/airflow/pull/35970#discussion_r1411835929 ## dev/README_RELEASE_PROVIDER_PACKAGES.md: ## @@ -1029,10 +1029,10 @@ do done # Check which old packages will be removed (you need Python 3.8+ and dev/requireme

Re: [PR] Add explanation about cleaning logs with helm [airflow]

2023-12-01 Thread via GitHub
eladkal commented on code in PR #35783: URL: https://github.com/apache/airflow/pull/35783#discussion_r1411838536 ## airflow/config_templates/config.yml: ## @@ -658,6 +658,8 @@ logging: description: | Whether the local log files for GCS, S3, WASB and OSS remote lo

Re: [PR] New breeze command to clean up previous provider artifacts [airflow]

2023-12-01 Thread via GitHub
potiuk commented on code in PR #35970: URL: https://github.com/apache/airflow/pull/35970#discussion_r1411847486 ## dev/README_RELEASE_PROVIDER_PACKAGES.md: ## @@ -1029,10 +1029,10 @@ do done # Check which old packages will be removed (you need Python 3.8+ and dev/requiremen

Re: [PR] Fix `enable_logging=True` not working in `DockerSwarmOperator` [airflow]

2023-12-01 Thread via GitHub
stavdav143 commented on PR #35677: URL: https://github.com/apache/airflow/pull/35677#issuecomment-1835787791 Hi @eladkal, Thanks for reviewing. yes I understand, I ve been facing some difficulties while trying to provide a demo gif. I will look into if I can add tests, to me it is a

[PR] Refacto `get_output_location` method in AthenaHook [airflow]

2023-12-01 Thread via GitHub
hussein-awala opened a new pull request, #35996: URL: https://github.com/apache/airflow/pull/35996 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] Add a cache for weaviate client [airflow]

2023-12-01 Thread via GitHub
Lee-W commented on code in PR #35983: URL: https://github.com/apache/airflow/pull/35983#discussion_r1411865058 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,18 +93,14 @@ def get_conn(self) -> WeaviateClient: url=url, auth_client_secret=auth_client_secre

Re: [PR] Add a cache for weaviate client [airflow]

2023-12-01 Thread via GitHub
ephraimbuddy commented on code in PR #35983: URL: https://github.com/apache/airflow/pull/35983#discussion_r1411874620 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,18 +93,14 @@ def get_conn(self) -> WeaviateClient: url=url, auth_client_secret=auth_clien

Re: [PR] Add a cache for weaviate client [airflow]

2023-12-01 Thread via GitHub
Lee-W commented on code in PR #35983: URL: https://github.com/apache/airflow/pull/35983#discussion_r1411876566 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,18 +93,14 @@ def get_conn(self) -> WeaviateClient: url=url, auth_client_secret=auth_client_secre

Re: [PR] Add a cache for weaviate client [airflow]

2023-12-01 Thread via GitHub
Lee-W commented on code in PR #35983: URL: https://github.com/apache/airflow/pull/35983#discussion_r1411878369 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,18 +93,14 @@ def get_conn(self) -> WeaviateClient: url=url, auth_client_secret=auth_client_secre

[PR] Improve typing hints for `only_client_type` decorator [airflow]

2023-12-01 Thread via GitHub
hussein-awala opened a new pull request, #35997: URL: https://github.com/apache/airflow/pull/35997 This PR improves the typing hints for `only_client_type` decorator to return the original type after applying the decorator. Required for #35966 -- This is an automated message from t

Re: [PR] Add a cache for weaviate client [airflow]

2023-12-01 Thread via GitHub
ephraimbuddy commented on code in PR #35983: URL: https://github.com/apache/airflow/pull/35983#discussion_r1411881928 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,18 +93,14 @@ def get_conn(self) -> WeaviateClient: url=url, auth_client_secret=auth_clien

Re: [PR] Feature/trino provider timezone [airflow]

2023-12-01 Thread via GitHub
Lee-W commented on code in PR #35963: URL: https://github.com/apache/airflow/pull/35963#discussion_r1411880984 ## airflow/providers/trino/hooks/trino.py: ## @@ -156,6 +156,7 @@ def get_conn(self) -> Connection: verify=_boolify(extra.get("verify", True)),

Re: [PR] New breeze command to clean up previous provider artifacts [airflow]

2023-12-01 Thread via GitHub
Lee-W commented on code in PR #35970: URL: https://github.com/apache/airflow/pull/35970#discussion_r1411889779 ## dev/breeze/src/airflow_breeze/commands/release_management_commands.py: ## @@ -1195,6 +1200,57 @@ def add_back_references( start_generating_back_references(site_

(airflow) branch constraints-main updated: Updating constraints. Github run id:7058186349

2023-12-01 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 204abe2334 Updating con

Re: [PR] Adding 1 second sleep time between retries for publishing tasks [airflow]

2023-12-01 Thread via GitHub
potiuk commented on PR #35791: URL: https://github.com/apache/airflow/pull/35791#issuecomment-1835835446 I don't think hardcoding it to 1 second is a good idea. it sounds pretty magical. Also it has potentially undesireable side effects - there might be cases where the retry is actually goo

[I] dag_hash changed when using the XComArgs feature [airflow]

2023-12-01 Thread via GitHub
zhangw opened a new issue, #35998: URL: https://github.com/apache/airflow/issues/35998 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened ## Airflow version 2.1.2 Dag file is simple but using the XComArgs feature, and I noti

Re: [I] dag_hash changed when using the XComArgs feature [airflow]

2023-12-01 Thread via GitHub
boring-cyborg[bot] commented on issue #35998: URL: https://github.com/apache/airflow/issues/35998#issuecomment-1835858719 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[PR] Refactor some methods in EmrContainerHook [airflow]

2023-12-01 Thread via GitHub
hussein-awala opened a new pull request, #35999: URL: https://github.com/apache/airflow/pull/35999 Avoid creating unnecessary variables, and replace break/return by a fast return. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] Add a cache for weaviate client [airflow]

2023-12-01 Thread via GitHub
ephraimbuddy commented on code in PR #35983: URL: https://github.com/apache/airflow/pull/35983#discussion_r1411923562 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,18 +93,14 @@ def get_conn(self) -> WeaviateClient: url=url, auth_client_secret=auth_clien

[PR] Make "placeholder" of ODBC configurable in UI [airflow]

2023-12-01 Thread via GitHub
jarnedemunter1 opened a new pull request, #36000: URL: https://github.com/apache/airflow/pull/36000 Parametrize the "placeholder" variable in the ODBC hook, making it configurable from the connection defined in the UI. For some databases, the placeholder is not

Re: [PR] Make "placeholder" of ODBC configurable in UI [airflow]

2023-12-01 Thread via GitHub
boring-cyborg[bot] commented on PR #36000: URL: https://github.com/apache/airflow/pull/36000#issuecomment-1835891181 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[PR] Fix a bug in get_iam_token for Redshift Serverless [airflow]

2023-12-01 Thread via GitHub
hussein-awala opened a new pull request, #36001: URL: https://github.com/apache/airflow/pull/36001 This PR fixes a typo in the method used to get the credentials in the Redshift Credentials client, where its name is `get_credentials` instead of `get_cluster_credentials`, also the arguments

Re: [PR] Fix a bug in get_iam_token for Redshift Serverless [airflow]

2023-12-01 Thread via GitHub
hussein-awala commented on PR #36001: URL: https://github.com/apache/airflow/pull/36001#issuecomment-1835894798 For reviewers: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/redshift-serverless/client/get_credentials.html#get-credentials -- This is an automated

(airflow) 01/01: Add support for chicken-egg providers to dockerhub release process

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git commit 470e0851934e7b71c1ffbc895abc1175d3af27ce Author: Jarek Potiuk AuthorD

(airflow) branch add-chicken-egg-providers-support-for-dockerhub-release created (now 470e085193)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git at 470e085193 Add support for chicken-egg providers to dockerhub relea

[PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
potiuk opened a new pull request, #36002: URL: https://github.com/apache/airflow/pull/36002 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)*

Re: [PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
potiuk commented on PR #36002: URL: https://github.com/apache/airflow/pull/36002#issuecomment-1835917049 Testing the workflow changes here: https://github.com/apache/airflow/actions/runs/7059328822 -- This is an automated message from the Apache Git Service. To respond to the message, ple

(airflow) branch add-chicken-egg-providers-support-for-dockerhub-release updated (470e085193 -> 284f6dfa7b)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git omit 470e085193 Add support for chicken-egg providers to dockerhub relea

(airflow) 01/01: Add support for chicken-egg providers to dockerhub release process

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git commit 284f6dfa7b8812af28c1795c994ff402622935c6 Author: Jarek Potiuk AuthorD

[PR] Switch "latest" image to point to newest supported Python version [airflow]

2023-12-01 Thread via GitHub
potiuk opened a new pull request, #36003: URL: https://github.com/apache/airflow/pull/36003 Following the lazy consensus to change the "latest" image to point to "newest" Python version, we are changing the release method to follow it. https://lists.apache.org/thread/0oxnvct24xlqsj76z

(airflow) 01/01: Add support for chicken-egg providers to dockerhub release process

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git commit 2f5fb2a9b649f11d768b33619735272aff120ae0 Author: Jarek Potiuk AuthorD

Re: [PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
potiuk commented on PR #36002: URL: https://github.com/apache/airflow/pull/36002#issuecomment-1835984846 Next attempt: https://github.com/apache/airflow/actions/runs/7059562344 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

(airflow) branch add-chicken-egg-providers-support-for-dockerhub-release updated (284f6dfa7b -> 2f5fb2a9b6)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git omit 284f6dfa7b Add support for chicken-egg providers to dockerhub relea

Re: [PR] Refactor `get_output_location` method in AthenaHook [airflow]

2023-12-01 Thread via GitHub
hussein-awala merged PR #35996: URL: https://github.com/apache/airflow/pull/35996 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated (396b1ba34e -> 9f212d4276)

2023-12-01 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 396b1ba34e Cleanup code for elasticsearch<8 (#35707) add 9f212d4276 Refacto get_output_location in AthenaHoo

Re: [PR] Switch "latest" image to point to newest supported Python version [airflow]

2023-12-01 Thread via GitHub
potiuk commented on code in PR #36003: URL: https://github.com/apache/airflow/pull/36003#discussion_r1412015133 ## dev/breeze/src/airflow_breeze/commands/release_management_commands.py: ## @@ -1368,19 +1369,18 @@ def release_prod_images( f"{dockerhub_repo}:{

(airflow) branch add-chicken-egg-providers-support-for-dockerhub-release updated (2f5fb2a9b6 -> cd0ecbcef2)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git omit 2f5fb2a9b6 Add support for chicken-egg providers to dockerhub relea

Re: [PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
potiuk commented on PR #36002: URL: https://github.com/apache/airflow/pull/36002#issuecomment-1836002019 Third attempt (getting closer) : https://github.com/apache/airflow/actions/runs/7059678993 -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] Make "placeholder" of ODBC configurable in UI [airflow]

2023-12-01 Thread via GitHub
hussein-awala commented on code in PR #36000: URL: https://github.com/apache/airflow/pull/36000#discussion_r1412018906 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -144,7 +144,7 @@ class DbApiHook(BaseHook): # Override with db-specific query to check connection

Re: [PR] Add a cache for weaviate client [airflow]

2023-12-01 Thread via GitHub
hussein-awala commented on code in PR #35983: URL: https://github.com/apache/airflow/pull/35983#discussion_r1412024381 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,18 +93,14 @@ def get_conn(self) -> WeaviateClient: url=url, auth_client_secret=auth_clie

Re: [PR] Add a cache for weaviate client [airflow]

2023-12-01 Thread via GitHub
hussein-awala commented on code in PR #35983: URL: https://github.com/apache/airflow/pull/35983#discussion_r1412024381 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,18 +93,14 @@ def get_conn(self) -> WeaviateClient: url=url, auth_client_secret=auth_clie

Re: [I] Commit failed: Local: No offset stored while using AwaitMessageTriggerFunctionSensor [airflow]

2023-12-01 Thread via GitHub
ddione84 commented on issue #32585: URL: https://github.com/apache/airflow/issues/32585#issuecomment-1836013287 I am also experencing the same problem with the same provider version and Airflow 2.7.2 -- This is an automated message from the Apache Git Service. To respond to the message, p

[PR] [AIP-44] Introduce Pydantic model for LogTemplate [airflow]

2023-12-01 Thread via GitHub
mhenc opened a new pull request, #36004: URL: https://github.com/apache/airflow/pull/36004 closes: #ISSUE related: #29312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

(airflow) branch add-chicken-egg-providers-support-for-dockerhub-release updated (cd0ecbcef2 -> 862d0b1cd0)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git omit cd0ecbcef2 Add support for chicken-egg providers to dockerhub relea

(airflow) 01/01: Add support for chicken-egg providers to dockerhub release process

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git commit 862d0b1cd0d4aa6427c3548c4b70058995affa7b Author: Jarek Potiuk AuthorD

Re: [PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
potiuk commented on PR #36002: URL: https://github.com/apache/airflow/pull/36002#issuecomment-1836048220 4th attempt - https://github.com/apache/airflow/actions/runs/7060055811 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Add a cache for weaviate client [airflow]

2023-12-01 Thread via GitHub
ephraimbuddy commented on code in PR #35983: URL: https://github.com/apache/airflow/pull/35983#discussion_r1412063903 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,18 +93,14 @@ def get_conn(self) -> WeaviateClient: url=url, auth_client_secret=auth_clien

(airflow) branch constraints-main updated: Updating constraints. Github run id:7059604534

2023-12-01 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new afb6ba58a9 Updating con

(airflow) branch add-chicken-egg-providers-support-for-dockerhub-release updated (862d0b1cd0 -> a82cbe0bb0)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git omit 862d0b1cd0 Add support for chicken-egg providers to dockerhub relea

(airflow) 01/01: Add support for chicken-egg providers to dockerhub release process

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git commit a82cbe0bb0c98ca4229148ce58ee4798e056c221 Author: Jarek Potiuk AuthorD

[PR] Fix warning message in `Connection.get_hook` in case of ImportError [airflow]

2023-12-01 Thread via GitHub
Taragolis opened a new pull request, #36005: URL: https://github.com/apache/airflow/pull/36005 Fix incorrect call of `warning.warn` by use `logging.Logger.error` which more semantic correct here. Two birds in one shot. --- **^ Add meaningful description above

Re: [PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
potiuk commented on PR #36002: URL: https://github.com/apache/airflow/pull/36002#issuecomment-1836075527 One step closer: https://github.com/apache/airflow/actions/runs/7060262963 - we need to also support case where airflow + other providers is installed from PyPI but the single provider f

Re: [I] Airflow v2.6.3 - "KeyError: 'Variable does not exist'" when it does exist [airflow]

2023-12-01 Thread via GitHub
babaMar commented on issue #33835: URL: https://github.com/apache/airflow/issues/33835#issuecomment-1836088173 Having the same problem after upgrading to 2.7.3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Pass conn ID to ObjectStoragePath via URI [airflow]

2023-12-01 Thread via GitHub
bolkedebruin merged PR #35913: URL: https://github.com/apache/airflow/pull/35913 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

(airflow) branch main updated: Pass conn ID to ObjectStoragePath via URI (#35913)

2023-12-01 Thread bolke
This is an automated email from the ASF dual-hosted git repository. bolke pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ab87cd02e4 Pass conn ID to ObjectStoragePath via URI

(airflow) branch add-chicken-egg-providers-support-for-dockerhub-release updated (a82cbe0bb0 -> 2b73d87310)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git omit a82cbe0bb0 Add support for chicken-egg providers to dockerhub relea

Re: [PR] Switch "latest" image to point to newest supported Python version [airflow]

2023-12-01 Thread via GitHub
potiuk merged PR #36003: URL: https://github.com/apache/airflow/pull/36003 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) 01/01: Add support for chicken-egg providers to dockerhub release process

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git commit 7ca71bd14239d26c274fffdec6bf4f89f340c030 Author: Jarek Potiuk AuthorD

(airflow) branch add-chicken-egg-providers-support-for-dockerhub-release updated (2b73d87310 -> 7ca71bd142)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-chicken-egg-providers-support-for-dockerhub-release in repository https://gitbox.apache.org/repos/asf/airflow.git omit 2b73d87310 Add support for chicken-egg providers to dockerhub relea

(airflow) branch main updated (ab87cd02e4 -> 4117f1b013)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from ab87cd02e4 Pass conn ID to ObjectStoragePath via URI (#35913) add 4117f1b013 Switch "latest" image to point to new

[PR] Fix reraise outside of try block in `AthenaHook.get_output_location` [airflow]

2023-12-01 Thread via GitHub
Taragolis opened a new pull request, #36008: URL: https://github.com/apache/airflow/pull/36008 In case if `response` is None or some similar than current code would try to reraise error outside of any exceptions In additional rewrite conditions to reduce "ladder-effect"

Re: [I] Scheduler executes tasks for temporary DagRuns created by the airflow tasks test command [airflow]

2023-12-01 Thread via GitHub
brki commented on issue #35994: URL: https://github.com/apache/airflow/issues/35994#issuecomment-1836128059 I also see that behaviour on 2.7.3. I don't know if it's related, but now whenever I test a task from the cli, regardless of whether or not the task finishes successfully, I see

Re: [PR] Add helper function for CRUD operations on weaviate's schema and class objects [airflow]

2023-12-01 Thread via GitHub
utkarsharma2 commented on code in PR #35919: URL: https://github.com/apache/airflow/pull/35919#discussion_r1412111940 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,22 +99,117 @@ def test_connection(self) -> tuple[bool, str]: self.log.error("Error testin

Re: [PR] Add helper function for CRUD operations on weaviate's schema and class objects [airflow]

2023-12-01 Thread via GitHub
utkarsharma2 commented on code in PR #35919: URL: https://github.com/apache/airflow/pull/35919#discussion_r1412115290 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,22 +99,117 @@ def test_connection(self) -> tuple[bool, str]: self.log.error("Error testin

Re: [PR] Add helper function for CRUD operations on weaviate's schema and class objects [airflow]

2023-12-01 Thread via GitHub
utkarsharma2 commented on code in PR #35919: URL: https://github.com/apache/airflow/pull/35919#discussion_r1412116784 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -94,22 +99,117 @@ def test_connection(self) -> tuple[bool, str]: self.log.error("Error testin

Re: [PR] Add helper function for CRUD operations on weaviate's schema and class objects [airflow]

2023-12-01 Thread via GitHub
utkarsharma2 commented on code in PR #35919: URL: https://github.com/apache/airflow/pull/35919#discussion_r1412124956 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -112,22 +117,170 @@ def test_connection(self) -> tuple[bool, str]: self.log.error("Error test

[PR] Fix reraise outside of try block in `AzureSynapsePipelineRunLink.get_fields_from_url` [airflow]

2023-12-01 Thread via GitHub
Taragolis opened a new pull request, #36009: URL: https://github.com/apache/airflow/pull/36009 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guideline

Re: [PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
potiuk commented on PR #36002: URL: https://github.com/apache/airflow/pull/36002#issuecomment-1836157199 OK. The workflow passed here (including image verification step) - (with some intermittent failure for 3.11 which I re-run) https://github.com/apache/airflow/actions/runs/70604441

Re: [PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
potiuk commented on PR #36002: URL: https://github.com/apache/airflow/pull/36002#issuecomment-1836159499 (note - you will need to pull the images again if you pulled them previously ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
ephraimbuddy commented on PR #36002: URL: https://github.com/apache/airflow/pull/36002#issuecomment-1836166383 I will work on getting a new beta. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

(airflow) branch main updated: Add a cache for weaviate client (#35983)

2023-12-01 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8be03c9937 Add a cache for weaviate client

Re: [PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
potiuk commented on PR #36002: URL: https://github.com/apache/airflow/pull/36002#issuecomment-1836169982 > I will work on getting a new beta. WDYT? I think it makes sense only if this one is merged an there are other fixes cherry-picked - but yeah - why not :) BTW. There is als

Re: [PR] Add a cache for weaviate client [airflow]

2023-12-01 Thread via GitHub
ephraimbuddy merged PR #35983: URL: https://github.com/apache/airflow/pull/35983 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

Re: [PR] Add helper function for CRUD operations on weaviate's schema and class objects [airflow]

2023-12-01 Thread via GitHub
utkarsharma2 commented on code in PR #35919: URL: https://github.com/apache/airflow/pull/35919#discussion_r1412141618 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -112,22 +117,170 @@ def test_connection(self) -> tuple[bool, str]: self.log.error("Error test

Re: [PR] Fix handling of single quotes in `RedshiftToS3Operator` [airflow]

2023-12-01 Thread via GitHub
vincbeck commented on code in PR #35986: URL: https://github.com/apache/airflow/pull/35986#discussion_r1412169388 ## airflow/providers/amazon/aws/transfers/redshift_to_s3.py: ## @@ -141,8 +141,10 @@ def __init__( def _build_unload_query( self, credentials_block: st

Re: [PR] Update example_ec2 to Create Instance Which Supports Hibernation (#35790) [airflow]

2023-12-01 Thread via GitHub
vincbeck commented on code in PR #35839: URL: https://github.com/apache/airflow/pull/35839#discussion_r1412170813 ## tests/system/providers/amazon/aws/example_ec2.py: ## @@ -50,11 +50,14 @@ def get_latest_ami_id(): # on how they name the new images. This page should have A

(airflow) branch main updated: Fix warning message in `Connection.get_hook` in case of ImportError (#36005)

2023-12-01 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 217c89325a Fix warning message in `Connection.get

Re: [PR] Fix warning message in `Connection.get_hook` in case of ImportError [airflow]

2023-12-01 Thread via GitHub
vincbeck merged PR #36005: URL: https://github.com/apache/airflow/pull/36005 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

(airflow) branch main updated: Add support for chicken-egg providers to dockerhub release process (#36002)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8829d1732c Add support for chicken-egg providers to

Re: [PR] Refactor some methods in EmrContainerHook [airflow]

2023-12-01 Thread via GitHub
hussein-awala merged PR #35999: URL: https://github.com/apache/airflow/pull/35999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [PR] Add support for chicken-egg providers to dockerhub release process [airflow]

2023-12-01 Thread via GitHub
potiuk merged PR #36002: URL: https://github.com/apache/airflow/pull/36002 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (8829d1732c -> b1e547ef53)

2023-12-01 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 8829d1732c Add support for chicken-egg providers to dockerhub release process (#36002) add b1e547ef53 Refac

Re: [PR] Add helper function for CRUD operations on weaviate's schema and class objects [airflow]

2023-12-01 Thread via GitHub
utkarsharma2 commented on code in PR #35919: URL: https://github.com/apache/airflow/pull/35919#discussion_r1412192152 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -112,22 +117,170 @@ def test_connection(self) -> tuple[bool, str]: self.log.error("Error test

(airflow) branch main updated: Improve typing hints for only_client_type decorator (#35997)

2023-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new decc6d9414 Improve typing hints for only_client_typ

Re: [PR] Improve typing hints for `only_client_type` decorator [airflow]

2023-12-01 Thread via GitHub
potiuk merged PR #35997: URL: https://github.com/apache/airflow/pull/35997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] Extend Notifier interface [airflow]

2023-12-01 Thread via GitHub
vchiapaikeo commented on issue #35381: URL: https://github.com/apache/airflow/issues/35381#issuecomment-1836239909 Happy to give this a shot. Will likely start work on it next week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [I] dag_hash changed when using the XComArgs feature [airflow]

2023-12-01 Thread via GitHub
potiuk commented on issue #35998: URL: https://github.com/apache/airflow/issues/35998#issuecomment-1836241002 Don't use `days_ago`. It's a bad practice from Airlow 1.10 which we since fixed in all (I believe) our examples and documentations. `days_ago` effectively calculates new start_date

Re: [I] dag_hash changed when using the XComArgs feature [airflow]

2023-12-01 Thread via GitHub
potiuk closed issue #35998: dag_hash changed when using the XComArgs feature URL: https://github.com/apache/airflow/issues/35998 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] [AIP-44] Migrate TaskInstance._run_raw_task to Internal API [airflow]

2023-12-01 Thread via GitHub
mhenc opened a new pull request, #36010: URL: https://github.com/apache/airflow/pull/36010 closes: #29312 related: #29312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Fix handling of single quotes in `RedshiftToS3Operator` [airflow]

2023-12-01 Thread via GitHub
vincbeck commented on code in PR #35986: URL: https://github.com/apache/airflow/pull/35986#discussion_r1412210256 ## airflow/providers/amazon/aws/transfers/redshift_to_s3.py: ## @@ -141,8 +141,10 @@ def __init__( def _build_unload_query( self, credentials_block: st

Re: [PR] Add type hints for AWS provider [airflow]

2023-12-01 Thread via GitHub
Taragolis commented on PR #35966: URL: https://github.com/apache/airflow/pull/35966#issuecomment-1836256294 I think it not simple. I've tried in the past to do with separate by client / resource types hooks, but it was before async support was added. In general it required to rework s

  1   2   >