Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-25 Thread via GitHub
Joffreybvn commented on code in PR #34964: URL: https://github.com/apache/airflow/pull/34964#discussion_r1370789066 ## tests/providers/docker/hooks/test_docker.py: ## @@ -36,7 +36,7 @@ TEST_CONN = {"host": "some.docker.registry.com", "login": "some_user", "password": "some_p4$

Re: [PR] Replace blocking IO with async IO in AsyncKubernetesHook [airflow]

2023-10-25 Thread via GitHub
potiuk commented on PR #35162: URL: https://github.com/apache/airflow/pull/35162#issuecomment-1780479215 OK. I think I figured out exactly what was going on and have a fix in #35191 that should work in all cases. The change I implemented to make #34729 work with the new pre-installed

[PR] Fix local k8s test venv installation to accomodate for local changes [airflow]

2023-10-25 Thread via GitHub
potiuk opened a new pull request, #35191: URL: https://github.com/apache/airflow/pull/35191 The #35099 switched installation of the local venv for Airflow to released packages, in order to accomodate for case from #34729 where we locally install a new provider that is preinstalled but has n

Re: [PR] Replace blocking IO with async IO in AsyncKubernetesHook [airflow]

2023-10-25 Thread via GitHub
potiuk commented on PR #35162: URL: https://github.com/apache/airflow/pull/35162#issuecomment-1780412471 Hmm. This is an interesting one. The change to use `apache-airflow` has been implemented few days ago - in order to accomodate an interaction with #34729 PR. Honestly, it's super strang

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-25 Thread via GitHub
eladkal commented on PR #35185: URL: https://github.com/apache/airflow/pull/35185#issuecomment-1780370460 Helm tests are failing :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[I] `GoogleBaseHook` is ignoring `key_secret_name` in `provide_gcp_credential_file_as_context` method [airflow]

2023-10-25 Thread via GitHub
teslur opened a new issue, #35190: URL: https://github.com/apache/airflow/issues/35190 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened When I ran [this example code](https://github.com/apache/airflow/blob/main/tests/system/prov

Re: [PR] The sanitize_for_serialization(cls, obj) function outputs an incorret time format [airflow-client-python]

2023-10-25 Thread via GitHub
junqilv commented on PR #99: URL: https://github.com/apache/airflow-client-python/pull/99#issuecomment-1780268698 > Could you please explain (in commit message ideally as well) what was the problem it solves ? I don't see why it was wrong b) it would require to have test added. When

Re: [I] Cannot install Helm chart via Argo CD [airflow]

2023-10-25 Thread via GitHub
jmilldotdev closed issue #35159: Cannot install Helm chart via Argo CD URL: https://github.com/apache/airflow/issues/35159 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [I] Cannot install Helm chart via Argo CD [airflow]

2023-10-25 Thread via GitHub
jmilldotdev commented on issue #35159: URL: https://github.com/apache/airflow/issues/35159#issuecomment-1780228968 skill issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Add S3 to Azure Blob Transfer Operator [airflow]

2023-10-25 Thread via GitHub
github-actions[bot] closed pull request #31585: Add S3 to Azure Blob Transfer Operator URL: https://github.com/apache/airflow/pull/31585 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] control task group concurrency: use allow list for mapping index [airflow]

2023-10-25 Thread via GitHub
github-actions[bot] closed pull request #32122: control task group concurrency: use allow list for mapping index URL: https://github.com/apache/airflow/pull/32122 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] PoC: Implement Docker Registry Auth Protocol [airflow]

2023-10-25 Thread via GitHub
github-actions[bot] closed pull request #33602: PoC: Implement Docker Registry Auth Protocol URL: https://github.com/apache/airflow/pull/33602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] control task group concurrency: use allow list for mapping index [another way] [airflow]

2023-10-25 Thread via GitHub
github-actions[bot] closed pull request #34067: control task group concurrency: use allow list for mapping index [another way] URL: https://github.com/apache/airflow/pull/34067 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Replace blocking IO with async IO in AsyncKubernetesHook [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on PR #35162: URL: https://github.com/apache/airflow/pull/35162#issuecomment-1780206536 I've checked again, previous error happen because we tried to install Airflow from PyPI with constraints for main branch. If it failed it tried to install airflow without constraint

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-25 Thread via GitHub
eladkal commented on PR #35185: URL: https://github.com/apache/airflow/pull/35185#issuecomment-1780076435 I am not sure why helm chart tests are skipped here cc @potiuk In previous attempt to address the issue https://github.com/apache/airflow/pull/34505 we had failing helm tests so I w

Re: [PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-25 Thread via GitHub
eladkal closed pull request #35185: Fix K8S executor override config using pod_override_object URL: https://github.com/apache/airflow/pull/35185 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[PR] Prepare docs 3rd wave of Providers October 2023 [airflow]

2023-10-25 Thread via GitHub
eladkal opened a new pull request, #35187: URL: https://github.com/apache/airflow/pull/35187 TODO: - [ ] add cncf.kubenetes (waiting for 1 more PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
vincbeck commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372312570 ## airflow/providers/amazon/aws/transfers/http_to_s3.py: ## @@ -0,0 +1,163 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[I] Airflow REST API `Get tasks for DAG` returns error if DAG task has trigger rule `one_done` [airflow]

2023-10-25 Thread via GitHub
deanobob opened a new issue, #35186: URL: https://github.com/apache/airflow/issues/35186 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened Experienced in version 2.6.1 but appears to be an issue in the latest version too.

[PR] Fix K8S executor override config using pod_override_object [airflow]

2023-10-25 Thread via GitHub
hussein-awala opened a new pull request, #35185: URL: https://github.com/apache/airflow/pull/35185 related: #22298 Currently, we don't use all the configs from `pod_override_object` where we override some of them by the default K8S executor configurations or the kube config file (ex:

[airflow] branch main updated (0bb56315e6 -> 0940d09859)

2023-10-25 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 0bb56315e6 Added `overrides` parameter to CloudRunExecuteJobOperator (#34874) add 0940d09859 Send column lineage

Re: [PR] openlineage, common.sql: send column lineage from SQL operators. [airflow]

2023-10-25 Thread via GitHub
eladkal merged PR #34843: URL: https://github.com/apache/airflow/pull/34843 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
simonprydden commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372280177 ## airflow/providers/amazon/aws/transfers/http_to_s3.py: ## @@ -0,0 +1,163 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contrib

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
vincbeck commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372268503 ## tests/system/providers/amazon/aws/example_http_to_s3.py: ## @@ -0,0 +1,83 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: Gla

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
simonprydden commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372265659 ## tests/system/providers/amazon/aws/example_http_to_s3.py: ## @@ -0,0 +1,83 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment:

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
vincbeck commented on PR #35176: URL: https://github.com/apache/airflow/pull/35176#issuecomment-1779949120 I guess it is safe to put it in review (unless you have a good reason) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
vincbeck commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372253036 ## airflow/providers/amazon/aws/transfers/http_to_s3.py: ## @@ -0,0 +1,163 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [I] Wild char support when setting a new role for a set of DAGs [airflow]

2023-10-25 Thread via GitHub
potiuk commented on issue #35178: URL: https://github.com/apache/airflow/issues/35178#issuecomment-1779945535 This is unlikely to happen as we are moving away from the current mode in favour of more flexible system https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+m

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372244799 ## docs/apache-airflow-providers-amazon/transfer/http_to_s3.rst: ## @@ -0,0 +1,53 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more co

Re: [PR] The sanitize_for_serialization(cls, obj) function outputs an incorret time format [airflow-client-python]

2023-10-25 Thread via GitHub
potiuk commented on PR #99: URL: https://github.com/apache/airflow-client-python/pull/99#issuecomment-1779932375 Could you please explain (in commit message ideally as well) what was the problem it solves ? I don't see why it was wrong b) it would require to have test added. -- This i

Re: [PR] AWS ECS Executor [airflow]

2023-10-25 Thread via GitHub
potiuk commented on PR #34381: URL: https://github.com/apache/airflow/pull/34381#issuecomment-1779927333 Nice! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[airflow] branch main updated: Added `overrides` parameter to CloudRunExecuteJobOperator (#34874)

2023-10-25 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0bb56315e6 Added `overrides` parameter to CloudRun

Re: [PR] Added `overrides` parameter to CloudRunExecuteJobOperator [airflow]

2023-10-25 Thread via GitHub
boring-cyborg[bot] commented on PR #34874: URL: https://github.com/apache/airflow/pull/34874#issuecomment-1779906158 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [PR] Added `overrides` parameter to CloudRunExecuteJobOperator [airflow]

2023-10-25 Thread via GitHub
eladkal merged PR #34874: URL: https://github.com/apache/airflow/pull/34874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
simonprydden commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372198163 ## docs/apache-airflow-providers-amazon/transfer/http_to_s3.rst: ## @@ -0,0 +1,53 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more

Re: [PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on code in PR #35176: URL: https://github.com/apache/airflow/pull/35176#discussion_r1372191952 ## docs/apache-airflow-providers-amazon/transfer/http_to_s3.rst: ## @@ -0,0 +1,53 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more co

Re: [I] Import error of new dags cannot be shown on web UI for common users [airflow]

2023-10-25 Thread via GitHub
ahipp13 commented on issue #29897: URL: https://github.com/apache/airflow/issues/29897#issuecomment-1779842177 @timlichen I am able to replicate this behavior with my setup. I have 2 roles: TEAM_ROLE: ![image](https://github.com/apache/airflow/assets/118911990/bb164e4c-0a62-49cf-a

Re: [I] Import error of new dags cannot be shown on web UI for common users [airflow]

2023-10-25 Thread via GitHub
timlichen commented on issue #29897: URL: https://github.com/apache/airflow/issues/29897#issuecomment-1779820918 I was unable to replicate this error/behavior when I looked into this previously, it may no longer be an issue. -- This is an automated message from the Apache Git Service. To

Re: [I] Import error of new dags cannot be shown on web UI for common users [airflow]

2023-10-25 Thread via GitHub
ahipp13 commented on issue #29897: URL: https://github.com/apache/airflow/issues/29897#issuecomment-1779790643 This is still open in 2.6.3. Commenting here because I want to know if there are any updates on this! -- This is an automated message from the Apache Git Service. To respond to t

Re: [I] Airflow Does Not Display ImportError for certain Roles [airflow]

2023-10-25 Thread via GitHub
ahipp13 closed issue #35177: Airflow Does Not Display ImportError for certain Roles URL: https://github.com/apache/airflow/issues/35177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] Airflow Does Not Display ImportError for certain Roles [airflow]

2023-10-25 Thread via GitHub
ahipp13 commented on issue #35177: URL: https://github.com/apache/airflow/issues/35177#issuecomment-1779788524 I found similar issue here: https://github.com/apache/airflow/issues/29897 going to close this issue. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Remove offset-based pagination from `list_jobs` function in `DatabricksHook` [airflow]

2023-10-25 Thread via GitHub
eladkal commented on PR #34926: URL: https://github.com/apache/airflow/pull/34926#issuecomment-1779763315 Please also add `5.0.0` entry to versions in `provider.yaml:` https://github.com/apache/airflow/blob/e9987d50598f70d84cbb2a5d964e21020e81c080/airflow/providers/databricks/provider

Re: [PR] Remove offset-based pagination from `list_jobs` function in `DatabricksHook` [airflow]

2023-10-25 Thread via GitHub
eladkal commented on code in PR #34926: URL: https://github.com/apache/airflow/pull/34926#discussion_r1372113523 ## airflow/providers/databricks/CHANGELOG.rst: ## @@ -61,6 +61,13 @@ Misc 4.5.0 . +Breaking changes + + +The ``offset`` parameter has been dep

Re: [PR] AWS ECS Executor [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on PR #34381: URL: https://github.com/apache/airflow/pull/34381#issuecomment-1779755027 Nice. Still try to find time to actually try it 🤣 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[airflow] branch main updated: Add AWS ECS Executor (#34381)

2023-10-25 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5f4d2b5dee Add AWS ECS Executor (#34381) 5f4d2b5de

Re: [PR] AWS ECS Executor [airflow]

2023-10-25 Thread via GitHub
eladkal commented on PR #34381: URL: https://github.com/apache/airflow/pull/34381#issuecomment-1779744622 🎉 🎉 🎉 🎉 🎉 🎉 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] AWS ECS Executor [airflow]

2023-10-25 Thread via GitHub
eladkal merged PR #34381: URL: https://github.com/apache/airflow/pull/34381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [I] Airflow Can't Terminate Tasks that are created by a DAG that uses DAG Decorator. [airflow]

2023-10-25 Thread via GitHub
Taragolis closed issue #35179: Airflow Can't Terminate Tasks that are created by a DAG that uses DAG Decorator. URL: https://github.com/apache/airflow/issues/35179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Update dev instructions to use breeze command directly with short package ids to build docs [airflow]

2023-10-25 Thread via GitHub
boring-cyborg[bot] commented on PR #35080: URL: https://github.com/apache/airflow/pull/35080#issuecomment-1779690654 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [PR] Update dev instructions to use breeze command directly with short package ids to build docs [airflow]

2023-10-25 Thread via GitHub
jedcunningham commented on PR #35080: URL: https://github.com/apache/airflow/pull/35080#issuecomment-1779692157 Thanks @anudeep-pv! Congrats on your first commit! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Update dev instructions to use breeze command directly with short package ids to build docs [airflow]

2023-10-25 Thread via GitHub
jedcunningham merged PR #35080: URL: https://github.com/apache/airflow/pull/35080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[airflow] branch main updated: Update dev instructions to use breeze command directly with short package ids to build docs (#35080)

2023-10-25 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f1f5560d97 Update dev instructions to use br

[airflow] branch main updated: Add distinct function to MongoHook in apache-airflow-providers-mongo (#34466)

2023-10-25 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 647e76716e Add distinct function to MongoHook in

Re: [PR] Add distinct function to MongoHook in apache-airflow-providers-mongo [airflow]

2023-10-25 Thread via GitHub
boring-cyborg[bot] commented on PR #34466: URL: https://github.com/apache/airflow/pull/34466#issuecomment-1779680349 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [PR] Add distinct function to MongoHook in apache-airflow-providers-mongo [airflow]

2023-10-25 Thread via GitHub
vincbeck merged PR #34466: URL: https://github.com/apache/airflow/pull/34466 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[PR] reset stats instance in daemon context [airflow]

2023-10-25 Thread via GitHub
pavansharma36 opened a new pull request, #35181: URL: https://github.com/apache/airflow/pull/35181 related: #33897 --- Fix services crashing in daemon mode when statsd is enabled. -- This is an automated message from the Apache Git Service. To

Re: [I] Chart: Allow mounting DAGs to a custom path in airflow containers. [airflow]

2023-10-25 Thread via GitHub
jedcunningham closed issue #34959: Chart: Allow mounting DAGs to a custom path in airflow containers. URL: https://github.com/apache/airflow/issues/34959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Reset stats logger in daemon context to reinitialize in child process [airflow]

2023-10-25 Thread via GitHub
pavansharma36 commented on PR #34769: URL: https://github.com/apache/airflow/pull/34769#issuecomment-1779653801 Will raise new PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[airflow] branch main updated: Chart: configurable mountPath for DAGs volume (#35083)

2023-10-25 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ac53a9aaab Chart: configurable mountPath for

Re: [I] Airflow Can't Terminate Tasks that are created by a DAG that uses DAG Decorator. [airflow]

2023-10-25 Thread via GitHub
ImmortalLotus commented on issue #35179: URL: https://github.com/apache/airflow/issues/35179#issuecomment-1779653517 the main thing is that the DAG runs correctly, however the pod isn't deleted. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Chart: configurable mountPath for DAGs volume [airflow]

2023-10-25 Thread via GitHub
jedcunningham merged PR #35083: URL: https://github.com/apache/airflow/pull/35083 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [PR] Reset stats logger in daemon context to reinitialize in child process [airflow]

2023-10-25 Thread via GitHub
pavansharma36 closed pull request #34769: Reset stats logger in daemon context to reinitialize in child process URL: https://github.com/apache/airflow/pull/34769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [I] Airflow Can't Terminate Tasks that are created by a DAG that uses DAG Decorator. [airflow]

2023-10-25 Thread via GitHub
boring-cyborg[bot] commented on issue #35179: URL: https://github.com/apache/airflow/issues/35179#issuecomment-1779648317 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] Airflow Can't Terminate Tasks that are created by a DAG that uses DAG Decorator. [airflow]

2023-10-25 Thread via GitHub
ImmortalLotus opened a new issue, #35179: URL: https://github.com/apache/airflow/issues/35179 ### Apache Airflow version 2.7.2 ### What happened airflow throws this error when trying to terminate a task that comes from a DAG that is created using DAG decorator and then i

Re: [PR] Add distinct function to MongoHook in apache-airflow-providers-mongo [airflow]

2023-10-25 Thread via GitHub
Taragolis commented on code in PR #34466: URL: https://github.com/apache/airflow/pull/34466#discussion_r1372017572 ## airflow/providers/mongo/hooks/mongo.py: ## @@ -366,3 +366,22 @@ def delete_many( collection = self.get_collection(mongo_collection, mongo_db=mongo_db)

[PR] The sanitize_for_serialization(cls, obj) function outputs an incorret time format [airflow-client-python]

2023-10-25 Thread via GitHub
junqilv opened a new pull request, #99: URL: https://github.com/apache/airflow-client-python/pull/99 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [I] The logical date parameter for DAGRun is incorrect [airflow-client-python]

2023-10-25 Thread via GitHub
junqilv commented on issue #98: URL: https://github.com/apache/airflow-client-python/issues/98#issuecomment-1779554271 The sanitize_for_serialization(cls, obj) function outputs an incorrect time format -- This is an automated message from the Apache Git Service. To respond to the message

[I] The logical date parameter for DAGRun is incorrect [airflow-client-python]

2023-10-25 Thread via GitHub
junqilv opened a new issue, #98: URL: https://github.com/apache/airflow-client-python/issues/98 `Exception when calling DAGRunAPI->post_dag_run: (400) Reason: BAD REQUEST HTTP response headers: HTTPHeaderDict({'Server': 'gunicorn', 'Date': 'Wed, 25 Oct 2023 15:29:05 GMT', 'Connection':

[airflow] branch main updated (36c5c111ec -> 95980a9bc5)

2023-10-25 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 36c5c111ec Permitting airflow kerberos to run in different modes (#35146) add 95980a9bc5 Enable encryption in

Re: [PR] Enable encryption in S3 download_files() hook. [airflow]

2023-10-25 Thread via GitHub
Taragolis merged PR #35037: URL: https://github.com/apache/airflow/pull/35037 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Add Cohere Provider [airflow]

2023-10-25 Thread via GitHub
utkarsharma2 commented on PR #34921: URL: https://github.com/apache/airflow/pull/34921#issuecomment-1779461088 > Do we need the .svg files to be added to this PR? Yes, we do need to add them, as the provider dependency changes, the breeze output command interface also changes and that

[PR] Implement Dag SLAs [airflow]

2023-10-25 Thread via GitHub
syun64 opened a new pull request, #33532: URL: https://github.com/apache/airflow/pull/33532 closes #32816 ### Summary This PR introduces Dag SLAs as a part of **AIP-57 Refactor SLA Feature**. After many discussions, the community has voted to introduce new

Re: [PR] Add Cohere Provider [airflow]

2023-10-25 Thread via GitHub
utkarsharma2 commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1371890752 ## airflow/providers/cohere/operators/embedding.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Add Cohere Provider [airflow]

2023-10-25 Thread via GitHub
utkarsharma2 commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1371886493 ## airflow/providers/cohere/operators/embedding.py: ## @@ -0,0 +1,82 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-25 Thread via GitHub
dacort commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1371878861 ## airflow/providers/amazon/aws/links/emr.py: ## @@ -66,3 +82,98 @@ def get_log_uri( return None log_uri = S3Hook.parse_s3_url(cluster_info["LogUri"])

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-25 Thread via GitHub
dacort commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1371859865 ## airflow/providers/amazon/aws/operators/emr.py: ## @@ -1352,6 +1391,101 @@ def on_kill(self) -> None: check_interval_seconds=self.waiter_delay,

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-25 Thread via GitHub
dacort commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1371857665 ## airflow/providers/amazon/aws/operators/emr.py: ## @@ -1173,13 +1184,35 @@ class EmrServerlessStartJobOperator(BaseOperator): "execution_role_arn",

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-25 Thread via GitHub
dacort commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1371855339 ## airflow/providers/amazon/aws/links/emr.py: ## @@ -66,3 +82,98 @@ def get_log_uri( return None log_uri = S3Hook.parse_s3_url(cluster_info["LogUri"])

Re: [I] Track SparkSubmitHook Yarn Cluster application with Yarn CLI [airflow]

2023-10-25 Thread via GitHub
vchiapaikeo commented on issue #24171: URL: https://github.com/apache/airflow/issues/24171#issuecomment-1779390194 Ooo @tokoko , I just recently learned that Spark master exposes a rest API with `spark.master.rest.enabled true`. Can you check if that's the case on your yarn deployment? If p

Re: [PR] Fix task id copy button copying wrong id [airflow]

2023-10-25 Thread via GitHub
bbovenzi commented on code in PR #34904: URL: https://github.com/apache/airflow/pull/34904#discussion_r1371812335 ## airflow/www/static/js/components/Clipboard.tsx: ## @@ -43,9 +43,13 @@ export const ClipboardButton = forwardRef( }, ref ) => { -const { hasCopied

Re: [I] logging/remote_logging - option to disable creation of cloudwatch log groups [airflow]

2023-10-25 Thread via GitHub
nwsparks commented on issue #35164: URL: https://github.com/apache/airflow/issues/35164#issuecomment-1779320595 so it does seem watchtower was updated to not do this every time, however the code for it is wrapped in a try/catch that first does describe log groups, then swallows any errors a

[airflow] branch main updated: Permitting airflow kerberos to run in different modes (#35146)

2023-10-25 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 36c5c111ec Permitting airflow kerberos to run in di

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-25 Thread via GitHub
potiuk merged PR #35146: URL: https://github.com/apache/airflow/pull/35146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] logging/remote_logging - option to disable creation of cloudwatch log groups [airflow]

2023-10-25 Thread via GitHub
nwsparks commented on issue #35164: URL: https://github.com/apache/airflow/issues/35164#issuecomment-1779293956 good suggestion, thanks @Taragolis -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch main updated: update README_RELEASE_PROVIDER_PACKAGES.md (#35175)

2023-10-25 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2a0cd3a8d8 update README_RELEASE_PROVIDER_PACKAGES

Re: [PR] update README_RELEASE_PROVIDER_PACKAGES.md [airflow]

2023-10-25 Thread via GitHub
eladkal merged PR #35175: URL: https://github.com/apache/airflow/pull/35175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [PR] Remove offset-based pagination from `list_jobs` function in `DatabricksHook` [airflow]

2023-10-25 Thread via GitHub
oleksiidav commented on PR #34926: URL: https://github.com/apache/airflow/pull/34926#issuecomment-1779249193 Hi @eladkal, could you ptal? This is ready to be merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[I] Airflow Does Not Display ImportError for certain Roles [airflow]

2023-10-25 Thread via GitHub
ahipp13 opened a new issue, #35177: URL: https://github.com/apache/airflow/issues/35177 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened So I am on Airflow 2.6.3, and we ran into a weird bug that I am wondering if it has been se

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-25 Thread via GitHub
amoghrajesh commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1779214261 @potiuk @eladkal @jedcunningham I'd like to run the PR again by all of you. CI is green now Take a look when you have some time -- This is an automated message from the Apa

[PR] Add http to s3 operator [airflow]

2023-10-25 Thread via GitHub
simonprydden opened a new pull request, #35176: URL: https://github.com/apache/airflow/pull/35176 Adding HttpToS3Operator and unit tests. --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow

[PR] update README_RELEASE_PROVIDER_PACKAGES.md [airflow]

2023-10-25 Thread via GitHub
eladkal opened a new pull request, #35175: URL: https://github.com/apache/airflow/pull/35175 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)

Re: [PR] Use timeout when inntializing _timeout_sec of TaskStateTrigger [airflow]

2023-10-25 Thread via GitHub
Xiroo commented on PR #34715: URL: https://github.com/apache/airflow/pull/34715#issuecomment-1779177551 @hussein-awala hello, I changed to use timeout parameter Can I get review for change? -- This is an automated message from the Apache Git Service. To respond to the message, plea

[airflow] branch constraints-main updated: Updating constraints. Github run id:6638965943

2023-10-25 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 7224f38902 Updating con

Re: [I] Add CSRF protection to "/logout" [airflow]

2023-10-25 Thread via GitHub
hterik commented on issue #33030: URL: https://github.com/apache/airflow/issues/33030#issuecomment-1779099322 Will this interfere with single-sign-*out*, when using Oauth2 or Saml login? If a user logs out from the IDP, the IDP somehow need to notify Airflow to clear the the session, tha

Re: [PR] Add Cohere Provider [airflow]

2023-10-25 Thread via GitHub
utkarsharma2 commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1371602720 ## airflow/providers/cohere/hooks/cohere.py: ## @@ -0,0 +1,80 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [I] Enhanced DAG Navigation with Search Bar on Dag View Page [airflow]

2023-10-25 Thread via GitHub
hterik commented on issue #33210: URL: https://github.com/apache/airflow/issues/33210#issuecomment-1779092233 Voting yes on the search bar, looks really useful and should be simple to implement. Voting no on the tabs. Difficult to implement and my browser already has tabs :) Somet

Re: [PR] Add Cohere Provider [airflow]

2023-10-25 Thread via GitHub
utkarsharma2 commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1371602720 ## airflow/providers/cohere/hooks/cohere.py: ## @@ -0,0 +1,80 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

[I] Node label/capability based scheduling (for CeleryExecutor) [airflow]

2023-10-25 Thread via GitHub
hterik opened a new issue, #35174: URL: https://github.com/apache/airflow/issues/35174 ### Description Schedule tasks based on labels/capabilities with and+or matching, instead of just queue names. ### Use case/motivation Imagine you have tasks that can only run on speci

Re: [PR] openlineage, aws: Add OpenLineage support for AthenaOperator. [airflow]

2023-10-25 Thread via GitHub
mobuchowski commented on code in PR #35090: URL: https://github.com/apache/airflow/pull/35090#discussion_r1371557024 ## airflow/providers/amazon/aws/hooks/athena.py: ## @@ -121,7 +122,26 @@ def run_query( self.log.info("Query execution id: %s", query_execution_id)

Re: [PR] Add Open Search Provider [airflow]

2023-10-25 Thread via GitHub
pateash commented on PR #34705: URL: https://github.com/apache/airflow/pull/34705#issuecomment-1778986522 Hi @cjames23, Just a quick question regarding this one. as we know OpenSearch is just a fork of Elasticsearch which happened couple of years ago, Can we not use Elastic search

  1   2   >