[PR] Add missing hyphen in `data-time` format in Params docs [airflow]

2024-01-23 Thread via GitHub
esthomw opened a new pull request, #36966: URL: https://github.com/apache/airflow/pull/36966 --- The `format` argument for datetime parameter in Params should be passed with hyphen - `date-time`. Without it Airflow interprets this format as `date`. *

Re: [PR] Add missing hyphen in `data-time` format in Params docs [airflow]

2024-01-23 Thread via GitHub
boring-cyborg[bot] commented on PR #36966: URL: https://github.com/apache/airflow/pull/36966#issuecomment-1905484241 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

Re: [PR] Revert "Update statsd-mapping.yml to support Airflow 2.X pool metrics (#36305) [airflow]

2024-01-23 Thread via GitHub
eladkal commented on PR #36959: URL: https://github.com/apache/airflow/pull/36959#issuecomment-1905484384 If we revert the PR what is the resolution to https://github.com/apache/airflow/issues/36245 ? -- This is an automated message from the Apache Git Service. To respond to the message,

[I] We use workers with Fargate, but there's a problem [airflow]

2024-01-23 Thread via GitHub
5pponent opened a new issue, #36967: URL: https://github.com/apache/airflow/issues/36967 ### Description When running tasks on Fargate, we need to make sure that capacity provisioning can be done on a per-task basis. Also, it would be nice to have multiple tasks running on one Fargat

Re: [I] We use workers with Fargate, but there's a problem [airflow]

2024-01-23 Thread via GitHub
boring-cyborg[bot] commented on issue #36967: URL: https://github.com/apache/airflow/issues/36967#issuecomment-1905504053 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Revert "Update statsd-mapping.yml to support Airflow 2.X pool metrics (#36305) [airflow]

2024-01-23 Thread via GitHub
shohamy7 commented on PR #36959: URL: https://github.com/apache/airflow/pull/36959#issuecomment-1905511860 @eladkal As far as I understand (and see), those metrics (`airflow.pool.queued_slots` & `airflow.pool.running_slots`) are exported (though in a slightly different way). It can b

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-23 Thread via GitHub
arjunanan6 commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1905521441 Test #36752 and it works as expected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Add Dataset support to objectstorage [airflow]

2024-01-23 Thread via GitHub
uranusjr commented on code in PR #36947: URL: https://github.com/apache/airflow/pull/36947#discussion_r1462883678 ## airflow/io/path.py: ## @@ -129,7 +130,11 @@ def __new__( return _cls(_cls._format_parsed_parts(drv, root, parts, **other_kwargs), **new_kwargs)

Re: [I] We use workers with Fargate, but there's a problem [airflow]

2024-01-23 Thread via GitHub
raphaelauv commented on issue #36967: URL: https://github.com/apache/airflow/issues/36967#issuecomment-1905582120 airflow desn't have the feature of kubernetes , airflow do not know and check the requirements of tasks -- This is an automated message from the Apache Git Service. To respond

Re: [PR] fix: DagRuns with UPSTREAM_FAILED tasks get stuck in the backfill. [airflow]

2024-01-23 Thread via GitHub
avkirilishin commented on PR #36954: URL: https://github.com/apache/airflow/pull/36954#issuecomment-1905612407 > LGTM. Can you check some test cases are failing? There was an issue with https://github.com/apache/airflow/pull/33242 It has already been fixed in https://github.com/apac

Re: [PR] fix: DagRuns with UPSTREAM_FAILED tasks get stuck in the backfill. [airflow]

2024-01-23 Thread via GitHub
avkirilishin commented on PR #36954: URL: https://github.com/apache/airflow/pull/36954#issuecomment-1905613720 > LGTM. Can you check some test cases are failing? There was an issue with https://github.com/apache/airflow/pull/33242 It has already been fixed in https://github.com/apac

Re: [PR] Fix assignment of template field in `__init__` in `custom_job` [airflow]

2024-01-23 Thread via GitHub
romsharon98 commented on code in PR #36789: URL: https://github.com/apache/airflow/pull/36789#discussion_r1462960112 ## airflow/providers/google/cloud/operators/vertex_ai/custom_job.py: ## @@ -445,9 +445,19 @@ def __init__( self, *, command: Sequence[s

[PR] fix(sentry): get ti from kwargs for TaskInstnace._schedule_downstream_tasks [airflow]

2024-01-23 Thread via GitHub
Lee-W opened a new pull request, #36968: URL: https://github.com/apache/airflow/pull/36968 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guideline

Re: [PR] Improve ExternalTaskSensor Async Implementation [airflow]

2024-01-23 Thread via GitHub
pankajastro commented on code in PR #36916: URL: https://github.com/apache/airflow/pull/36916#discussion_r1462985345 ## airflow/hooks/utils.py: ## @@ -0,0 +1,120 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the

Re: [PR] Improve ExternalTaskSensor Async Implementation [airflow]

2024-01-23 Thread via GitHub
eladkal commented on code in PR #36916: URL: https://github.com/apache/airflow/pull/36916#discussion_r1462987666 ## airflow/hooks/utils.py: ## @@ -0,0 +1,125 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: I don't think this is the right plac

Re: [PR] Improve ExternalTaskSensor Async Implementation [airflow]

2024-01-23 Thread via GitHub
pankajastro commented on code in PR #36916: URL: https://github.com/apache/airflow/pull/36916#discussion_r1462989027 ## airflow/hooks/utils.py: ## @@ -0,0 +1,125 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: Hmm, that makes sense, I'll move

Re: [PR] feat: full support for google credentials in gcloud-aio clients [airflow]

2024-01-23 Thread via GitHub
nathadfield commented on PR #36849: URL: https://github.com/apache/airflow/pull/36849#issuecomment-1905667804 It would be great to get some attention on this and #36903 from the reviewers. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-23 Thread via GitHub
potiuk commented on PR #36961: URL: https://github.com/apache/airflow/pull/36961#issuecomment-1905669633 I addressed all comments @jedcunningham and @aritra24 - but I have to close the PR to open it `apache` repository because removed `images` folder prevents from building CI image in `buil

(airflow) branch refactor-contributing-docs created (now 22dc7d4359)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git at 22dc7d4359 Split contributing docs to multiple files This branch includes the following new comm

[PR] Split contributing docs to multiple files [airflow]

2024-01-23 Thread via GitHub
potiuk opened a new pull request, #36969: URL: https://github.com/apache/airflow/pull/36969 Following #36936 and the fact that GitHub stopped rendering big .rst files, we also split CONTRIBUTING.rst into multiple files. It will be much easier to follow and it will render in GitHub.

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-23 Thread via GitHub
potiuk closed pull request #36961: Split contributing docs to multiple files URL: https://github.com/apache/airflow/pull/36961 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[I] Log Links Aren't Clickable in Grid Details Logs Tab [airflow]

2024-01-23 Thread via GitHub
travis-cook-sfdc opened a new issue, #35280: URL: https://github.com/apache/airflow/issues/35280 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened Currently using Airflow 2.6.2 but didn't see any fixes in the changelog for this o

Re: [I] Log Links Aren't Clickable in Grid Details Logs Tab [airflow]

2024-01-23 Thread via GitHub
amoghrajesh commented on issue #35280: URL: https://github.com/apache/airflow/issues/35280#issuecomment-1905671480 Reopening as it is being discussed here https://github.com/apache/airflow/pull/36829 -- This is an automated message from the Apache Git Service. To respond to the message, p

(airflow) branch refactor-contributing-docs updated (22dc7d4359 -> 063cea95f1)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git discard 22dc7d4359 Split contributing docs to multiple files add 063cea95f1 Split contributing docs

(airflow) branch refactor-contributing-docs updated (063cea95f1 -> 1ca022f944)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 063cea95f1 Split contributing docs to multiple files add 1ca022f944 Split contributing docs

(airflow) branch refactor-contributing-docs updated (1ca022f944 -> 36006a7fc6)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 1ca022f944 Split contributing docs to multiple files add 36006a7fc6 Split contributing docs

(airflow) branch refactor-contributing-docs updated (36006a7fc6 -> cec93f035f)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 36006a7fc6 Split contributing docs to multiple files add cec93f035f Split contributing docs

(airflow) branch refactor-contributing-docs updated (cec93f035f -> ed58b76ad4)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit cec93f035f Split contributing docs to multiple files add ed58b76ad4 Split contributing docs

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-23 Thread via GitHub
eladkal commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1463001545 ## airflow/providers/teradata/transfers/teradata_to_teradata.py: ## @@ -0,0 +1,91 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contr

[PR] Fix location requirement in DataflowTemplatedJobStartOperator [airflow]

2024-01-23 Thread via GitHub
shardbread opened a new pull request, #36970: URL: https://github.com/apache/airflow/pull/36970 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelin

Re: [PR] Fix location requirement in DataflowTemplatedJobStartOperator [airflow]

2024-01-23 Thread via GitHub
boring-cyborg[bot] commented on PR #36970: URL: https://github.com/apache/airflow/pull/36970#issuecomment-1905693793 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

(airflow) branch refactor-contributing-docs updated (b5f73ddb4d -> 8a1d523b01)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git discard b5f73ddb4d Split contributing docs to multiple files add 8a1d523b01 Split contributing docs

(airflow) branch refactor-contributing-docs updated (ed58b76ad4 -> b5f73ddb4d)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit ed58b76ad4 Split contributing docs to multiple files add b5f73ddb4d Split contributing docs

(airflow) branch refactor-contributing-docs updated (8a1d523b01 -> 78ddf2ab48)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 8a1d523b01 Split contributing docs to multiple files add 78ddf2ab48 Split contributing docs

(airflow) branch refactor-contributing-docs updated (78ddf2ab48 -> c70d6b5e08)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 78ddf2ab48 Split contributing docs to multiple files add c70d6b5e08 Split contributing docs

(airflow) branch refactor-contributing-docs updated (c70d6b5e08 -> d70eb6b5b9)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git discard c70d6b5e08 Split contributing docs to multiple files add d70eb6b5b9 Split contributing docs

Re: [PR] feat: full support for google credentials in gcloud-aio clients [airflow]

2024-01-23 Thread via GitHub
pankajastro commented on code in PR #36849: URL: https://github.com/apache/airflow/pull/36849#discussion_r1463026613 ## airflow/providers/google/common/hooks/base_google.py: ## @@ -639,6 +689,7 @@ async def get_sync_hook(self) -> Any: self._sync_hook = await sync_t

(airflow) branch refactor-contributing-docs updated (d70eb6b5b9 -> 7b9811b541)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git discard d70eb6b5b9 Split contributing docs to multiple files add 7b9811b541 Split contributing docs

Re: [PR] Airflow Teradata Provider: Initial implementation of Airflow Teradata Provider [airflow]

2024-01-23 Thread via GitHub
sunilkmallam commented on code in PR #36953: URL: https://github.com/apache/airflow/pull/36953#discussion_r1463031416 ## airflow/providers/teradata/transfers/teradata_to_teradata.py: ## @@ -0,0 +1,91 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

(airflow) branch refactor-contributing-docs updated (7b9811b541 -> cce17300ed)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 7b9811b541 Split contributing docs to multiple files add cce17300ed Split contributing docs

(airflow) branch refactor-contributing-docs updated (cce17300ed -> 8ca41e08b5)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git discard cce17300ed Split contributing docs to multiple files add 8ca41e08b5 Split contributing docs

Re: [PR] Fix google operators handling of impersonation chain [airflow]

2024-01-23 Thread via GitHub
pankajastro merged PR #36903: URL: https://github.com/apache/airflow/pull/36903 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airf

(airflow) branch refactor-contributing-docs updated (8ca41e08b5 -> e01fe004b7)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git discard 8ca41e08b5 Split contributing docs to multiple files add e01fe004b7 Split contributing docs

(airflow) branch main updated: Fix google operators handling of impersonation chain (#36903)

2024-01-23 Thread pankaj
This is an automated email from the ASF dual-hosted git repository. pankaj pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1c14767638 Fix google operators handling of imperso

(airflow) branch refactor-contributing-docs updated (e01fe004b7 -> b506679ffd)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit e01fe004b7 Split contributing docs to multiple files add b506679ffd Split contributing docs

Re: [PR] Fix location requirement in DataflowTemplatedJobStartOperator [airflow]

2024-01-23 Thread via GitHub
shardbread closed pull request #36970: Fix location requirement in DataflowTemplatedJobStartOperator URL: https://github.com/apache/airflow/pull/36970 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

(airflow) branch refactor-contributing-docs updated (b506679ffd -> 77255de769)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit b506679ffd Split contributing docs to multiple files add 77255de769 Split contributing docs

(airflow) branch refactor-contributing-docs updated (77255de769 -> 51908177ed)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 77255de769 Split contributing docs to multiple files add 51908177ed Split contributing docs

(airflow) branch refactor-contributing-docs updated (51908177ed -> e3adcd3869)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 51908177ed Split contributing docs to multiple files add e3adcd3869 Split contributing docs

Re: [PR] Revert "Update statsd-mapping.yml to support Airflow 2.X pool metrics (#36305) [airflow]

2024-01-23 Thread via GitHub
jedcunningham merged PR #36959: URL: https://github.com/apache/airflow/pull/36959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

(airflow) branch main updated (1c14767638 -> b7f84c4fa7)

2024-01-23 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 1c14767638 Fix google operators handling of impersonation chain (#36903) add b7f84c4fa7 Revert "Update sta

(airflow) branch refactor-contributing-docs updated (e3adcd3869 -> 9ac475c5bd)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git discard e3adcd3869 Split contributing docs to multiple files add 9ac475c5bd Split contributing docs

Re: [PR] feat: full support for google credentials in gcloud-aio clients [airflow]

2024-01-23 Thread via GitHub
m1racoli commented on code in PR #36849: URL: https://github.com/apache/airflow/pull/36849#discussion_r1463062271 ## airflow/providers/google/common/hooks/base_google.py: ## @@ -639,6 +689,7 @@ async def get_sync_hook(self) -> Any: self._sync_hook = await sync_to_a

Re: [PR] fix task_instance can not get task attribute correctly when turning on sentry [airflow]

2024-01-23 Thread via GitHub
jedcunningham commented on code in PR #36968: URL: https://github.com/apache/airflow/pull/36968#discussion_r1463064242 ## tests/models/test_taskinstance.py: ## @@ -3192,6 +3192,22 @@ def raise_skip_exception(): assert State.SKIPPED == ti.state assert callback_f

(airflow) branch refactor-contributing-docs updated (9ac475c5bd -> 8b002fecd2)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 9ac475c5bd Split contributing docs to multiple files add 8b002fecd2 Split contributing docs

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-23 Thread via GitHub
jedcunningham commented on PR #36961: URL: https://github.com/apache/airflow/pull/36961#issuecomment-1905766870 No problem (I had only partially reviewed it anyways 👍). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

(airflow) branch refactor-contributing-docs updated (8b002fecd2 -> 56b39816bf)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git discard 8b002fecd2 Split contributing docs to multiple files add 56b39816bf Split contributing docs

Re: [PR] feat: full support for google credentials in gcloud-aio clients [airflow]

2024-01-23 Thread via GitHub
m1racoli commented on code in PR #36849: URL: https://github.com/apache/airflow/pull/36849#discussion_r1463100649 ## airflow/providers/google/common/hooks/base_google.py: ## @@ -639,6 +689,7 @@ async def get_sync_hook(self) -> Any: self._sync_hook = await sync_to_a

Re: [PR] fix task_instance can not get task attribute correctly when turning on sentry [airflow]

2024-01-23 Thread via GitHub
Lee-W commented on code in PR #36968: URL: https://github.com/apache/airflow/pull/36968#discussion_r1463101163 ## tests/models/test_taskinstance.py: ## @@ -3192,6 +3192,22 @@ def raise_skip_exception(): assert State.SKIPPED == ti.state assert callback_function.

Re: [PR] feat: full support for google credentials in gcloud-aio clients [airflow]

2024-01-23 Thread via GitHub
m1racoli commented on code in PR #36849: URL: https://github.com/apache/airflow/pull/36849#discussion_r1463100649 ## airflow/providers/google/common/hooks/base_google.py: ## @@ -639,6 +689,7 @@ async def get_sync_hook(self) -> Any: self._sync_hook = await sync_to_a

(airflow) branch refactor-contributing-docs updated (56b39816bf -> 5ca3171914)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 56b39816bf Split contributing docs to multiple files add 5ca3171914 Split contributing docs

(airflow) branch refactor-contributing-docs updated (5ca3171914 -> aad8537003)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit 5ca3171914 Split contributing docs to multiple files add aad8537003 Split contributing docs

Re: [PR] The task is stuck in a queued state forever in case of pod launch errors [airflow]

2024-01-23 Thread via GitHub
hussein-awala commented on code in PR #36882: URL: https://github.com/apache/airflow/pull/36882#discussion_r1463107125 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py: ## @@ -434,9 +434,9 @@ def sync(self) -> None: )

(airflow) branch refactor-contributing-docs updated (aad8537003 -> db304088a5)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch refactor-contributing-docs in repository https://gitbox.apache.org/repos/asf/airflow.git omit aad8537003 Split contributing docs to multiple files add db304088a5 Split contributing docs

Re: [PR] feat: full support for google credentials in gcloud-aio clients [airflow]

2024-01-23 Thread via GitHub
pankajastro merged PR #36849: URL: https://github.com/apache/airflow/pull/36849 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airf

Re: [I] Impersonation logic missing in BigQuery Async operators [airflow]

2024-01-23 Thread via GitHub
pankajastro closed issue #34727: Impersonation logic missing in BigQuery Async operators URL: https://github.com/apache/airflow/issues/34727 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

(airflow) branch main updated (b7f84c4fa7 -> fbd21eda1e)

2024-01-23 Thread pankaj
This is an automated email from the ASF dual-hosted git repository. pankaj pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from b7f84c4fa7 Revert "Update statsd-mapping.yml to support Airflow 2.X pool metrics (#36305)" (#36959) add fbd21eda1

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-23 Thread via GitHub
potiuk commented on PR #36961: URL: https://github.com/apache/airflow/pull/36961#issuecomment-1905816653 The new PR -> even more split (testing is now spearate doc for every kind of tests we have + we have better ordering and navigation of documents in the contributor documentation - and ni

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-23 Thread via GitHub
potiuk commented on PR #36969: URL: https://github.com/apache/airflow/pull/36969#issuecomment-1905833272 All right. I think it should be basically READY for review. I am quite sure there are few broken links that I mised. So I'd really love more pairs of eyes looking at it: https://github.

Re: [PR] Split contributing docs to multiple files [airflow]

2024-01-23 Thread via GitHub
potiuk commented on PR #36969: URL: https://github.com/apache/airflow/pull/36969#issuecomment-1905837553 I also removed some outdated content and refreshed some other - everything I noticed was outdated or misplaced (I moved a few chapters around), so generally the structure and content sho

Re: [PR] fix task_instance can not get task attribute correctly when turning on sentry [airflow]

2024-01-23 Thread via GitHub
Lee-W commented on code in PR #36968: URL: https://github.com/apache/airflow/pull/36968#discussion_r1463128170 ## tests/models/test_taskinstance.py: ## @@ -3192,6 +3192,22 @@ def raise_skip_exception(): assert State.SKIPPED == ti.state assert callback_function.

Re: [PR] Add missing hyphen in `date-time` format in Params docs [airflow]

2024-01-23 Thread via GitHub
potiuk merged PR #36966: URL: https://github.com/apache/airflow/pull/36966 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Add missing hyphen in `date-time` format in Params docs [airflow]

2024-01-23 Thread via GitHub
boring-cyborg[bot] commented on PR #36966: URL: https://github.com/apache/airflow/pull/36966#issuecomment-1905843672 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

(airflow) branch main updated: Add missing hyphen in `data-time` format in Params docs (#36966)

2024-01-23 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0381f7f98e Add missing hyphen in `data-time` format

Re: [PR] Add data_interval_start and data_interval_end in dagrun create API endpoint [airflow]

2024-01-23 Thread via GitHub
karakanb commented on PR #36630: URL: https://github.com/apache/airflow/pull/36630#issuecomment-1905852345 @uranusjr addressed all of them, could you please take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Add dataset create endpoint [airflow]

2024-01-23 Thread via GitHub
edumuellerFSL commented on PR #36929: URL: https://github.com/apache/airflow/pull/36929#issuecomment-1905854281 > This wouldn't provide any functionality until there's a way to update a dataset via API, right? Hi @RNHTTR that's right, let me convert this to a draft for the moment. -

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-23 Thread via GitHub
Lee-W commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1905855808 Verified https://github.com/apache/airflow/pull/36892, https://github.com/apache/airflow/pull/36946 and https://github.com/apache/airflow/pull/36894. will continue work on the rest

Re: [I] Title: Error When Using Multiple DataSets as Triggers in Apache AirFlow [airflow]

2024-01-23 Thread via GitHub
SoftDed commented on issue #36837: URL: https://github.com/apache/airflow/issues/36837#issuecomment-1905869197 Another observation. I wrote my own xcom backend. If I use S3Hook and fetch data from Connection in the metadata database, everything breaks. However, if I retrieve d

Re: [PR] Add Dataset support to objectstorage [airflow]

2024-01-23 Thread via GitHub
bolkedebruin commented on code in PR #36947: URL: https://github.com/apache/airflow/pull/36947#discussion_r1463152779 ## airflow/io/path.py: ## @@ -129,7 +130,11 @@ def __new__( return _cls(_cls._format_parsed_parts(drv, root, parts, **other_kwargs), **new_kwargs

Re: [PR] Improve ExternalTaskSensor Async Implementation [airflow]

2024-01-23 Thread via GitHub
pankajastro commented on code in PR #36916: URL: https://github.com/apache/airflow/pull/36916#discussion_r1463237047 ## airflow/sensors/external_task.py: ## @@ -351,29 +348,30 @@ def execute(self, context: Context) -> None: super().execute(context) else:

Re: [PR] k8s spark operator compatibility [airflow]

2024-01-23 Thread via GitHub
assaf-xm commented on PR #36268: URL: https://github.com/apache/airflow/pull/36268#issuecomment-1906020596 @blcksrx is there any progress with the review of this PR? (this is a small PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[I] Custom headers for airflow webserver [airflow]

2024-01-23 Thread via GitHub
sustarun opened a new issue, #36971: URL: https://github.com/apache/airflow/issues/36971 ### Description I needed a way to add custom headers to the airflow webserver. I was able to do it but it required changes in the python code of airflow package which obviously is not the right p

Re: [I] Custom headers for airflow webserver [airflow]

2024-01-23 Thread via GitHub
boring-cyborg[bot] commented on issue #36971: URL: https://github.com/apache/airflow/issues/36971#issuecomment-1906041089 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Add Dataset support to objectstorage [airflow]

2024-01-23 Thread via GitHub
bolkedebruin merged PR #36947: URL: https://github.com/apache/airflow/pull/36947 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

(airflow) branch main updated: Make Datasets Pathlike (#36947)

2024-01-23 Thread bolke
This is an automated email from the ASF dual-hosted git repository. bolke pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f3b7cfc992 Make Datasets Pathlike (#36947) f3b7cfc99

Re: [PR] fix task_instance can not get task attribute correctly when turning on sentry [airflow]

2024-01-23 Thread via GitHub
jkramer-ginkgo commented on PR #36968: URL: https://github.com/apache/airflow/pull/36968#issuecomment-1906125382 I'm not totally convinced this is the best fix. Per my [comment](https://github.com/apache/airflow/issues/36957#issue-2094747829) I don't think `@Sentry.enrich_errors` makes sens

Re: [I] Airflow metrics with tags documentation need to be updated [airflow]

2024-01-23 Thread via GitHub
dirrao closed issue #36414: Airflow metrics with tags documentation need to be updated URL: https://github.com/apache/airflow/issues/36414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] heartbeat recovery message [airflow]

2024-01-23 Thread via GitHub
Bowrna commented on code in PR #34457: URL: https://github.com/apache/airflow/pull/34457#discussion_r1463345865 ## airflow/jobs/job.py: ## @@ -213,9 +218,12 @@ def heartbeat( session.merge(self) self.latest_heartbeat = timezone.utcnow()

Re: [PR] heartbeat recovery message [airflow]

2024-01-23 Thread via GitHub
Bowrna commented on code in PR #34457: URL: https://github.com/apache/airflow/pull/34457#discussion_r1463353839 ## airflow/jobs/job.py: ## @@ -126,7 +127,19 @@ def executor(self): def heartrate(self) -> float: return Job._heartrate(self.job_type) -def is_aliv

Re: [I] Be able to use loguru as logging/logger [airflow]

2024-01-23 Thread via GitHub
boring-cyborg[bot] commented on issue #36972: URL: https://github.com/apache/airflow/issues/36972#issuecomment-1906224263 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] Be able to use loguru as logging/logger [airflow]

2024-01-23 Thread via GitHub
lucasfcnunes opened a new issue, #36972: URL: https://github.com/apache/airflow/issues/36972 ### Description https://github.com/Delgan/loguru loguru is a popular third-party logging framework for Python on GitHub, with about 17.2k stars at the time of writing. It aims to eas

Re: [PR] The task is stuck in a queued state forever in case of pod launch errors [airflow]

2024-01-23 Thread via GitHub
dirrao commented on code in PR #36882: URL: https://github.com/apache/airflow/pull/36882#discussion_r1463426729 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py: ## @@ -434,9 +434,9 @@ def sync(self) -> None: ) self.

Re: [I] Status of testing Providers that were prepared on January 22, 2024 [airflow]

2024-01-23 Thread via GitHub
potiuk commented on issue #36948: URL: https://github.com/apache/airflow/issues/36948#issuecomment-1906259070 All my changes work ! :tada: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[I] add support for dataset triggers with an or condition [airflow]

2024-01-23 Thread via GitHub
alonahmias opened a new issue, #36973: URL: https://github.com/apache/airflow/issues/36973 ### Description I want to be able to schedule a dag with an or condition between the datasets For example i have the following datasets: dataset_a dataset_b I want my dag to be schedu

Re: [I] add support for dataset triggers with an or condition [airflow]

2024-01-23 Thread via GitHub
boring-cyborg[bot] commented on issue #36973: URL: https://github.com/apache/airflow/issues/36973#issuecomment-1906264245 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] The task is stuck in a queued state forever in case of pod launch errors [airflow]

2024-01-23 Thread via GitHub
shohamy7 commented on code in PR #36882: URL: https://github.com/apache/airflow/pull/36882#discussion_r1463452920 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py: ## @@ -434,9 +434,9 @@ def sync(self) -> None: ) sel

Re: [I] Tasks taking too long time after 2.7.0 Airflow update [airflow]

2024-01-23 Thread via GitHub
BenoCharlo commented on issue #33688: URL: https://github.com/apache/airflow/issues/33688#issuecomment-1906317844 Hello everyone, just got to this thread because I'm facing same issue on AF 2.7.2. I'm using this version because this is the latest version available on AWS MWAA. I'm experinci

[PR] Add scopes into a GCP token [airflow]

2024-01-23 Thread via GitHub
moiseenkov opened a new pull request, #36974: URL: https://github.com/apache/airflow/pull/36974 This PR provides a small fix that adds missing scopes into a token. Relates to #34727 and continues #36849, #36903. In our use case we have a DAG (see below) with `BigQueryInsertJobOp

Re: [PR] Add scopes into a GCP token [airflow]

2024-01-23 Thread via GitHub
VladaZakharova commented on PR #36974: URL: https://github.com/apache/airflow/pull/36974#issuecomment-1906348282 Hi @m1racoli @Lee-W @pankajastro ! Can you please check changes from this PR? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message

Re: [I] Tasks taking too long time after 2.7.0 Airflow update [airflow]

2024-01-23 Thread via GitHub
pankajkoti commented on issue #33688: URL: https://github.com/apache/airflow/issues/33688#issuecomment-1906354803 @BenoCharlo Do you have a local Airflow env setup? If yes, could you try running your DAGs locally with Airflow 2.7.2 vs your previous version and see if you are able to reprodu

  1   2   3   >