Re: [PR] Add `impersonation_scopes` to BigQuery [airflow]

2024-03-29 Thread via GitHub
ying-w commented on PR #38169: URL: https://github.com/apache/airflow/pull/38169#issuecomment-2027931420 i think i fixed the test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Remove select_column option in TaskInstance.get_task_instance [airflow]

2024-03-29 Thread via GitHub
dstandish commented on PR #38571: URL: https://github.com/apache/airflow/pull/38571#issuecomment-2027918211 Yeah there was some weird error related to a test of sensor. I don’t think it’s really an issue probably something with the test but I was still working through it -- This is an

Re: [PR] Fix ti.start_date is set even in pre-running state by ti.set_state [airflow]

2024-03-29 Thread via GitHub
kzosabe commented on code in PR #38631: URL: https://github.com/apache/airflow/pull/38631#discussion_r1545086130 ## tests/models/test_taskinstance.py: ## @@ -1511,6 +1511,24 @@ def test_respects_prev_dagrun_dep(self, create_task_instance): ): assert

[I] States of the time field corresponding to ti.state is not clear [airflow]

2024-03-29 Thread via GitHub
kzosabe opened a new issue, #38632: URL: https://github.com/apache/airflow/issues/38632 ### What do you see as an issue? There was a problem with the emitting of statistics logs regarding execution time introduced in https://github.com/apache/airflow/pull/30612 . Problem

[PR] Fix ti.start_date is set even in pre-running state by ti.set_state [airflow]

2024-03-29 Thread via GitHub
kzosabe opened a new pull request, #38631: URL: https://github.com/apache/airflow/pull/38631 related: https://github.com/apache/airflow/pull/34771#discussion_r1542257954 From the above discussion of PR, ti.start_date is normally set when ti transitions to the RUNNING

Re: [PR] Use explicit and easier to use runs-on approach for CI workflows [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38630: URL: https://github.com/apache/airflow/pull/38630#issuecomment-2027862488 Closing - all looks good here. The PR to merge is #38601 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Use explicit and easier to use runs-on approach for CI workflows [airflow]

2024-03-29 Thread via GitHub
potiuk closed pull request #38630: Use explicit and easier to use runs-on approach for CI workflows URL: https://github.com/apache/airflow/pull/38630 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Use explicit and easier to use runs-on approach for CI workflows [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38601: URL: https://github.com/apache/airflow/pull/38601#issuecomment-2027862357 All good - tested all cases: regular builds, public runner builds, maintainer build from PR in apache repo, canary build - this one is ready to merge I think. Looking forward to reviews

Re: [PR] Fix TaskHandlerWithCustomFormatter now adds prefix only once [airflow]

2024-03-29 Thread via GitHub
Lee-W commented on code in PR #38502: URL: https://github.com/apache/airflow/pull/38502#discussion_r1544998812 ## airflow/utils/log/task_handler_with_custom_formatter.py: ## @@ -45,7 +45,7 @@ def set_context(self, ti) -> None: :param ti: :return: """

Re: [PR] Add k8s recommended labels [airflow]

2024-03-29 Thread via GitHub
github-actions[bot] commented on PR #34735: URL: https://github.com/apache/airflow/pull/34735#issuecomment-2027836374 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] bump open api image to v7.3.0 [airflow]

2024-03-29 Thread via GitHub
github-actions[bot] commented on PR #34805: URL: https://github.com/apache/airflow/pull/34805#issuecomment-2027836367 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] feat(helm): Add `enable` property to metric-exporter sidecar [airflow]

2024-03-29 Thread via GitHub
github-actions[bot] commented on PR #35633: URL: https://github.com/apache/airflow/pull/35633#issuecomment-2027836355 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [I] KubernetesPodOperator prints empty log lines without respecting the log format [airflow]

2024-03-29 Thread via GitHub
github-actions[bot] commented on issue #36571: URL: https://github.com/apache/airflow/issues/36571#issuecomment-2027836346 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

(airflow) branch main updated (a24860ab82 -> 50a4c951fc)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from a24860ab82 Use generic utils for cleanup db and operate with temp objects in TestXcomObjectStoreBackend (#37166)

Re: [PR] Raise an error on Airflow Deprecation warnings in tests [airflow]

2024-03-29 Thread via GitHub
potiuk merged PR #38504: URL: https://github.com/apache/airflow/pull/38504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Use generic utils for cleanup db and operate with temp objects in TestXcomObjectStoreBackend (#37166)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new a24860ab82 Use generic utils for cleanup db and

Re: [PR] Use generic utils for cleanup db and operate with temp objects in TestXcomObjectStoreBackend [airflow]

2024-03-29 Thread via GitHub
potiuk merged PR #37166: URL: https://github.com/apache/airflow/pull/37166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Remove select_column option in TaskInstance.get_task_instance [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38571: URL: https://github.com/apache/airflow/pull/38571#issuecomment-2027829935 Those test failure looked like abrupt failure of the docker engine in the middle of testing. I re-run just the tests and if it appears again then we have something interesting here.

Re: [PR] Avoid to use `functools.lru_cache` in class methods in dev [airflow]

2024-03-29 Thread via GitHub
potiuk merged PR #38627: URL: https://github.com/apache/airflow/pull/38627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Avoid to use `functools.lru_cache` in class methods in dev (#38627)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new a4ee43024a Avoid to use `functools.lru_cache` in

Re: [PR] Use explicit and easier to use runs-on approach for CI workflows [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38601: URL: https://github.com/apache/airflow/pull/38601#issuecomment-2027813850 Running it with `use public runners` to check if everything is fine for a `regular` run - also running it as `canary` build in #38630 -- This is an automated message from the Apache

[PR] Use explicit and easier to use runs-on approach for CI workflows [airflow]

2024-03-29 Thread via GitHub
potiuk opened a new pull request, #38630: URL: https://github.com/apache/airflow/pull/38630 Depending on selective checks, but also on the job executed, we choose whether to run job on public runners or self-hosted runners. So far the set of labels to select the runners were passed in a

(airflow) 01/01: Use explicit and easier to use runs-on approach for CI workflows

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch consistent-approach-for-runs-on-usage in repository https://gitbox.apache.org/repos/asf/airflow.git commit 8d2ae089a243b18510ab4214aa890626addc2fed Author: Jarek Potiuk AuthorDate: Thu Mar 28

(airflow) branch consistent-approach-for-runs-on-usage updated (6ad6c1b4a2 -> 8d2ae089a2)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch consistent-approach-for-runs-on-usage in repository https://gitbox.apache.org/repos/asf/airflow.git discard 6ad6c1b4a2 Use explicit and easier to use runs-on approach for CI workflows add

Re: [PR] Use explicit and easier to use runs-on approach for CI workflows [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38601: URL: https://github.com/apache/airflow/pull/38601#issuecomment-2027810175 Green :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Postgre hook 5.5.1 not writing executed sql statements in task logs [airflow]

2024-03-29 Thread via GitHub
potiuk closed issue #38623: Postgre hook 5.5.1 not writing executed sql statements in task logs URL: https://github.com/apache/airflow/issues/38623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Fix `total_entries` count on the event logs endpoint [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38625: URL: https://github.com/apache/airflow/pull/38625#issuecomment-2027803387 nice catch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] wip: Avoid SQLAchemy Query API usage in tests [airflow]

2024-03-29 Thread via GitHub
Taragolis closed pull request #37242: wip: Avoid SQLAchemy Query API usage in tests URL: https://github.com/apache/airflow/pull/37242 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] Fix inital value for _batch_id attribute in livy operator [airflow]

2024-03-29 Thread via GitHub
alizademhdi opened a new pull request, #38628: URL: https://github.com/apache/airflow/pull/38628 ### Related issue #37898 ### Explain when the livy operator is rewaked up after trigger is time out, the _batch_id is not initialized, which cause it to raise this exeption:

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-03-29 Thread via GitHub
Taragolis commented on PR #37757: URL: https://github.com/apache/airflow/pull/37757#issuecomment-2027785730 WeightRule is enum, and members of enum are singletons, so it is pretty safe to use it there, but in the other cases it still looks like a problem -- This is an automated message

(airflow) branch constraints-main updated: Updating constraints. Github run id:8485402497

2024-03-29 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 5b2b01ee5e Updating

Re: [I] Triggering a DAG Run from the DAG Graph View does not redirect to the triggered DAG Run page [airflow]

2024-03-29 Thread via GitHub
jscheffl commented on issue #37935: URL: https://github.com/apache/airflow/issues/37935#issuecomment-2027774422 I tested this to fix it finally in Airflow 2.9.0 as upcoming release. I needed to realize that due to changes in UI structure and rework of the layout the problem has been

Re: [I] Triggering a DAG Run from the DAG Graph View does not redirect to the triggered DAG Run page [airflow]

2024-03-29 Thread via GitHub
jscheffl closed issue #37935: Triggering a DAG Run from the DAG Graph View does not redirect to the triggered DAG Run page URL: https://github.com/apache/airflow/issues/37935 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-03-29 Thread via GitHub
Taragolis commented on PR #37757: URL: https://github.com/apache/airflow/pull/37757#issuecomment-2027769895 @potiuk I've also found that `ruff` do not detect compat decorator `airflow.compat.functools.cache` and we use it in some methods places into the core, which might cause memory

Re: [I] Postgre hook 5.5.1 not writing executed sql statements in task logs [airflow]

2024-03-29 Thread via GitHub
eladkal commented on issue #38623: URL: https://github.com/apache/airflow/issues/38623#issuecomment-2027766822 You are asking for assistance to do something that is highly discouraged (running hooks and make DB calls as top level code). I don't think invistigating it is a good use of

[PR] Avoid to use `functools.lru_cache` in class methods in dev [airflow]

2024-03-29 Thread via GitHub
Taragolis opened a new pull request, #38627: URL: https://github.com/apache/airflow/pull/38627 Part of the https://github.com/apache/airflow/pull/37757, without use [`methodtools`](https://github.com/youknowone/methodtools) --- **^ Add meaningful description

Re: [PR] Use generic utils for cleanup db and operate with temp objects in TestXcomObjectStoreBackend [airflow]

2024-03-29 Thread via GitHub
Taragolis commented on PR #37166: URL: https://github.com/apache/airflow/pull/37166#issuecomment-2027724000 This PR better merge (and possibly rebase) after https://github.com/apache/airflow/pull/38608 -- This is an automated message from the Apache Git Service. To respond to the

(airflow) branch main updated (6e27ba99af -> ff491a6732)

2024-03-29 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 6e27ba99af Conditionally skip few more tests for "Pydantic Removed" test (#38620) add ff491a6732 Configure

Re: [PR] Configure retention policy for temporary files which created by `tmp_path` fixture [airflow]

2024-03-29 Thread via GitHub
Taragolis merged PR #38613: URL: https://github.com/apache/airflow/pull/38613 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Use generic utils for cleanup db and operate with temp objects in TestXcomObjectStoreBackend [airflow]

2024-03-29 Thread via GitHub
Taragolis commented on code in PR #37166: URL: https://github.com/apache/airflow/pull/37166#discussion_r1544845557 ## tests/providers/common/io/xcom/test_backend.py: ## @@ -86,28 +87,19 @@ def task_instance(task_instance_factory): class TestXcomObjectStoreBackend: -

Re: [PR] Remove select_column option in TaskInstance.get_task_instance [airflow]

2024-03-29 Thread via GitHub
jedcunningham commented on PR #38571: URL: https://github.com/apache/airflow/pull/38571#issuecomment-2027702306 > I think there were still some tests failing @jedcunningham The 1 I looked at didn't seem obviously related, in the 10 seconds I looked at it. So I figured a fresh run

Re: [PR] Remove select_column option in TaskInstance.get_task_instance [airflow]

2024-03-29 Thread via GitHub
dstandish commented on PR #38571: URL: https://github.com/apache/airflow/pull/38571#issuecomment-2027699902 I think there were still some tests failing @jedcunningham -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Support multiple Docker hosts in `docker_url` attribute. [airflow]

2024-03-29 Thread via GitHub
oboki commented on code in PR #38466: URL: https://github.com/apache/airflow/pull/38466#discussion_r1544832963 ## airflow/providers/docker/hooks/docker.py: ## @@ -146,15 +151,23 @@ def construct_tls_config( @cached_property def api_client(self) -> APIClient:

Re: [PR] Edited timetable docs [airflow]

2024-03-29 Thread via GitHub
collinmcnulty commented on code in PR #38505: URL: https://github.com/apache/airflow/pull/38505#discussion_r1544829258 ## docs/apache-airflow/authoring-and-scheduling/timetable.rst: ## @@ -216,46 +215,47 @@ Timetables comparisons Differences between the two cron timetables

Re: [I] `BigQueryInsertJobOperator` sometimes fails to acquire impersonated credentials when in deferred mode [airflow]

2024-03-29 Thread via GitHub
collinmcnulty commented on issue #38532: URL: https://github.com/apache/airflow/issues/38532#issuecomment-2027685514 As a workaround, setting `force_rerun=False` and setting retries should allow another retry to pick up and monitor the original job instead of starting a new one. -- This

Re: [PR] Mark more fields on API as dump-only [airflow]

2024-03-29 Thread via GitHub
jedcunningham commented on code in PR #38616: URL: https://github.com/apache/airflow/pull/38616#discussion_r1544818510 ## airflow/api_connexion/schemas/dag_schema.py: ## @@ -96,21 +96,21 @@ class DAGDetailSchema(DAGSchema): """DAG details.""" owners =

[PR] Rename `allowed_filter_attrs` to `allowed_sort_attrs` [airflow]

2024-03-29 Thread via GitHub
jedcunningham opened a new pull request, #38626: URL: https://github.com/apache/airflow/pull/38626 These are the attrs we allow sorting on, not filtering on. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] Fix `total_entries` count on the event logs endpoint [airflow]

2024-03-29 Thread via GitHub
jedcunningham opened a new pull request, #38625: URL: https://github.com/apache/airflow/pull/38625 The `total_entries` count should reflect how many log entries match the filters provided, not simply how many rows are in the table total. I took the opportunity to do a little cleanup

Re: [PR] Add exclude/include events filters to audit log [airflow]

2024-03-29 Thread via GitHub
jedcunningham commented on PR #38506: URL: https://github.com/apache/airflow/pull/38506#issuecomment-2027547597 Found a few issues while giving this a spin. "No options" - when you first enter into the filter form field, it says there are no options. Even when you start typing say

Re: [PR] Add exclude/include events filters to audit log [airflow]

2024-03-29 Thread via GitHub
jedcunningham commented on code in PR #38506: URL: https://github.com/apache/airflow/pull/38506#discussion_r1544728878 ## airflow/www/views.py: ## @@ -2782,6 +2782,9 @@ def grid(self, dag_id: str, session: Session = NEW_SESSION):

[PR] Disable support of a legacy `LocalPath` in favor of stdlib `pathlib.Path` [airflow]

2024-03-29 Thread via GitHub
Taragolis opened a new pull request, #38624: URL: https://github.com/apache/airflow/pull/38624 It is already disabled into the other tests, this is a legacy feature which might be removed into the future version of pytest --- **^ Add meaningful

(airflow) branch main updated (75264404cf -> 6e27ba99af)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 75264404cf Remove image tag from Helm test name (#38621) add 6e27ba99af Conditionally skip few more tests for

Re: [PR] Conditionally skip few more tests for "Pydantic Removed" test [airflow]

2024-03-29 Thread via GitHub
potiuk merged PR #38620: URL: https://github.com/apache/airflow/pull/38620 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[I] Postgre hook 5.5.1 not writing executed sql statements in task logs [airflow]

2024-03-29 Thread via GitHub
Mrhacktivist opened a new issue, #38623: URL: https://github.com/apache/airflow/issues/38623 ### Apache Airflow Provider(s) postgres ### Versions of Apache Airflow Providers 5.5.1 ### Apache Airflow version 2.6.3 ### Operating System amazon

(airflow) 01/01: Use explicit and easier to use runs-on approach for CI workflows

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch consistent-approach-for-runs-on-usage in repository https://gitbox.apache.org/repos/asf/airflow.git commit 6ad6c1b4a276caae4bed4e56c4381cd2aaebfdf7 Author: Jarek Potiuk AuthorDate: Thu Mar 28

(airflow) branch consistent-approach-for-runs-on-usage updated (cad758764f -> 6ad6c1b4a2)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch consistent-approach-for-runs-on-usage in repository https://gitbox.apache.org/repos/asf/airflow.git omit cad758764f Use explicit and easier to use runs-on approach for CI workflows add

Re: [PR] Refactor DAG tabs navigation [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38592: URL: https://github.com/apache/airflow/pull/38592#issuecomment-2027354786 Agree. Love that. Let's make it a highlight of 2.9.0. And the changes proposed by Jens are good. There is one small caveat of that - multiple rows of tabs are quite confusing.

Re: [PR] Conditionally skip few more tests for "Pydantic Removed" test [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38620: URL: https://github.com/apache/airflow/pull/38620#issuecomment-2027341415 Should fix main failing with: https://github.com/apache/airflow/assets/595491/d282e4bc-6805-4453-8b61-ced6e5bc0fec;> -- This is an automated message from the Apache Git

(airflow) branch main updated: Remove image tag from Helm test name (#38621)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 75264404cf Remove image tag from Helm test name

Re: [PR] Remove image tag from Helm test name [airflow]

2024-03-29 Thread via GitHub
potiuk merged PR #38621: URL: https://github.com/apache/airflow/pull/38621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Remove image tag from Helm test name [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38621: URL: https://github.com/apache/airflow/pull/38621#issuecomment-2027340130 Avoids: https://github.com/apache/airflow/assets/595491/350ee9a6-4e61-4e11-9e29-946d3c4a30b5;> and this :

[PR] Remove image tag from Helm test name [airflow]

2024-03-29 Thread via GitHub
potiuk opened a new pull request, #38621: URL: https://github.com/apache/airflow/pull/38621 The image tag makes no difference. While we are using the image to run Helm tests (for now) the fact that we are using specific image shoudl not be part of the job name (it's printed in the steps

[PR] Conditionally skip few more tests for "Pydantic Removed" test [airflow]

2024-03-29 Thread via GitHub
potiuk opened a new pull request, #38620: URL: https://github.com/apache/airflow/pull/38620 Turns out that we need to conditionally skip a few more tests in order to make "Pydantic Removed" tests happy after #38617 --- **^ Add meaningful description

Re: [PR] Hide internals of some groupped workflows [airflow]

2024-03-29 Thread via GitHub
potiuk merged PR #38604: URL: https://github.com/apache/airflow/pull/38604 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated (88a284dd59 -> 3f4872beea)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 88a284dd59 Do not assume that `sys.path` always changed in `pytest` (#38612) add 3f4872beea Hide internals of

Re: [PR] Hide internals of some groupped workflows [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38604: URL: https://github.com/apache/airflow/pull/38604#issuecomment-2027327357 Merging. the "No pydantic" tests in main will be handled separately (need to skip some more tests in NoPydantic case). -- This is an automated message from the Apache Git Service. To

Re: [I] Pgbouncer not set properly upon using data.metadataSecretName [airflow]

2024-03-29 Thread via GitHub
VladimirYushkevich commented on issue #33924: URL: https://github.com/apache/airflow/issues/33924#issuecomment-2027324190 We experienced a similar issue. I don't mind fixing it, but I need to clarify a few things first: * `data.metadataSecretName` is not for `pgbouncer`; it is for other

Re: [PR] [Experiment] Use `importlib` import mode for `pytest` [airflow]

2024-03-29 Thread via GitHub
Taragolis commented on code in PR #38594: URL: https://github.com/apache/airflow/pull/38594#discussion_r1544572827 ## pyproject.toml: ## @@ -445,7 +445,15 @@ fixture-parentheses = false # * Disable `flaky` plugin for pytest. This plugin conflicts with `rerunfailures` because

(airflow) branch pytest-import-mode-importlib updated (586451344a -> 5be23afeab)

2024-03-29 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch pytest-import-mode-importlib in repository https://gitbox.apache.org/repos/asf/airflow.git discard 586451344a [Experiment] Use `importlib` import mode for `pytest` add a9a6976dd2 docs:

Re: [PR] Raise an error on Airflow Deprecation warnings in tests [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38504: URL: https://github.com/apache/airflow/pull/38504#issuecomment-2027274595 Added it to the https://github.com/apache/airflow/pull/38604 - once we merge it, `full tests needed` should also trigger special tests. -- This is an automated message from the

Re: [PR] Hide internals of some groupped workflows [airflow]

2024-03-29 Thread via GitHub
potiuk commented on code in PR #38604: URL: https://github.com/apache/airflow/pull/38604#discussion_r1544534177 ## .github/workflows/special-tests.yml: ## @@ -134,7 +134,7 @@ jobs: contents: read packages: read secrets: inherit -if: inputs.canary-run ==

Re: [PR] Raise an error on Airflow Deprecation warnings in tests [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38504: URL: https://github.com/apache/airflow/pull/38504#issuecomment-2027262468 Just realized. Maybe we should add a special if for just "full tests needed" - that would be also a better criteria/way to trigger the special tests. -- This is an automated message

Re: [PR] Raise an error on Airflow Deprecation warnings in tests [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38504: URL: https://github.com/apache/airflow/pull/38504#issuecomment-2027258022 BTW. This one https://github.com/apache/airflow/pull/38604 - will turn such "skipped" special tests into a single line in output, rather than showing all the skipped sub-workflows. --

Re: [PR] Raise an error on Airflow Deprecation warnings in tests [airflow]

2024-03-29 Thread via GitHub
potiuk closed pull request #38504: Raise an error on Airflow Deprecation warnings in tests URL: https://github.com/apache/airflow/pull/38504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Raise an error on Airflow Deprecation warnings in tests [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38504: URL: https://github.com/apache/airflow/pull/38504#issuecomment-2027255850 so way to trigger the special tests is to add "upgrade to newer dependencies" . Let me do it now. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Raise an error on Airflow Deprecation warnings in tests [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38504: URL: https://github.com/apache/airflow/pull/38504#issuecomment-2027253984 > One thing what confuse me, that why special tests skipped 樂 We are currently skipping them for regular runs - in one of the recent optimization I changed them to be run only:

Re: [PR] Hide internals of some groupped workflows [airflow]

2024-03-29 Thread via GitHub
aritra24 commented on code in PR #38604: URL: https://github.com/apache/airflow/pull/38604#discussion_r1544497045 ## .github/workflows/special-tests.yml: ## @@ -134,7 +134,7 @@ jobs: contents: read packages: read secrets: inherit -if: inputs.canary-run ==

Re: [PR] Use explicit and easier to use runs-on approach for CI workflows [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38601: URL: https://github.com/apache/airflow/pull/38601#issuecomment-2027245024 This one also looks good-to-go with much easier way to manage what runners are used (and I am going to have some follow-ups once this is better organized). -- This is an automated

Re: [PR] Hide internals of some groupped workflows [airflow]

2024-03-29 Thread via GitHub
potiuk commented on PR #38604: URL: https://github.com/apache/airflow/pull/38604#issuecomment-2027243784 Should be good to go - seems to pass all the variants -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Raise an error on Airflow Deprecation warnings in tests [airflow]

2024-03-29 Thread via GitHub
Taragolis commented on PR #38504: URL: https://github.com/apache/airflow/pull/38504#issuecomment-2027242847 One thing what confuse me, that why special tests skipped 樂 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Use explicit and easier to use runs-on approach for CI workflows [airflow]

2024-03-29 Thread via GitHub
potiuk closed pull request #38605: Use explicit and easier to use runs-on approach for CI workflows URL: https://github.com/apache/airflow/pull/38605 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Do not assume that `sys.path` always changed in `pytest` [airflow]

2024-03-29 Thread via GitHub
potiuk merged PR #38612: URL: https://github.com/apache/airflow/pull/38612 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch consistent-approach-for-runs-on-usage updated (b8056fa284 -> cad758764f)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch consistent-approach-for-runs-on-usage in repository https://gitbox.apache.org/repos/asf/airflow.git discard b8056fa284 Use explicit and easier to use runs-on approach for CI workflows add

(airflow) branch main updated: Do not assume that `sys.path` always changed in `pytest` (#38612)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 88a284dd59 Do not assume that `sys.path` always

Re: [PR] Introduce Amazon Bedrock service [airflow]

2024-03-29 Thread via GitHub
Taragolis commented on code in PR #38602: URL: https://github.com/apache/airflow/pull/38602#discussion_r1544489075 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [PR] add templated field support for extra containers [airflow]

2024-03-29 Thread via GitHub
Avihais12344 commented on code in PR #38510: URL: https://github.com/apache/airflow/pull/38510#discussion_r1544482974 ## chart/templates/dag-processor/dag-processor-deployment.yaml: ## @@ -227,7 +227,7 @@ spec: {{- end }} {{- end }} {{- if

Re: [PR] Introduce Amazon Bedrock service [airflow]

2024-03-29 Thread via GitHub
eladkal commented on code in PR #38602: URL: https://github.com/apache/airflow/pull/38602#discussion_r1544481253 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [PR] Introduce Amazon Bedrock service [airflow]

2024-03-29 Thread via GitHub
Taragolis commented on code in PR #38602: URL: https://github.com/apache/airflow/pull/38602#discussion_r1544478951 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

(airflow) branch main updated: Remove google-cloud-aiplatform when tests without pydantic are run (#38617)

2024-03-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 69df34e52d Remove google-cloud-aiplatform when

Re: [PR] Remove google-cloud-aiplatform when tests without pydantic are run [airflow]

2024-03-29 Thread via GitHub
potiuk merged PR #38617: URL: https://github.com/apache/airflow/pull/38617 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Introduce Amazon Bedrock service [airflow]

2024-03-29 Thread via GitHub
eladkal commented on code in PR #38602: URL: https://github.com/apache/airflow/pull/38602#discussion_r1544469400 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

(airflow) branch main updated: Add padding to bottom of log block. (#38610)

2024-03-29 Thread jscheffl
This is an automated email from the ASF dual-hosted git repository. jscheffl pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1d8413ec4f Add padding to bottom of log block.

Re: [PR] Add padding to bottom of log block. [airflow]

2024-03-29 Thread via GitHub
jscheffl merged PR #38610: URL: https://github.com/apache/airflow/pull/38610 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] add repair_run supporrt to DatabricksRunNowOperator in deferrable mode [airflow]

2024-03-29 Thread via GitHub
Lee-W opened a new pull request, #38619: URL: https://github.com/apache/airflow/pull/38619 … --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Do not assume that `sys.path` always changed in `pytest` [airflow]

2024-03-29 Thread via GitHub
Taragolis commented on code in PR #38612: URL: https://github.com/apache/airflow/pull/38612#discussion_r1544422003 ## kubernetes_tests/conftest.py: ## @@ -16,11 +16,25 @@ # under the License. from __future__ import annotations +from pathlib import Path + import pytest

Re: [PR] Introduce Amazon Bedrock service [airflow]

2024-03-29 Thread via GitHub
Taragolis commented on code in PR #38602: URL: https://github.com/apache/airflow/pull/38602#discussion_r1544400797 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [PR] Refactor DAG tabs navigation [airflow]

2024-03-29 Thread via GitHub
jscheffl commented on PR #38592: URL: https://github.com/apache/airflow/pull/38592#issuecomment-2027093889 My Feedback: 1. @ephraimbuddy even if a late arrival, can we get this into 2.9? 2. The DAG/Run/Task Details are the most common and standard informations. I'd propose to put

Re: [PR] Do not assume that `sys.path` always changed in `pytest` [airflow]

2024-03-29 Thread via GitHub
potiuk commented on code in PR #38612: URL: https://github.com/apache/airflow/pull/38612#discussion_r1544354786 ## kubernetes_tests/conftest.py: ## @@ -16,11 +16,25 @@ # under the License. from __future__ import annotations +from pathlib import Path + import pytest

Re: [PR] Remove google-cloud-aiplatform when tests without pydantic are run [airflow]

2024-03-29 Thread via GitHub
potiuk closed pull request #38617: Remove google-cloud-aiplatform when tests without pydantic are run URL: https://github.com/apache/airflow/pull/38617 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] bugfix: properly serialize nested attrs classes [airflow]

2024-03-29 Thread via GitHub
tomrutter commented on PR #38591: URL: https://github.com/apache/airflow/pull/38591#issuecomment-2027063267 Thanks for review and merge! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

  1   2   >