Re: [I] Status of testing Providers that were prepared on July 09, 2024 [airflow]

2024-07-10 Thread via GitHub
dirrao commented on issue #40661: URL: https://github.com/apache/airflow/issues/40661#issuecomment-2219754355 I have verified [#40629)](https://github.com/apache/airflow/pull/40629) and [#40506)](https://github.com/apache/airflow/pull/40506). They are working as expected. -- This is an

Re: [I] Allow config a static ip for the pgbouncer clusterip service [airflow]

2024-07-10 Thread via GitHub
fbertos commented on issue #40676: URL: https://github.com/apache/airflow/issues/40676#issuecomment-2219804772 Thank you @potiuk. I will do it as soon as I can, I promise. Thank you for all your work and time. -- This is an automated message from the Apache Git Service. To respond to the

[PR] openlineage: remove deprecated parentRun facet key [airflow]

2024-07-10 Thread via GitHub
kacpermuda opened a new pull request, #40681: URL: https://github.com/apache/airflow/pull/40681 This PR removes deprecated `parentRun` key for ParentRunFacet, leaving only `parent` key. related: https://github.com/OpenLineage/OpenLineage/pull/2660 --- **^ Add

[PR] openlineage: add deferrable information to task info in airflowRunFacet [airflow]

2024-07-10 Thread via GitHub
kacpermuda opened a new pull request, #40682: URL: https://github.com/apache/airflow/pull/40682 After #38264 the deferrable information is missing from taskInfo, re-adding it. --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](ht

Re: [I] Status of testing Providers that were prepared on July 09, 2024 [airflow]

2024-07-10 Thread via GitHub
BenediktRamsauer commented on issue #40661: URL: https://github.com/apache/airflow/issues/40661#issuecomment-2219938168 I have tested [#40642](https://github.com/apache/airflow/pull/40642) and it is working. -- This is an automated message from the Apache Git Service. To respond to the me

[PR] Limit pgvector version to 0.2.5 to avoid recent issue in latest releases(0.3.0) [airflow]

2024-07-10 Thread via GitHub
vatsrahul1001 opened a new pull request, #40683: URL: https://github.com/apache/airflow/pull/40683 After the 0.3.0 release, we have encountered an issue where the register_vector function throws an error if our PostgreSQL database only has the vector extension installed, and not halfvec or

[I] AwsAuthManager seems to be incompatible with Airflow 2.8 [airflow]

2024-07-10 Thread via GitHub
potiuk opened a new issue, #40684: URL: https://github.com/apache/airflow/issues/40684 ### Body As of recently the compatibility tests for Airlfow 2.8.2 started to fail for AwsAuthManager and it looks like the Auth Manager is not compatible with 2.8 BaseAuthManager. Example fa

Re: [I] Status of testing Providers that were prepared on July 09, 2024 [airflow]

2024-07-10 Thread via GitHub
potiuk commented on issue #40661: URL: https://github.com/apache/airflow/issues/40661#issuecomment-2220043976 I am not sure if that qualifies for RC2, but currently in main AWS auth manager fails compatibility test for 2.8 - i described it in https://github.com/apache/airflow/issues/40684 -

Re: [PR] openlineage: remove deprecated parentRun facet key [airflow]

2024-07-10 Thread via GitHub
potiuk commented on PR #40681: URL: https://github.com/apache/airflow/pull/40681#issuecomment-2220064696 The failure is unrelated and will be handled separately https://github.com/apache/airflow/issues/40684 -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] openlineage: add deferrable information to task info in airflowRunFacet [airflow]

2024-07-10 Thread via GitHub
potiuk merged PR #40682: URL: https://github.com/apache/airflow/pull/40682 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] openlineage: remove deprecated parentRun facet key [airflow]

2024-07-10 Thread via GitHub
potiuk merged PR #40681: URL: https://github.com/apache/airflow/pull/40681 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: openlineage: remove deprecated parentRun facet key (#40681)

2024-07-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 14b075b4c9 openlineage: remove deprecated parentRun

(airflow) branch main updated: openlineage: add deferrable information to task info in airflow run facet (#40682)

2024-07-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 165b91014f openlineage: add deferrable information

Re: [PR] Limit pgvector version to 0.2.5 to avoid recent issue in latest releases(0.3.0) [airflow]

2024-07-10 Thread via GitHub
potiuk commented on code in PR #40683: URL: https://github.com/apache/airflow/pull/40683#discussion_r1671984187 ## airflow/providers/pgvector/provider.yaml: ## @@ -43,7 +43,9 @@ integrations: dependencies: - apache-airflow>=2.7.0 - apache-airflow-providers-postgres>=5.7.1

[PR] Fix Custom Training Job operators to accept results without a model in def mode [airflow]

2024-07-10 Thread via GitHub
e-galan opened a new pull request, #40685: URL: https://github.com/apache/airflow/pull/40685 This PR should address #40476 - Fix the deferrable mode for Custom Training Job operators so that they could process training pipeline results that do not produce a managed mod

Re: [I] Vertex AI job operators in deferrable mode assume job uses Managed Model [airflow]

2024-07-10 Thread via GitHub
e-galan commented on issue #40476: URL: https://github.com/apache/airflow/issues/40476#issuecomment-2220122193 Hi @vignesh-sc , I have just submitted a community PR to address the issue. You are welcome to check it. -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] Fix behavior for reattach_state parameter in BigQueryInsertJobOperator [airflow]

2024-07-10 Thread via GitHub
VladaZakharova commented on PR #40664: URL: https://github.com/apache/airflow/pull/40664#issuecomment-2220122200 Hi @potiuk @Lee-W ! Can you please check changes in this PR? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] [AIP-62] Translate AIP-60 URI to OpenLineage [airflow]

2024-07-10 Thread via GitHub
kacpermuda commented on PR #40173: URL: https://github.com/apache/airflow/pull/40173#issuecomment-2220124220 I believe this PR is ready to be reviewed, however it should probably NOT be merged before #40335 gets merged. -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Fix behavior for reattach_state parameter in BigQueryInsertJobOperator [airflow]

2024-07-10 Thread via GitHub
potiuk merged PR #40664: URL: https://github.com/apache/airflow/pull/40664 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Fix behavior for reattach_state parameter in BigQueryInsertJobOperator (#40664)

2024-07-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c29eeb18ec Fix behavior for reattach_state paramete

Re: [PR] updating pgvector to !=0.3.0 to avoid recent issue in latest releases(0.3.0) [airflow]

2024-07-10 Thread via GitHub
potiuk commented on code in PR #40683: URL: https://github.com/apache/airflow/pull/40683#discussion_r1672006147 ## airflow/providers/pgvector/provider.yaml: ## @@ -43,7 +43,9 @@ integrations: dependencies: - apache-airflow>=2.7.0 - apache-airflow-providers-postgres>=5.7.1

[PR] feat(aws): provide the context to check_fn in S3 sensor [airflow]

2024-07-10 Thread via GitHub
hussein-awala opened a new pull request, #40686: URL: https://github.com/apache/airflow/pull/40686 This change will help to compare context variables and file metadata attributes, for example, compare the run logical date and the file's last modified date. -- This is an automated message

Re: [PR] feat(aws): provide the context to check_fn in S3 sensor [airflow]

2024-07-10 Thread via GitHub
hussein-awala commented on PR #40686: URL: https://github.com/apache/airflow/pull/40686#issuecomment-2220263033 The failed test is not related to this change: ``` FAILED tests/providers/amazon/aws/auth_manager/test_aws_auth_manager.py::TestAwsAuthManager::test_aws_auth_manager_index -

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-07-10 Thread via GitHub
viitoare commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2220263998 I also encountered the same problem. During debugging, I found a very strange phenomenon. In the airflow.cfg file, I set the parallelism to 300, but when I printed the value of se

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-07-10 Thread via GitHub
potiuk commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2220285045 > I found a very strange phenomenon. In the airflow.cfg file, I set the parallelism to 300, but when I printed the value of self.parallelism in the base_executor.py file, it was 32

Re: [PR] [AIP-62] Translate AIP-60 URI to OpenLineage [airflow]

2024-07-10 Thread via GitHub
dolfinus commented on code in PR #40173: URL: https://github.com/apache/airflow/pull/40173#discussion_r1672110969 ## tests/providers/common/io/datasets/test_file.py: ## @@ -0,0 +1,57 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [I] Executor reports task instance (...) finished (failed) although the task says it's queued [airflow]

2024-07-10 Thread via GitHub
viitoare commented on issue #39717: URL: https://github.com/apache/airflow/issues/39717#issuecomment-2220298270 > > I found a very strange phenomenon. In the airflow.cfg file, I set the parallelism to 300, but when I printed the value of self.parallelism in the base_executor.py file, it was

Re: [I] glob syntax can be similar to .gitignore in .airflowignore files [airflow]

2024-07-10 Thread via GitHub
poorvirohidekar commented on issue #40672: URL: https://github.com/apache/airflow/issues/40672#issuecomment-2220354452 @potiuk This seems to be interesting and I would like to try this with some assistance. Could you please assign it to me? -- This is an automated message from the Apache

Re: [I] glob syntax can be similar to .gitignore in .airflowignore files [airflow]

2024-07-10 Thread via GitHub
potiuk commented on issue #40672: URL: https://github.com/apache/airflow/issues/40672#issuecomment-2220385336 > @potiuk This seems to be interesting and I would like to try this with some assistance. Could you please assign it to me? Just to set an expectation - there will not be much

Re: [PR] Align timers and timing metrics (ms) across all metrics loggers [airflow]

2024-07-10 Thread via GitHub
dirrao commented on code in PR #39908: URL: https://github.com/apache/airflow/pull/39908#discussion_r1672235407 ## tests/_internals/forbidden_warnings.py: ## @@ -62,6 +62,11 @@ def pytest_itemcollected(self, item: pytest.Item): # Add marker at the beginning of the m

[PR] Add priority_weight to ti_trigger_id idx [airflow]

2024-07-10 Thread via GitHub
vchiapaikeo opened a new pull request, #40688: URL: https://github.com/apache/airflow/pull/40688 This is my first db migration change so not exactly sure how to test it. If you have any tips, please let me know! --- **^ Add meaningful description above**

Re: [PR] Use sentinel to elide the dag object on reserialization [airflow]

2024-07-10 Thread via GitHub
dstandish commented on code in PR #39825: URL: https://github.com/apache/airflow/pull/39825#discussion_r1672328952 ## airflow/models/baseoperator.py: ## @@ -1245,11 +1245,21 @@ def dag(self) -> DAG: # type: ignore[override] @dag.setter def dag(self, dag: DAG | None):

Re: [PR] Use sentinel to elide the dag object on reserialization [airflow]

2024-07-10 Thread via GitHub
dstandish commented on code in PR #39825: URL: https://github.com/apache/airflow/pull/39825#discussion_r1672330199 ## airflow/models/baseoperator.py: ## @@ -1245,11 +1245,21 @@ def dag(self) -> DAG: # type: ignore[override] @dag.setter def dag(self, dag: DAG | None):

Re: [PR] Add priority_weight to ti_trigger_id idx [airflow]

2024-07-10 Thread via GitHub
vchiapaikeo closed pull request #40688: Add priority_weight to ti_trigger_id idx URL: https://github.com/apache/airflow/pull/40688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [I] Expensive Trigger Query Does Not Use Index (even after forcing a hint) Due to Additional Column [airflow]

2024-07-10 Thread via GitHub
vchiapaikeo commented on issue #40674: URL: https://github.com/apache/airflow/issues/40674#issuecomment-2220618288 Actually, I'm not convinced this additional index will fix things. I'm going to attempt to revise the query instead as that seems to produce consistently better results in the

Re: [I] AwsAuthManager seems to be incompatible with Airflow 2.8 [airflow]

2024-07-10 Thread via GitHub
vincbeck commented on issue #40684: URL: https://github.com/apache/airflow/issues/40684#issuecomment-2220670275 Looking into it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Use sentinel to mark dag as removed on reserialization [airflow]

2024-07-10 Thread via GitHub
dstandish commented on code in PR #39825: URL: https://github.com/apache/airflow/pull/39825#discussion_r1672398193 ## airflow/models/baseoperator.py: ## @@ -1245,11 +1245,21 @@ def dag(self) -> DAG: # type: ignore[override] @dag.setter def dag(self, dag: DAG | None):

Re: [PR] Airbyte ol integration [airflow]

2024-07-10 Thread via GitHub
boring-cyborg[bot] commented on PR #40689: URL: https://github.com/apache/airflow/pull/40689#issuecomment-2220676726 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

[PR] Airbyte ol integration [airflow]

2024-07-10 Thread via GitHub
Imbruced opened a new pull request, #40689: URL: https://github.com/apache/airflow/pull/40689 --- **Add open lineage data from airbyte operator** This mr aims to add ol support for airbyte operator. It adds schema and column level lineage and simple sta

Re: [I] AwsAuthManager seems to be incompatible with Airflow 2.8 [airflow]

2024-07-10 Thread via GitHub
vincbeck commented on issue #40684: URL: https://github.com/apache/airflow/issues/40684#issuecomment-2220700957 > make AWSAuthManager 2.9+ only (we will have to exclude the tests for 2.8 and add some code that detects airflow version and provide meaningful error message) I think this

[PR] Make `AwsAuthManager` compatible with only Airflow >= 2.9 [airflow]

2024-07-10 Thread via GitHub
vincbeck opened a new pull request, #40690: URL: https://github.com/apache/airflow/pull/40690 Resolves #40684 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs

Re: [I] AwsAuthManager seems to be incompatible with Airflow 2.8 [airflow]

2024-07-10 Thread via GitHub
vincbeck commented on issue #40684: URL: https://github.com/apache/airflow/issues/40684#issuecomment-2220734556 #40690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[PR] Chart: add git-sync-ssh secret template to dagprocessor [airflow]

2024-07-10 Thread via GitHub
Aakcht opened a new pull request, #40691: URL: https://github.com/apache/airflow/pull/40691 For some reason in #39936 git-ssh-key-secret was not added to DAG processor, this PR adds it to DAG processor. --- **^ Add meaningful description above** Read

[PR] feat(slack): add unfurl options to slack notifier [airflow]

2024-07-10 Thread via GitHub
armand-sauzay opened a new pull request, #40692: URL: https://github.com/apache/airflow/pull/40692 Enable toggling the unfurling of media and URLs for the SlackNotifier. In order to not unfurl media and have alerts messages that are shorter than the unfurli

Re: [PR] feat(slack): add unfurl options to slack notifier [airflow]

2024-07-10 Thread via GitHub
boring-cyborg[bot] commented on PR #40692: URL: https://github.com/apache/airflow/pull/40692#issuecomment-2220742884 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

Re: [PR] feat(slack): add unfurl options to slack notifier [airflow]

2024-07-10 Thread via GitHub
armand-sauzay closed pull request #40692: feat(slack): add unfurl options to slack notifier URL: https://github.com/apache/airflow/pull/40692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[PR] Schedule on dataset alias [airflow]

2024-07-10 Thread via GitHub
Lee-W opened a new pull request, #40693: URL: https://github.com/apache/airflow/pull/40693 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull

[PR] Add the ability to toggle the unfurl behavior for slack notifier [airflow]

2024-07-10 Thread via GitHub
armand-sauzay opened a new pull request, #40694: URL: https://github.com/apache/airflow/pull/40694 Enable toggling the unfurling of media and URLs for the SlackNotifier. In order to not unfurl media and have alerts messages that are shorter than the unfurli

Re: [PR] Make `AwsAuthManager` compatible with only Airflow >= 2.9 [airflow]

2024-07-10 Thread via GitHub
potiuk commented on code in PR #40690: URL: https://github.com/apache/airflow/pull/40690#discussion_r1672491431 ## airflow/providers/amazon/aws/auth_manager/aws_auth_manager.py: ## @@ -197,8 +206,14 @@ def is_authorized_view( ) def is_authorized_custom_view( -

Re: [PR] feat(aws): provide the context to check_fn in S3 sensor [airflow]

2024-07-10 Thread via GitHub
hussein-awala merged PR #40686: URL: https://github.com/apache/airflow/pull/40686 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [PR] Make `AwsAuthManager` compatible with only Airflow >= 2.9 [airflow]

2024-07-10 Thread via GitHub
potiuk commented on PR #40690: URL: https://github.com/apache/airflow/pull/40690#issuecomment-2220778921 I think we also need to add the 2_9_PLUS skipif for the failing test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

(airflow) branch main updated: feat(aws): provide the context to check_fn in S3 sensor (#40686)

2024-07-10 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c627311863 feat(aws): provide the context to

[PR] Add priority_weight to ti_trigger_id idx [airflow]

2024-07-10 Thread via GitHub
vchiapaikeo opened a new pull request, #40695: URL: https://github.com/apache/airflow/pull/40695 closes: [#ISSUE](https://github.com/apache/airflow/issues/40674) --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://githu

Re: [I] Expensive Trigger Query Does Not Use Index (even after forcing a hint) Due to Additional Column [airflow]

2024-07-10 Thread via GitHub
vchiapaikeo commented on issue #40674: URL: https://github.com/apache/airflow/issues/40674#issuecomment-2220804204 There's a small chance that we may need to add an index hint to the query as well. Reopening the PR and we can come back to adding that index hint in the query if we need to

Re: [PR] Make `AwsAuthManager` compatible with only Airflow >= 2.9 [airflow]

2024-07-10 Thread via GitHub
vincbeck commented on code in PR #40690: URL: https://github.com/apache/airflow/pull/40690#discussion_r1672504696 ## airflow/providers/amazon/aws/auth_manager/aws_auth_manager.py: ## @@ -197,8 +206,14 @@ def is_authorized_view( ) def is_authorized_custom_view( -

Re: [PR] Make `AwsAuthManager` compatible with only Airflow >= 2.9 [airflow]

2024-07-10 Thread via GitHub
vincbeck commented on PR #40690: URL: https://github.com/apache/airflow/pull/40690#issuecomment-2220816362 > I think we also need to add the 2_9_PLUS skipif for the failing test Do we? Should not it be automatic? If airflow < 2.9 is used (which is the case in the compatibility tests),

Re: [PR] Add priority_weight to ti_trigger_id idx [airflow]

2024-07-10 Thread via GitHub
vchiapaikeo commented on PR #40695: URL: https://github.com/apache/airflow/pull/40695#issuecomment-2220820416 There is also a FK defined around this --> ``` ForeignKeyConstraint( [trigger_id], ["trigger.id"], name="task_instance_t

[PR] Add `ti.blocked_by_upstream` to limit repetitive deps check in the scheduler [airflow]

2024-07-10 Thread via GitHub
ephraimbuddy opened a new pull request, #40696: URL: https://github.com/apache/airflow/pull/40696 TI.are_dependencies_met run over and over even when no changes have happened that would allow it to pass. This causes the scheduler loop to get slower and slower as more blocked TIs pile up

Re: [PR] updating pgvector to !=0.3.0 to avoid recent issue in latest releases(0.3.0) [airflow]

2024-07-10 Thread via GitHub
pankajkoti commented on PR #40683: URL: https://github.com/apache/airflow/pull/40683#issuecomment-2220865259 The failing tests seem all unrelated to this change. I guess we're good to merge this PR? -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] Make `AwsAuthManager` compatible with only Airflow >= 2.9 [airflow]

2024-07-10 Thread via GitHub
vincbeck commented on PR #40690: URL: https://github.com/apache/airflow/pull/40690#issuecomment-2220868617 You're right -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] AIP-64: Add REST API endpoints for TI try level details [airflow]

2024-07-10 Thread via GitHub
jedcunningham commented on code in PR #40441: URL: https://github.com/apache/airflow/pull/40441#discussion_r1672500761 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -754,3 +754,79 @@ def get_mapped_task_instance_dependencies( return get_task_instance_d

Re: [PR] Make `AwsAuthManager` compatible with only Airflow >= 2.9 [airflow]

2024-07-10 Thread via GitHub
potiuk commented on PR #40690: URL: https://github.com/apache/airflow/pull/40690#issuecomment-2220879348 > Do we? Should not it be automatic? If airflow < 2.9 is used (which is the case in the compatibility tests), then AwsAuthManager raises AirflowOptionalProviderFeatureException, then the

Re: [PR] updating pgvector to !=0.3.0 to avoid recent issue in latest releases(0.3.0) [airflow]

2024-07-10 Thread via GitHub
potiuk merged PR #40683: URL: https://github.com/apache/airflow/pull/40683 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (c627311863 -> 5d36f27df2)

2024-07-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from c627311863 feat(aws): provide the context to check_fn in S3 sensor (#40686) add 5d36f27df2 updating pgvector to !

Re: [PR] updating pgvector to !=0.3.0 to avoid recent issue in latest releases(0.3.0) [airflow]

2024-07-10 Thread via GitHub
potiuk commented on PR #40683: URL: https://github.com/apache/airflow/pull/40683#issuecomment-2220887989 Yep. The failing test will be fixed with https://github.com/apache/airflow/pull/40690 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Make `AwsAuthManager` compatible with only Airflow >= 2.9 [airflow]

2024-07-10 Thread via GitHub
vincbeck commented on PR #40690: URL: https://github.com/apache/airflow/pull/40690#issuecomment-2220893824 > > Do we? Should not it be automatic? If airflow < 2.9 is used (which is the case in the compatibility tests), then AwsAuthManager raises AirflowOptionalProviderFeatureException, then

Re: [I] AwsAuthManager seems to be incompatible with Airflow 2.8 [airflow]

2024-07-10 Thread via GitHub
vincbeck closed issue #40684: AwsAuthManager seems to be incompatible with Airflow 2.8 URL: https://github.com/apache/airflow/issues/40684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Make `AwsAuthManager` compatible with only Airflow >= 2.9 [airflow]

2024-07-10 Thread via GitHub
vincbeck merged PR #40690: URL: https://github.com/apache/airflow/pull/40690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

(airflow) branch main updated: Make `AwsAuthManager` compatible with only Airflow >= 2.9 (#40690)

2024-07-10 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 224cb75be1 Make `AwsAuthManager` compatible with

Re: [PR] Make `AwsAuthManager` compatible with only Airflow >= 2.9 [airflow]

2024-07-10 Thread via GitHub
potiuk commented on PR #40690: URL: https://github.com/apache/airflow/pull/40690#issuecomment-2220948438 > I see. It is reassuring to know where this misunderstanding comes from :) Thanks! We are living on a bleeding edge. What was valid yesterday, might not be today ;) -- This i

Re: [PR] Add `ti.blocked_by_upstream` to limit repetitive deps check in the scheduler [airflow]

2024-07-10 Thread via GitHub
jedcunningham commented on code in PR #40696: URL: https://github.com/apache/airflow/pull/40696#discussion_r1672574284 ## airflow/models/dagrun.py: ## @@ -948,9 +948,12 @@ def _filter_tis_and_exclude_removed(dag: DAG, tis: list[TI]) -> Iterable[TI]: tis = list(_filte

Re: [PR] Add the ability to toggle the unfurl behavior for slack notifier [airflow]

2024-07-10 Thread via GitHub
potiuk commented on code in PR #40694: URL: https://github.com/apache/airflow/pull/40694#discussion_r1672670298 ## airflow/providers/slack/notifications/slack.py: ## @@ -64,6 +64,8 @@ def __init__( proxy: str | None = None, timeout: int | None = None,

Re: [PR] Chart: add git-sync-ssh secret template to dagprocessor [airflow]

2024-07-10 Thread via GitHub
potiuk merged PR #40691: URL: https://github.com/apache/airflow/pull/40691 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (224cb75be1 -> b17a9fdd1f)

2024-07-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 224cb75be1 Make `AwsAuthManager` compatible with only Airflow >= 2.9 (#40690) add b17a9fdd1f Chart: add git-sync-

Re: [PR] Add the ability to toggle the unfurl behavior for slack notifier [airflow]

2024-07-10 Thread via GitHub
armand-sauzay commented on code in PR #40694: URL: https://github.com/apache/airflow/pull/40694#discussion_r1672696701 ## airflow/providers/slack/notifications/slack.py: ## @@ -64,6 +64,8 @@ def __init__( proxy: str | None = None, timeout: int | None = None,

Re: [PR] Add the ability to toggle the unfurl behavior for slack notifier [airflow]

2024-07-10 Thread via GitHub
potiuk commented on code in PR #40694: URL: https://github.com/apache/airflow/pull/40694#discussion_r1672702692 ## airflow/providers/slack/notifications/slack.py: ## @@ -64,6 +64,8 @@ def __init__( proxy: str | None = None, timeout: int | None = None,

Re: [PR] Add the ability to toggle the unfurl behavior for slack notifier [airflow]

2024-07-10 Thread via GitHub
potiuk merged PR #40694: URL: https://github.com/apache/airflow/pull/40694 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Add the ability to toggle the unfurl behavior for slack notifier [airflow]

2024-07-10 Thread via GitHub
boring-cyborg[bot] commented on PR #40694: URL: https://github.com/apache/airflow/pull/40694#issuecomment-2221079899 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

(airflow) branch main updated: feat(slack): add unfurl options to slack notifier (#40694)

2024-07-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c25858cb30 feat(slack): add unfurl options to slack

Re: [PR] Send important executor logs to task logs [airflow]

2024-07-10 Thread via GitHub
vincbeck merged PR #40468: URL: https://github.com/apache/airflow/pull/40468 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

(airflow) branch main updated (c25858cb30 -> aca140a2c1)

2024-07-10 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from c25858cb30 feat(slack): add unfurl options to slack notifier (#40694) add aca140a2c1 Send important executor log

Re: [PR] Send important executor logs to task logs [airflow]

2024-07-10 Thread via GitHub
vincbeck commented on code in PR #40468: URL: https://github.com/apache/airflow/pull/40468#discussion_r1672707812 ## airflow/executors/base_executor.py: ## @@ -284,8 +289,11 @@ def trigger_tasks(self, open_slots: int) -> None: self.log.info("queued but still

Re: [PR] Add the ability to toggle the unfurl behavior for slack notifier [airflow]

2024-07-10 Thread via GitHub
armand-sauzay commented on code in PR #40694: URL: https://github.com/apache/airflow/pull/40694#discussion_r1672727001 ## airflow/providers/slack/notifications/slack.py: ## @@ -64,6 +64,8 @@ def __init__( proxy: str | None = None, timeout: int | None = None,

(airflow) branch main updated: Scheduler job: main loop and event processing for multi executors (#40017)

2024-07-10 Thread onikolas
This is an automated email from the ASF dual-hosted git repository. onikolas pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8e2b198186 Scheduler job: main loop and event pro

Re: [PR] AIP-61 - Scheduler job: main loop and event processing for multi executors [airflow]

2024-07-10 Thread via GitHub
o-nikolas merged PR #40017: URL: https://github.com/apache/airflow/pull/40017 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Support hybrid executors in backfill jobs [airflow]

2024-07-10 Thread via GitHub
o-nikolas commented on code in PR #40472: URL: https://github.com/apache/airflow/pull/40472#discussion_r1672750190 ## airflow/exceptions.py: ## @@ -458,3 +458,7 @@ def __str__(self): "Error deserializing result. Note that result deserialization " "is no

Re: [PR] Add `ti.blocked_by_upstream` to limit repetitive deps check in the scheduler [airflow]

2024-07-10 Thread via GitHub
ephraimbuddy commented on code in PR #40696: URL: https://github.com/apache/airflow/pull/40696#discussion_r1672752799 ## airflow/models/dagrun.py: ## @@ -966,6 +969,14 @@ def _filter_tis_and_exclude_removed(dag: DAG, tis: list[TI]) -> Iterable[TI]: new_unfinish

(airflow) branch constraints-main updated: Updating constraints. Github run id:9879231866

2024-07-10 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 3adbfdc873 Updating con

(airflow) branch main updated (8e2b198186 -> 23b9add867)

2024-07-10 Thread onikolas
This is an automated email from the ASF dual-hosted git repository. onikolas pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 8e2b198186 Scheduler job: main loop and event processing for multi executors (#40017) add 23b9add867 Support hy

Re: [PR] Support hybrid executors in backfill jobs [airflow]

2024-07-10 Thread via GitHub
o-nikolas merged PR #40472: URL: https://github.com/apache/airflow/pull/40472 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[PR] Send important executor logs to task logs in `AwsBatchExecutor` [airflow]

2024-07-10 Thread via GitHub
vincbeck opened a new pull request, #40698: URL: https://github.com/apache/airflow/pull/40698 Related to #40468 but for `AwsBatchExecutor` --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflo

Re: [PR] Make XCom display as react json [airflow]

2024-07-10 Thread via GitHub
jscheffl commented on code in PR #40640: URL: https://github.com/apache/airflow/pull/40640#discussion_r1672830793 ## airflow/www/static/js/dag/details/taskInstance/Xcom/XcomEntry.tsx: ## @@ -66,6 +67,10 @@ const XcomEntry = ({ No value found for XCom key );

[I] A wild discussion turns to an issue [airflow]

2024-07-10 Thread via GitHub
snowsky opened a new issue, #40699: URL: https://github.com/apache/airflow/issues/40699 ### Apache Airflow version 2.9.2 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? I don't mind to challenge @potiuk's authority if I

Re: [I] A wild discussion turns to an issue [airflow]

2024-07-10 Thread via GitHub
boring-cyborg[bot] commented on issue #40699: URL: https://github.com/apache/airflow/issues/40699#issuecomment-2221285764 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[PR] Use correct unknown executor exception in scheduler job [airflow]

2024-07-10 Thread via GitHub
o-nikolas opened a new pull request, #40700: URL: https://github.com/apache/airflow/pull/40700 A new exception for unknown executors was in the backfill changes for hybrid executors that was not present in the scheduler changes. Now that they're both merged, update the scheduler job to use

Re: [I] A wild discussion turns to an issue [airflow]

2024-07-10 Thread via GitHub
potiuk closed issue #40699: A wild discussion turns to an issue URL: https://github.com/apache/airflow/issues/40699 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [I] A wild discussion turns to an issue [airflow]

2024-07-10 Thread via GitHub
potiuk commented on issue #40699: URL: https://github.com/apache/airflow/issues/40699#issuecomment-2221349640 You can derive answer from the discussion -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] Add documentation for hybrid executor and ungate the feature [airflow]

2024-07-10 Thread via GitHub
o-nikolas opened a new pull request, #40701: URL: https://github.com/apache/airflow/pull/40701 Publicly "release" the hybrid executor feature, including: - Documentation - Remove feature gating - Remove mocks that disabled the feature gate for testing purposes - Removed tests

Re: [PR] Use correct unknown executor exception in scheduler job [airflow]

2024-07-10 Thread via GitHub
o-nikolas merged PR #40700: URL: https://github.com/apache/airflow/pull/40700 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated (23b9add867 -> cf1278c9ca)

2024-07-10 Thread onikolas
This is an automated email from the ASF dual-hosted git repository. onikolas pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 23b9add867 Support hybrid executors in backfill jobs (#40472) add cf1278c9ca Use correct unknown executor except

  1   2   >