Re: [PR] ADD prototype version dark mode for Airflow UI [airflow]

2024-05-09 Thread via GitHub
YounHS commented on PR #39355: URL: https://github.com/apache/airflow/pull/39355#issuecomment-2102087153 Hello, @jscheffl I fix the flickering problem. for this solution, I add custom function in views.py. It just a prototype, so please excuse the messy code. -- This is an automated mess

Re: [PR] fix: use `sqlalchemy_url` property in `get_uri` for postgresql provider [airflow]

2024-05-09 Thread via GitHub
Taragolis merged PR #38831: URL: https://github.com/apache/airflow/pull/38831 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [I] Connection to Postgres failed due to special characters in a password [airflow]

2024-05-09 Thread via GitHub
Taragolis closed issue #38187: Connection to Postgres failed due to special characters in a password URL: https://github.com/apache/airflow/issues/38187 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

(airflow) branch main updated: fix: use `sqlalchemy_url` property in `get_uri` for postgresql provider (#38831)

2024-05-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4ada175e3d fix: use `sqlalchemy_url` property in

[PR] Run unit tests with airflow installed from packages [airflow]

2024-05-09 Thread via GitHub
potiuk opened a new pull request, #39513: URL: https://github.com/apache/airflow/pull/39513 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pul

Re: [PR] Run unit tests with airflow installed from packages [airflow]

2024-05-09 Thread via GitHub
potiuk closed pull request #39513: Run unit tests with airflow installed from packages URL: https://github.com/apache/airflow/pull/39513 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[I] mypy errors - Argument 1 has incompatible type "XComArg" [airflow]

2024-05-09 Thread via GitHub
ampx-mg opened a new issue, #39514: URL: https://github.com/apache/airflow/issues/39514 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.8.1 ### What happened? Ever since upgrad

Re: [PR] Fix deferrable mode for BeamRunJavaPipelineOperator [airflow]

2024-05-09 Thread via GitHub
VladaZakharova commented on PR #39371: URL: https://github.com/apache/airflow/pull/39371#issuecomment-2102274847 Hi @eladkal @potiuk ! Can you please check the changes here? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] Resolve deprecations in `docker` provider tests [airflow]

2024-05-09 Thread via GitHub
Taragolis opened a new pull request, #39515: URL: https://github.com/apache/airflow/pull/39515 related: https://github.com/apache/airflow/issues/38642 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/ai

[PR] Resolve deprecations in `apache.livy` provider tests [airflow]

2024-05-09 Thread via GitHub
Taragolis opened a new pull request, #39516: URL: https://github.com/apache/airflow/pull/39516 related: https://github.com/apache/airflow/issues/38642 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/ai

[PR] Resolve deprecations in `atlassian.jira` provider tests [airflow]

2024-05-09 Thread via GitHub
Taragolis opened a new pull request, #39517: URL: https://github.com/apache/airflow/pull/39517 related: https://github.com/apache/airflow/issues/38642 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/ai

Re: [PR] Add encryption_configuration parameter to BigQueryCheckOperator and BigQueryTableCheckOperator [airflow]

2024-05-09 Thread via GitHub
VladaZakharova commented on PR #39432: URL: https://github.com/apache/airflow/pull/39432#issuecomment-2102290274 Hi @potiuk ! Can you please check changes here? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[PR] Resolve deprecations in `slack` provider tests [airflow]

2024-05-09 Thread via GitHub
Taragolis opened a new pull request, #39518: URL: https://github.com/apache/airflow/pull/39518 related: https://github.com/apache/airflow/issues/38642 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/ai

Re: [PR] Deprecation of AutoML services: Add deprecation warnings and raise exceptions for already deprecated ones [airflow]

2024-05-09 Thread via GitHub
VladaZakharova commented on PR #38673: URL: https://github.com/apache/airflow/pull/38673#issuecomment-2102293010 Hey @shahar1 ! Would be nice if you can check the changes here again -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] Resolve deprecations in `vertica` provider tests [airflow]

2024-05-09 Thread via GitHub
Taragolis opened a new pull request, #39519: URL: https://github.com/apache/airflow/pull/39519 https://github.com/apache/airflow/pull/39518 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/

[PR] [OPENLINEAGE] Add new DAG job structure facet [airflow]

2024-05-09 Thread via GitHub
kacpermuda opened a new pull request, #39520: URL: https://github.com/apache/airflow/pull/39520 closes: #39467 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pul

Re: [PR] [OPENLINEAGE] Add new DAG job structure facet [airflow]

2024-05-09 Thread via GitHub
kacpermuda commented on code in PR #39520: URL: https://github.com/apache/airflow/pull/39520#discussion_r1595241671 ## airflow/providers/openlineage/utils/utils.py: ## @@ -268,6 +269,91 @@ def get_airflow_run_facet( } +def get_dag_structure_job_facet(dag_run: DagRun): +

[I] Unsupported credential type AzureIdentityCredentialAdapter when using AzureDataLakeStorageV2Hook with DefaultAzureCredential as authentication method (AKS workload indentity) [airflow]

2024-05-09 Thread via GitHub
coy2k opened a new issue, #39521: URL: https://github.com/apache/airflow/issues/39521 ### Apache Airflow Provider(s) microsoft-azure ### Versions of Apache Airflow Providers apache-airflow-providers-microsoft-azure==9.0.1 azure-identity==1.15.0 azure-storage-blob==1

Re: [I] Unsupported credential type AzureIdentityCredentialAdapter when using AzureDataLakeStorageV2Hook with DefaultAzureCredential as authentication method (AKS workload indentity) [airflow]

2024-05-09 Thread via GitHub
boring-cyborg[bot] commented on issue #39521: URL: https://github.com/apache/airflow/issues/39521#issuecomment-2102467376 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Add unit tests for EksCreateClusterTrigger and EksDeleteClusterTrigger [airflow]

2024-05-09 Thread via GitHub
potiuk merged PR #39508: URL: https://github.com/apache/airflow/pull/39508 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (4ada175e3d -> fa9977655a)

2024-05-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 4ada175e3d fix: use `sqlalchemy_url` property in `get_uri` for postgresql provider (#38831) add fa9977655a Add un

Re: [PR] Fix side effects in DBT Cloud tests [airflow]

2024-05-09 Thread via GitHub
potiuk commented on PR #39511: URL: https://github.com/apache/airflow/pull/39511#issuecomment-2102488857 NICE! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] Fix: Reuse ProcessPoolExecutor in CeleryExecutor [airflow]

2024-05-09 Thread via GitHub
kaxil commented on PR #39484: URL: https://github.com/apache/airflow/pull/39484#issuecomment-2102503655 @vatsrahul1001 is going to run some performance benchmarks with the changes in this PR too in upcoming days. -- This is an automated message from the Apache Git Service. To respond to

[I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]

2024-05-09 Thread via GitHub
EMT-Swaroop opened a new issue, #39522: URL: https://github.com/apache/airflow/issues/39522 ### Apache Airflow version 2.9.1 ### If "Other Airflow 2 version" selected, which one? 2.9.0 ### What happened? According to airflow documentation, from 2.8.2 airflow

Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]

2024-05-09 Thread via GitHub
boring-cyborg[bot] commented on issue #39522: URL: https://github.com/apache/airflow/issues/39522#issuecomment-2102548022 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Add encryption_configuration parameter to BigQueryCheckOperator and BigQueryTableCheckOperator [airflow]

2024-05-09 Thread via GitHub
potiuk merged PR #39432: URL: https://github.com/apache/airflow/pull/39432 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Add encryption_configuration parameter to BigQueryCheckOperator and BigQueryTableCheckOperator (#39432)

2024-05-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c7c680ee3b Add encryption_configuration parameter t

[PR] [OpenLineage] Remove deprecated facets [airflow]

2024-05-09 Thread via GitHub
kacpermuda opened a new pull request, #39523: URL: https://github.com/apache/airflow/pull/39523 Remove AirflowMappedTaskRunFacet and UnknownOperatorAttributeRunFacet as they have been deprecated in last release. --- **^ Add meaningful description above** Read

Re: [PR] Fix deferrable mode for BeamRunJavaPipelineOperator [airflow]

2024-05-09 Thread via GitHub
potiuk commented on code in PR #39371: URL: https://github.com/apache/airflow/pull/39371#discussion_r1595379583 ## airflow/providers/apache/beam/triggers/beam.py: ## @@ -233,6 +235,22 @@ async def run(self) -> AsyncIterator[TriggerEvent]: # type: ignore[override]

Re: [PR] Fix deferrable mode for BeamRunJavaPipelineOperator [airflow]

2024-05-09 Thread via GitHub
potiuk commented on code in PR #39371: URL: https://github.com/apache/airflow/pull/39371#discussion_r1595380774 ## airflow/providers/apache/beam/triggers/beam.py: ## @@ -233,6 +235,22 @@ async def run(self) -> AsyncIterator[TriggerEvent]: # type: ignore[override]

Re: [PR] Configure HttpHook's auth_type from Connection [airflow]

2024-05-09 Thread via GitHub
potiuk commented on PR #35591: URL: https://github.com/apache/airflow/pull/35591#issuecomment-2102576504 It's quite a big change. Generally speaking reviewing 700+ PRs will take weeks (if people have time). I wonder if there there is a way to split out parts of it into smaller PRs ? -- T

Re: [PR] Configure HttpHook's auth_type from Connection [airflow]

2024-05-09 Thread via GitHub
potiuk commented on PR #35591: URL: https://github.com/apache/airflow/pull/35591#issuecomment-2102577563 Usually what we do in such case - we take the bits of such PR, make new PR, review, merge, rebase (rinse and repeat). -- This is an automated message from the Apache Git Service. To re

Re: [PR] adding callback hook for on/off state of dag [airflow]

2024-05-09 Thread via GitHub
potiuk commented on PR #38683: URL: https://github.com/apache/airflow/pull/38683#issuecomment-2102584708 > @uranusjr @potiuk Can you give me directions in this part? Look at history of `airflow/migrations` -> this is where migration scripts are maintained and we use alembic to generat

Re: [I] Error rendering template: '_GeneratorContextManager' object has no attribute 'query' [airflow]

2024-05-09 Thread via GitHub
raphaelauv commented on issue #39162: URL: https://github.com/apache/airflow/issues/39162#issuecomment-2102632406 It's only sometimes. the same task ( mapped ) is not always problematic for all dag_run -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] Add task failed dependencies to details page. [airflow]

2024-05-09 Thread via GitHub
tirkarthi commented on PR #38449: URL: https://github.com/apache/airflow/pull/38449#issuecomment-2102692338 @jscheffl Thanks, updated the version number and regenerated the files. I have looked into auto refresh and found a pattern where auto refresh is done based on `useAutoRefresh`. The A

Re: [PR] Fix side effects in DBT Cloud tests [airflow]

2024-05-09 Thread via GitHub
josh-fell merged PR #39511: URL: https://github.com/apache/airflow/pull/39511 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated (c7c680ee3b -> f973502c79)

2024-05-09 Thread joshfell
This is an automated email from the ASF dual-hosted git repository. joshfell pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from c7c680ee3b Add encryption_configuration parameter to BigQueryCheckOperator and BigQueryTableCheckOperator (#39432)

[I] SparkSubmitOperator not mark task as success after spark job complete job [airflow]

2024-05-09 Thread via GitHub
dannyeuu opened a new issue, #39524: URL: https://github.com/apache/airflow/issues/39524 ### Apache Airflow Provider(s) apache-spark ### Versions of Apache Airflow Providers apache-airflow-providers-apache-hive==6.1.2 apache-airflow-providers-apache-spark==4.1.1 apa

Re: [I] SparkSubmitOperator not mark task as success after spark job complete job [airflow]

2024-05-09 Thread via GitHub
boring-cyborg[bot] commented on issue #39524: URL: https://github.com/apache/airflow/issues/39524#issuecomment-2102781976 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[PR] Resolve DbtCloudJobRunAsyncSensor deprecation in system tests [airflow]

2024-05-09 Thread via GitHub
josh-fell opened a new pull request, #39525: URL: https://github.com/apache/airflow/pull/39525 Related: #39485 The DbtCloudJobRunAsyncSensor is deprecated in favor of using DbtCloudJobRunSensor with `deferrable=True` set. Removing this sensor from the dbt Cloud system test and updati

Re: [PR] Resolve DbtCloudJobRunAsyncSensor deprecation in system tests [airflow]

2024-05-09 Thread via GitHub
Taragolis commented on PR #39525: URL: https://github.com/apache/airflow/pull/39525#issuecomment-2102811244 Nice! Could you also remove this line https://github.com/apache/airflow/blob/0be44788e4610393d4ed46c39a87d386c703a9f7/tests/always/test_example_dags.py#L54 -- This is an

Re: [PR] Resolve DbtCloudJobRunAsyncSensor deprecation in system tests [airflow]

2024-05-09 Thread via GitHub
josh-fell commented on PR #39525: URL: https://github.com/apache/airflow/pull/39525#issuecomment-2102811854 > Nice! > > Could you also remove this line > > https://github.com/apache/airflow/blob/0be44788e4610393d4ed46c39a87d386c703a9f7/tests/always/test_example_dags.py#L54

Re: [PR] Resolve DbtCloudJobRunAsyncSensor deprecation in system tests [airflow]

2024-05-09 Thread via GitHub
josh-fell commented on PR #39525: URL: https://github.com/apache/airflow/pull/39525#issuecomment-2102816857 Removed in the system test from the ignore list in https://github.com/apache/airflow/pull/39525/commits/50558409741d0289e0cdd3db3bf59c521ece60d3 -- This is an automated message from

Re: [I] Resolve `AirflowProviderDeprecationWarning` in providers system tests [airflow]

2024-05-09 Thread via GitHub
josh-fell commented on issue #39485: URL: https://github.com/apache/airflow/issues/39485#issuecomment-2102819669 Happy to help out where I can here. Thanks for putting this together @Taragolis! -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Add Scarf based telemetry [airflow]

2024-05-09 Thread via GitHub
jedcunningham commented on code in PR #39510: URL: https://github.com/apache/airflow/pull/39510#discussion_r1595576751 ## docs/apache-airflow/installation/installing-from-pypi.rst: ## @@ -331,6 +331,17 @@ dependencies compatible with just airflow core at the moment Airflow was

Re: [PR] Add Scarf based telemetry [airflow]

2024-05-09 Thread via GitHub
jedcunningham commented on code in PR #39510: URL: https://github.com/apache/airflow/pull/39510#discussion_r1595591336 ## airflow/config_templates/config.yml: ## @@ -2591,3 +2591,22 @@ sensors: type: float example: ~ default: "604800" +scarf_analytics: + de

[PR] Add indexes on dag_id column in refencing tables to speed up deletion of dag records [airflow]

2024-05-09 Thread via GitHub
pankajkoti opened a new pull request, #39526: URL: https://github.com/apache/airflow/pull/39526 When the dag records count gets huge, and users try to delete DAG and DAG runs that are no longer needed or are stale, it is observed that the deletion is significantly slow. The reason f

[I] Rendered Template link missing from `Details` tab [airflow]

2024-05-09 Thread via GitHub
nathadfield opened a new issue, #39527: URL: https://github.com/apache/airflow/issues/39527 ### Apache Airflow version 2.9.1 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? In Airflow v2.9.1, the `Rendered Template` lin

Re: [I] SparkSubmitOperator not mark task as success after spark job complete job [airflow]

2024-05-09 Thread via GitHub
RNHTTR commented on issue #39524: URL: https://github.com/apache/airflow/issues/39524#issuecomment-2102882568 Can you see if this is occurring on the latest version of the [spark provider package](https://airflow.apache.org/docs/apache-airflow-providers-apache-spark/stable/index.html) (4.8

Re: [PR] Resolve DbtCloudJobRunAsyncSensor deprecation in system tests [airflow]

2024-05-09 Thread via GitHub
josh-fell merged PR #39525: URL: https://github.com/apache/airflow/pull/39525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated (f973502c79 -> 6455f7f92f)

2024-05-09 Thread joshfell
This is an automated email from the ASF dual-hosted git repository. joshfell pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from f973502c79 Fix side effects in DBT Cloud tests (#39511) add 6455f7f92f Resolve DbtCloudJobRunAsyncSensor depreca

Re: [PR] Add indexes on dag_id column in refencing tables to speed up deletion of dag records [airflow]

2024-05-09 Thread via GitHub
Taragolis commented on PR #39526: URL: https://github.com/apache/airflow/pull/39526#issuecomment-2102905192 Seems like for MySQL indexes for FK already exists -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [I] Unable to see dag_id and task_id tagging for mentioned metrics [airflow]

2024-05-09 Thread via GitHub
RNHTTR commented on issue #39522: URL: https://github.com/apache/airflow/issues/39522#issuecomment-2102933102 @dirrao can you take a look? Also, it would probably be a good idea to add to the documentation what metrics tagging even is -- This is an automated message from the Apache

Re: [PR] Add Scarf based telemetry [airflow]

2024-05-09 Thread via GitHub
arjundevarajan commented on code in PR #39510: URL: https://github.com/apache/airflow/pull/39510#discussion_r1595652416 ## airflow/config_templates/config.yml: ## @@ -2591,3 +2591,22 @@ sensors: type: float example: ~ default: "604800" +scarf_analytics: + d

Re: [PR] Configure HttpHook's auth_type from Connection [airflow]

2024-05-09 Thread via GitHub
dabla commented on PR #35591: URL: https://github.com/apache/airflow/pull/35591#issuecomment-2102970449 It’s a bit hard to split as all modifications are linked to each other. I was already happy I was able to pull it off imho ☺️ -- This is an automated message from the Apache Git Servic

[PR] Fix `xmlsec` version pin [airflow]

2024-05-09 Thread via GitHub
jedcunningham opened a new pull request, #39528: URL: https://github.com/apache/airflow/pull/39528 `xmlsec` is only used when the `python3-saml` additional extra is used in the Amazon provider. We don't need it as a requirement for the Amazon provider in general. -- This is an automated

Re: [I] Ensure teardown doesn't run until all other tasks complete [airflow]

2024-05-09 Thread via GitHub
dstandish commented on issue #29332: URL: https://github.com/apache/airflow/issues/29332#issuecomment-2103011886 looks like what we need to do is find all the tasks “in the scope” of the teardown _(and that is, those tasks that are in between the teardown and its setup(s))_ an

Re: [PR] Add task failed dependencies to details page. [airflow]

2024-05-09 Thread via GitHub
bbovenzi commented on PR #38449: URL: https://github.com/apache/airflow/pull/38449#issuecomment-2103044992 > @jscheffl Thanks, updated the version number and regenerated the files. I have looked into auto refresh and found a pattern where auto refresh is done based on `useAutoRefresh`. The

[I] Task Instance Note accordion doesn't collapse [airflow]

2024-05-09 Thread via GitHub
RNHTTR opened a new issue, #39529: URL: https://github.com/apache/airflow/issues/39529 ### Apache Airflow version 2.9.1 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? Pressing the chevron to collapse task instance note

Re: [PR] Add Scarf based telemetry [airflow]

2024-05-09 Thread via GitHub
jedcunningham commented on code in PR #39510: URL: https://github.com/apache/airflow/pull/39510#discussion_r1595725147 ## airflow/config_templates/config.yml: ## @@ -2591,3 +2591,22 @@ sensors: type: float example: ~ default: "604800" +scarf_analytics: + de

Re: [PR] Resolve deprecations in `docker` provider tests [airflow]

2024-05-09 Thread via GitHub
Taragolis merged PR #39515: URL: https://github.com/apache/airflow/pull/39515 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Resolve deprecations in always connections tests [airflow]

2024-05-09 Thread via GitHub
Taragolis merged PR #39506: URL: https://github.com/apache/airflow/pull/39506 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated (6455f7f92f -> 8940e16d2a)

2024-05-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 6455f7f92f Resolve DbtCloudJobRunAsyncSensor deprecation in system tests (#39525) add 8940e16d2a Resolve depre

Re: [PR] Resolve deprecations in Other tests [airflow]

2024-05-09 Thread via GitHub
Taragolis merged PR #39507: URL: https://github.com/apache/airflow/pull/39507 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated: Resolve deprecations in Other tests (#39507)

2024-05-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 11057453c3 Resolve deprecations in Other tests (

Re: [PR] Resolve deprecations in `apache.livy` provider tests [airflow]

2024-05-09 Thread via GitHub
Taragolis merged PR #39516: URL: https://github.com/apache/airflow/pull/39516 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated: Resolve deprecations in `docker` provider tests (#39515)

2024-05-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5c6d379fd9 Resolve deprecations in `docker` prov

(airflow) branch main updated: Resolve deprecations in `apache.livy` provider tests (#39516)

2024-05-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 9fa2d6d1a6 Resolve deprecations in `apache.livy`

Re: [PR] Resolve deprecations in `atlassian.jira` provider tests [airflow]

2024-05-09 Thread via GitHub
Taragolis merged PR #39517: URL: https://github.com/apache/airflow/pull/39517 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Adding note for provider dependencies [airflow]

2024-05-09 Thread via GitHub
amoghrajesh merged PR #39512: URL: https://github.com/apache/airflow/pull/39512 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airf

(airflow) branch main updated (9fa2d6d1a6 -> 8d6e190b65)

2024-05-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 9fa2d6d1a6 Resolve deprecations in `apache.livy` provider tests (#39516) add 8d6e190b65 Resolve deprecations i

(airflow) branch main updated (8d6e190b65 -> 6fbcaaad36)

2024-05-09 Thread amoghdesai
This is an automated email from the ASF dual-hosted git repository. amoghdesai pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 8d6e190b65 Resolve deprecations in `atlassian.jira` provider tests (#39517) add 6fbcaaad36 Adding note for pr

Re: [PR] Resolve deprecations in `slack` provider tests [airflow]

2024-05-09 Thread via GitHub
Taragolis merged PR #39518: URL: https://github.com/apache/airflow/pull/39518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [PR] Add Scarf based telemetry [airflow]

2024-05-09 Thread via GitHub
jedcunningham commented on code in PR #39510: URL: https://github.com/apache/airflow/pull/39510#discussion_r1595738375 ## airflow/config_templates/config.yml: ## @@ -2591,3 +2591,22 @@ sensors: type: float example: ~ default: "604800" +scarf_analytics: + de

(airflow) branch main updated: Resolve deprecations in `slack` provider tests (#39518)

2024-05-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f6098a46e2 Resolve deprecations in `slack` provi

Re: [PR] Resolve deprecations in `vertica` provider tests [airflow]

2024-05-09 Thread via GitHub
Taragolis merged PR #39519: URL: https://github.com/apache/airflow/pull/39519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated (f6098a46e2 -> 4f13f1b7db)

2024-05-09 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from f6098a46e2 Resolve deprecations in `slack` provider tests (#39518) add 4f13f1b7db Resolve deprecations in `vert

Re: [I] Compatiblity wiht xmlsec 1.3.14 for Amazon Provider [airflow]

2024-05-09 Thread via GitHub
ferruzzi commented on issue #39103: URL: https://github.com/apache/airflow/issues/39103#issuecomment-2103087169 @potiuk - I'm sorry, I got distracted by a shiny new thing and completely forgot about this after you tagged me @jedcunningham - Thanks for fixing it. -- This is an autom

Re: [PR] Log info about task instance clearing/marking as succcess/failure [airflow]

2024-05-09 Thread via GitHub
ephraimbuddy commented on PR #39440: URL: https://github.com/apache/airflow/pull/39440#issuecomment-2103099381 > This broke REST API clearing/marking as failed/success. Still investigating This has been resolved -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Add indexes on dag_id column in refencing tables to speed up deletion of dag records [airflow]

2024-05-09 Thread via GitHub
pankajkoti commented on PR #39526: URL: https://github.com/apache/airflow/pull/39526#issuecomment-2103105096 > Seems like for MySQL indexes for FK already exists Wondering if I should put these operations under ``` if conn.dialect.name != "mysql": create_index ```

Re: [I] apache-airflow-providers-amazon added xmlsec as new dependency and pinned to a version that doesn't have wheels for new python versions [airflow]

2024-05-09 Thread via GitHub
potiuk commented on issue #39437: URL: https://github.com/apache/airflow/issues/39437#issuecomment-2103123923 Nope - it's been added specifically on request and by @vincbeck - https://github.com/apache/airflow/pull/35488 to support authentication for AWS -- This is an automated message

Re: [I] apache-airflow-providers-amazon added xmlsec as new dependency and pinned to a version that doesn't have wheels for new python versions [airflow]

2024-05-09 Thread via GitHub
potiuk commented on issue #39437: URL: https://github.com/apache/airflow/issues/39437#issuecomment-2103134101 And yes - it's likely a bit complex but It means that libxmlsec1 (system library) that is used and apparently needed by Amazon provider is clashing with llbxml python bindings that

Re: [I] apache-airflow-providers-amazon added xmlsec as new dependency and pinned to a version that doesn't have wheels for new python versions [airflow]

2024-05-09 Thread via GitHub
potiuk commented on issue #39437: URL: https://github.com/apache/airflow/issues/39437#issuecomment-2103138304 The whole problem will be solved IF amazon internal authentication (I guess) can use libxmlsec2 . But I have no idea if this is possible - if so - then follow-up to https://github.c

Re: [I] apache-airflow-providers-amazon added xmlsec as new dependency and pinned to a version that doesn't have wheels for new python versions [airflow]

2024-05-09 Thread via GitHub
jedcunningham commented on issue #39437: URL: https://github.com/apache/airflow/issues/39437#issuecomment-2103140092 Yep, that all makes sense. I'm not suggesting #39528 is the long term fix. This just brings us back to status-quo before the new xmlsec release broke stuff (xmlsec is optiona

Re: [I] apache-airflow-providers-amazon added xmlsec as new dependency and pinned to a version that doesn't have wheels for new python versions [airflow]

2024-05-09 Thread via GitHub
jedcunningham commented on issue #39437: URL: https://github.com/apache/airflow/issues/39437#issuecomment-2103143580 > While yes - you can still build the wheel with installing the packages you mentioned, it's a blocker or unnecessary hurdle for people who would like to install amazon provi

Re: [I] apache-airflow-providers-amazon added xmlsec as new dependency and pinned to a version that doesn't have wheels for new python versions [airflow]

2024-05-09 Thread via GitHub
potiuk commented on issue #39437: URL: https://github.com/apache/airflow/issues/39437#issuecomment-2103149149 The question is , whether Amazon provider works with libxmlsec2 at all ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [I] apache-airflow-providers-amazon added xmlsec as new dependency and pinned to a version that doesn't have wheels for new python versions [airflow]

2024-05-09 Thread via GitHub
potiuk commented on issue #39437: URL: https://github.com/apache/airflow/issues/39437#issuecomment-2103153517 Because if it does not, this is not a solution to the problem, it will allow to upgrade xmlsec to 1.3.14, but the specific authentication **might** (and likely will not) work for Am

Re: [I] apache-airflow-providers-amazon added xmlsec as new dependency and pinned to a version that doesn't have wheels for new python versions [airflow]

2024-05-09 Thread via GitHub
potiuk commented on issue #39437: URL: https://github.com/apache/airflow/issues/39437#issuecomment-2103157867 And yeah. I have no idea what it impact it will have - but if we "solve" it now without investigating and possibly bumping libxmlsec2 it might mask the issue - that's all I have to

Re: [PR] Fix: Reuse ProcessPoolExecutor in CeleryExecutor [airflow]

2024-05-09 Thread via GitHub
vatsrahul1001 commented on PR #39484: URL: https://github.com/apache/airflow/pull/39484#issuecomment-2103162890 We conducted a throughput benchmarking test, executing 15k DAG's comprised of simple BashOperator tasks. We measured the success rate of tasks completed within a 2-minute interval

Re: [PR] Add support for cancelling job when deferrable Databricks operator is killed [airflow]

2024-05-09 Thread via GitHub
akaul commented on PR #39373: URL: https://github.com/apache/airflow/pull/39373#issuecomment-2103183950 PR's been updated to inspect for the task instance's state before cancelling the Databricks job. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Fix `xmlsec` version pin [airflow]

2024-05-09 Thread via GitHub
potiuk commented on PR #39528: URL: https://github.com/apache/airflow/pull/39528#issuecomment-2103203725 BTW. It's rather strange that the build does not fail now. I wonder if there is some caching effect - because 3 weeks ago (we do not see the outputs any more) - libxmlsec 1.3.14 caused t

Re: [I] DAGs are able to see historical dataset events when created new [airflow]

2024-05-09 Thread via GitHub
tosheer commented on issue #39456: URL: https://github.com/apache/airflow/issues/39456#issuecomment-2103205004 @RNHTTR i have updated the dag definition as i was reproducing issue on local with a more simpler DAG definition. can you please try again and see if you can reproduce the issue.

Re: [PR] Configure HttpHook's auth_type from Connection [airflow]

2024-05-09 Thread via GitHub
potiuk commented on PR #35591: URL: https://github.com/apache/airflow/pull/35591#issuecomment-2103210925 > It’s a bit hard to split as all modifications are linked to each other. I was already happy I was able to pull it off imho ☺️ I know, but maybe worth trying. There is a lot to re

Re: [I] apache-airflow-providers-amazon added xmlsec as new dependency and pinned to a version that doesn't have wheels for new python versions [airflow]

2024-05-09 Thread via GitHub
jedcunningham commented on issue #39437: URL: https://github.com/apache/airflow/issues/39437#issuecomment-2103213943 To state it another way, your concern is if folks are installing like this: ``` apache-airflow-providers-amazon python3-saml ``` Not: ``` apach

Re: [PR] ADD prototype version dark mode for Airflow UI [airflow]

2024-05-09 Thread via GitHub
jscheffl commented on code in PR #39355: URL: https://github.com/apache/airflow/pull/39355#discussion_r1595836314 ## airflow/www/templates/appbuilder/navbar.html: ## @@ -50,6 +50,13 @@ {% include 'appbuilder/navbar_menu.html' %} + Review Comment:

Re: [I] DAGs are able to see historical dataset events when created new [airflow]

2024-05-09 Thread via GitHub
tosheer commented on issue #39456: URL: https://github.com/apache/airflow/issues/39456#issuecomment-2103222163 Logs ``` [2024-05-09, 18:24:03 UTC] {local_task_job_runner.py:120} ▶ Pre task execution logs [2024-05-09, 18:24:04 UTC] {subprocess.py:63} INFO - Tmp dir root location: /t

Re: [PR] Fix `xmlsec` version pin [airflow]

2024-05-09 Thread via GitHub
jedcunningham commented on PR #39528: URL: https://github.com/apache/airflow/pull/39528#issuecomment-2103227024 My CI image still ends up with `1.3.13`, so it must be built using the extra. That's my theory at least :) -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Add Scarf based telemetry [airflow]

2024-05-09 Thread via GitHub
kaxil commented on code in PR #39510: URL: https://github.com/apache/airflow/pull/39510#discussion_r1595845591 ## airflow/config_templates/config.yml: ## @@ -2591,3 +2591,22 @@ sensors: type: float example: ~ default: "604800" +scarf_analytics: + descriptio

Re: [PR] Fix `xmlsec` version pin [airflow]

2024-05-09 Thread via GitHub
jedcunningham commented on PR #39528: URL: https://github.com/apache/airflow/pull/39528#issuecomment-2103231381 Seems we also have a saml extra: https://github.com/apache/airflow/blob/4f13f1b7dbe14033a5ede747c66cbe7e1b933041/hatch_build.py#L134 Should we add the xmlsec pin ther

[PR] openlineage: migrate OpenLineage provider to V2 facets. [airflow]

2024-05-09 Thread via GitHub
JDarDagran opened a new pull request, #39530: URL: https://github.com/apache/airflow/pull/39530 In https://github.com/OpenLineage/OpenLineage/pull/2520 released in `openlineage-python==1.13.1` there were introduced V2 classes for facets and events. This PR aims to migrate ex

  1   2   3   >