(airflow) branch main updated (7622192441 -> e92cf81805)

2024-08-16 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 7622192441 Fix `ElasticsearchSQLHook` fails with `AttributeError: __enter__` (#41537) add e92cf81805 Fix `AwsTas

Re: [I] EcsRunTaskOperator log output in UI does not contain all events [airflow]

2024-08-16 Thread via GitHub
eladkal closed issue #40875: EcsRunTaskOperator log output in UI does not contain all events URL: https://github.com/apache/airflow/issues/40875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] Fix `AwsTaskLogFetcher` missing logs [airflow]

2024-08-16 Thread via GitHub
eladkal merged PR #41515: URL: https://github.com/apache/airflow/pull/41515 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [PR] Added logging device and logging device options [airflow]

2024-08-16 Thread via GitHub
geraj1010 commented on code in PR #41416: URL: https://github.com/apache/airflow/pull/41416#discussion_r1720610991 ## airflow/providers/docker/operators/docker_swarm.py: ## @@ -116,6 +126,8 @@ def __init__( networks: list[str | types.NetworkAttachmentConfig] | None = No

Re: [PR] Added logging device and logging device options [airflow]

2024-08-16 Thread via GitHub
geraj1010 commented on PR #41416: URL: https://github.com/apache/airflow/pull/41416#issuecomment-2294633490 @Lee-W I have added two unit tests for checking if `logging_driver` works and for entering an invalid/unsupported `logging_driver` value. -- This is an automated message from th

Re: [PR] Added logging device and logging device options [airflow]

2024-08-16 Thread via GitHub
geraj1010 commented on code in PR #41416: URL: https://github.com/apache/airflow/pull/41416#discussion_r1720610991 ## airflow/providers/docker/operators/docker_swarm.py: ## @@ -116,6 +126,8 @@ def __init__( networks: list[str | types.NetworkAttachmentConfig] | None = No

Re: [I] Import error of new dags cannot be shown on web UI for common users [airflow]

2024-08-16 Thread via GitHub
bmoon4 commented on issue #29897: URL: https://github.com/apache/airflow/issues/29897#issuecomment-2294622876 Been observing this issue for long time ... and my team ended up building a dashboard(Grafana) which uses Airflow API calls to get import error data. -- This is an automated mess

(airflow) branch main updated (8c2f56375a -> 7622192441)

2024-08-16 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 8c2f56375a Skip task in example_dynamodb_to_s3.py (#41546) add 7622192441 Fix `ElasticsearchSQLHook` fails with `

Re: [I] ElasticsearchSQLHook fails with AttributeError: __enter__ [airflow]

2024-08-16 Thread via GitHub
eladkal closed issue #41486: ElasticsearchSQLHook fails with AttributeError: __enter__ URL: https://github.com/apache/airflow/issues/41486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] add execute_sql API to fix ElasticSearchSQLHook [airflow]

2024-08-16 Thread via GitHub
eladkal merged PR #41537: URL: https://github.com/apache/airflow/pull/41537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [I] Switch from 'cloudant' to 'ibmcloudant' [airflow]

2024-08-16 Thread via GitHub
topherinternational commented on issue #21004: URL: https://github.com/apache/airflow/issues/21004#issuecomment-2294614693 @uranusjr @potiuk I took a look at this as a good-first-issue, and I think there's a PMC-level question to be answered before I want to start writing any code. T

[PR] Fix: Keep compatibility with old FAB versions [airflow]

2024-08-16 Thread via GitHub
joaopamaral opened a new pull request, #41549: URL: https://github.com/apache/airflow/pull/41549 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rs

Re: [PR] restore python 3.12 support for papermill [airflow]

2024-08-16 Thread via GitHub
boring-cyborg[bot] commented on PR #41548: URL: https://github.com/apache/airflow/pull/41548#issuecomment-2294591349 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

[PR] restore python 3.12 support for papermill [airflow]

2024-08-16 Thread via GitHub
morokosi opened a new pull request, #41548: URL: https://github.com/apache/airflow/pull/41548 The papermill provider was previously excluded from releasing a version compatible with Python 3.12 due to version compatibility issues with the aiohttp dependency. However, papermi

Re: [I] Display key in dynamic task mapping using dict in UI view of Mapped Tasks' map index [airflow]

2024-08-16 Thread via GitHub
fredthomsen commented on issue #39390: URL: https://github.com/apache/airflow/issues/39390#issuecomment-2294514118 @kiaradlf yes that is correct. If that is your use case then `map_index_template` will not help you. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Add possibility to override the conn type for Druid [airflow]

2024-08-16 Thread via GitHub
github-actions[bot] closed pull request #39897: Add possibility to override the conn type for Druid URL: https://github.com/apache/airflow/pull/39897 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Fix problem with inability to remove fields from Connection form [airflow]

2024-08-16 Thread via GitHub
github-actions[bot] commented on PR #40421: URL: https://github.com/apache/airflow/pull/40421#issuecomment-2294473295 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] Revamp Contributor's quick start [airflow]

2024-08-16 Thread via GitHub
github-actions[bot] commented on PR #39837: URL: https://github.com/apache/airflow/pull/39837#issuecomment-2294473357 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] [AIP-49] OpenTelemetry Traces for Apache Airflow Part 2 [airflow]

2024-08-16 Thread via GitHub
howardyoo commented on code in PR #40802: URL: https://github.com/apache/airflow/pull/40802#discussion_r1720477207 ## airflow/executors/base_executor.py: ## @@ -302,15 +319,40 @@ def trigger_tasks(self, open_slots: int) -> None: if key in self.attempts:

Re: [I] LambdaInvokeFunctionOperator throwing ReadTimeout error even when the actual lambda invocation completed within the time limits [airflow]

2024-08-16 Thread via GitHub
gopidesupavan commented on issue #41498: URL: https://github.com/apache/airflow/issues/41498#issuecomment-2294438830 Hi @rawwar i have tried the steps you have mentioned from my local, but not been able to re produce the issue, in my case its getting succeeded. is there any other informatio

Re: [I] Access control error after upgrading from version 2.9.3 to 2.10.0 [airflow]

2024-08-16 Thread via GitHub
joaopamaral commented on issue #41540: URL: https://github.com/apache/airflow/issues/41540#issuecomment-2294409786 FYI, the error is that we are changing the access_control format (to include the resource name) in airflow 2.10, but old FAB version doesn't know how to handle it. I'll include

Re: [PR] [AIP-49] OpenTelemetry Traces for Apache Airflow Part 2 [airflow]

2024-08-16 Thread via GitHub
ashb commented on code in PR #40802: URL: https://github.com/apache/airflow/pull/40802#discussion_r1720411361 ## airflow/executors/base_executor.py: ## @@ -302,15 +319,40 @@ def trigger_tasks(self, open_slots: int) -> None: if key in self.attempts:

Re: [I] Access control error after upgrading from version 2.9.3 to 2.10.0 [airflow]

2024-08-16 Thread via GitHub
joaopamaral commented on issue #41540: URL: https://github.com/apache/airflow/issues/41540#issuecomment-2294361453 Hey @FelipeRamos-neuro, can you confirm if this error vanishes if you navigate to a different link and back to dags? E.g try to go to dataset and navigate back to the dags?

Re: [I] Access control error after upgrading from version 2.9.3 to 2.10.0 [airflow]

2024-08-16 Thread via GitHub
joaopamaral commented on issue #41540: URL: https://github.com/apache/airflow/issues/41540#issuecomment-2294349121 Hey @FelipeRamos-neuro, I was able to reproduce the error. I'll open the PR to fix it. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Skip task in example_dynamodb_to_s3.py [airflow]

2024-08-16 Thread via GitHub
potiuk merged PR #41546: URL: https://github.com/apache/airflow/pull/41546 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Skip task in example_dynamodb_to_s3.py (#41546)

2024-08-16 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8c2f56375a Skip task in example_dynamodb_to_s3.py (

(airflow) branch constraints-2-10 updated: Updating constraints. Github run id:10425175902

2024-08-16 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-10 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-10 by this push: new 89843b3313 Updating con

Re: [PR] Fix UI rendering issues if XCom values are INT, FLOAT or BOOL in web UI [airflow]

2024-08-16 Thread via GitHub
jscheffl commented on code in PR #41516: URL: https://github.com/apache/airflow/pull/41516#discussion_r1720260008 ## airflow/www/static/js/components/RenderedJsonField.tsx: ## @@ -47,7 +47,7 @@ const JsonParse = (content: string | object) => { contentJson = content;

(airflow) branch v2-10-test updated: Fix Non-DB test calculation for main builds (#41499) (#41543)

2024-08-16 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-10-test by this push: new e36e521dcb Fix Non-DB test calculation

Re: [PR] Fix Non-DB test calculation for main builds (#41499) [airflow]

2024-08-16 Thread via GitHub
potiuk merged PR #41543: URL: https://github.com/apache/airflow/pull/41543 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[I] `airflow db migrate` throws dataset alias reference columns and Cryptographic Paramik warnings [airflow]

2024-08-16 Thread via GitHub
anthony-joyce opened a new issue, #41547: URL: https://github.com/apache/airflow/issues/41547 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.10 ### What happened? After upgrad

Re: [PR] Add incremental export and cross account export functionality in `DynamoDBToS3Operator` [airflow]

2024-08-16 Thread via GitHub
ferruzzi commented on code in PR #41304: URL: https://github.com/apache/airflow/pull/41304#discussion_r1720242191 ## tests/system/providers/amazon/aws/example_dynamodb_to_s3.py: ## @@ -195,7 +225,8 @@ def delete_dynamodb_table(table_name: str): backup_db_segment_1,

Re: [PR] Add incremental export and cross account export functionality in `DynamoDBToS3Operator` [airflow]

2024-08-16 Thread via GitHub
ferruzzi commented on code in PR #41304: URL: https://github.com/apache/airflow/pull/41304#discussion_r1720240078 ## tests/system/providers/amazon/aws/example_dynamodb_to_s3.py: ## @@ -162,18 +171,39 @@ def delete_dynamodb_table(table_name: str): # [END howto_transfer_dynam

Re: [PR] Remove Database Isolation feature flag and run DB isolation tests [airflow]

2024-08-16 Thread via GitHub
jscheffl commented on PR #41067: URL: https://github.com/apache/airflow/pull/41067#issuecomment-2294028892 > > Yeah. This is a concern. Should we have a branch called v2-dev where all airflow 2 PRs will go into, then the release manager would cherry-pick to v2-10-test when releasing? I thin

[PR] Skip task in example_dynamodb_to_s3.py [airflow]

2024-08-16 Thread via GitHub
ferruzzi opened a new pull request, #41546: URL: https://github.com/apache/airflow/pull/41546 `backup_db_to_point_in_time_incremental_export` requires at least a 15-minute window of available data, causing the system tests to fail. Moving it into a branching task group will let us keep the

[PR] Add specific Slack operators [airflow]

2024-08-16 Thread via GitHub
fritz-astronomer opened a new pull request, #41545: URL: https://github.com/apache/airflow/pull/41545 Adding `*SlackOperator` to perform specific, common, API actions --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://gith

Re: [PR] added a new condition before launching the self._scan_stale_dags() fu… [airflow]

2024-08-16 Thread via GitHub
tchakib commented on PR #39159: URL: https://github.com/apache/airflow/pull/39159#issuecomment-2294005048 Hello @ephraimbuddy, Can we know the reason for postponing this future version to 2.11 instead of 2.10?” I’m asking because we’re eagerly awaiting this future version in order

Re: [PR] Provider Airbyte: breaking change, update provider to use Airbyte API Python SDK [airflow]

2024-08-16 Thread via GitHub
potiuk commented on PR #41122: URL: https://github.com/apache/airflow/pull/41122#issuecomment-2293987742 You need to rebase/run pre-commit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Add Backfill Button to DAGs Overview and Individual DAG Pages in the Webserver UI [airflow]

2024-08-16 Thread via GitHub
potiuk closed issue #41542: Add Backfill Button to DAGs Overview and Individual DAG Pages in the Webserver UI URL: https://github.com/apache/airflow/issues/41542 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [I] Add Backfill Button to DAGs Overview and Individual DAG Pages in the Webserver UI [airflow]

2024-08-16 Thread via GitHub
potiuk commented on issue #41542: URL: https://github.com/apache/airflow/issues/41542#issuecomment-2293947151 This is far more than addinga "button". This is planned for Airflow 3 and the whole Airlfow Improvement Proposal about how to implement it https://cwiki.apache.org/confluence/displa

Re: [PR] Fix Non-DB test calculation for main builds (#41499) [airflow]

2024-08-16 Thread via GitHub
potiuk commented on PR #41543: URL: https://github.com/apache/airflow/pull/41543#issuecomment-2293944034 BAckporting pytest8 workaroud to v2-10-test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[PR] Fix Non-DB test calculation for main builds (#41499) [airflow]

2024-08-16 Thread via GitHub
potiuk opened a new pull request, #41543: URL: https://github.com/apache/airflow/pull/41543 Pytest has a weird behaviour that it will not collect tests from parent folder when subfolder of it is specified after the parent folder. This caused some non-db tests from providers folder have been

(airflow) branch v2-10-test updated: Fix failing pydantic v1 tests (#41534) (#41541)

2024-08-16 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-10-test by this push: new fa4ee68da1 Fix failing pydantic v1 test

Re: [PR] Fix failing pydantic v1 tests (#41534) [airflow]

2024-08-16 Thread via GitHub
potiuk merged PR #41541: URL: https://github.com/apache/airflow/pull/41541 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[I] Add Backfill Button to DAGs Overview and Individual DAG Pages in the Webserver UI [airflow]

2024-08-16 Thread via GitHub
pedro-cf opened a new issue, #41542: URL: https://github.com/apache/airflow/issues/41542 ### Description This feature request proposes the addition of a "Backfill" button to the Apache Airflow webserver UI, both on the DAGs overview page and within individual DAG pages. This button w

[PR] Fix failing pydantic v1 tests (#41534) [airflow]

2024-08-16 Thread via GitHub
potiuk opened a new pull request, #41541: URL: https://github.com/apache/airflow/pull/41541 We need to exclude some versions of Pydantic v1 because it conflicts with aws provider. (cherry picked from commit a033c5f15a033c751419506ea77ffdbacdd37705) -

Re: [PR] Add SIGHUP signal before SIGTERM for task runner termination [airflow]

2024-08-16 Thread via GitHub
potiuk commented on PR #41329: URL: https://github.com/apache/airflow/pull/41329#issuecomment-2293889509 IF I am not mistaken - this one also adds timeout in supervising process. Was that intentional ? (the commint message only mentions adding SIGHUP/SIGTERM) ? -- This is an automated me

Re: [PR] Fix running on_success_callback if `AirflowSkipException` raised [airflow]

2024-08-16 Thread via GitHub
eladkal commented on code in PR #40936: URL: https://github.com/apache/airflow/pull/40936#discussion_r1720119586 ## newsfragments/40936.bugfix.rst: ## @@ -0,0 +1 @@ +Fix ``on_success_callback`` will not run if task is skipped. Review Comment: We need more details here. It do

Re: [PR] Add support for executing multiple sequential sql statements in AthenaOperator [defer support] [airflow]

2024-08-16 Thread via GitHub
eladkal commented on PR #41511: URL: https://github.com/apache/airflow/pull/41511#issuecomment-2293878954 > Note: I haven’t worked on tests or docs yet, I’ll proceed with those once we’re happy with the implementation. If now you belive we need both then what I am looking for is how t

Re: [PR] Airflow 3: store_serialized_dags deprecated parameter usage removal [airflow]

2024-08-16 Thread via GitHub
potiuk merged PR #40931: URL: https://github.com/apache/airflow/pull/40931 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Refactor DataprocCreateBatchOperator and Dataproc system tests [airflow]

2024-08-16 Thread via GitHub
potiuk merged PR #41527: URL: https://github.com/apache/airflow/pull/41527 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Delete experimental API [airflow]

2024-08-16 Thread via GitHub
vincbeck merged PR #41434: URL: https://github.com/apache/airflow/pull/41434 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] Remove Database Isolation feature flag and run DB isolation tests [airflow]

2024-08-16 Thread via GitHub
potiuk commented on PR #41067: URL: https://github.com/apache/airflow/pull/41067#issuecomment-2293830073 > Yeah. This is a concern. Should we have a branch called v2-dev where all airflow 2 PRs will go into, then the release manager would cherry-pick to v2-10-test when releasing? I think th

Re: [PR] Feature: Allow set Dag Run resource into Dag Level permission [airflow]

2024-08-16 Thread via GitHub
potiuk commented on PR #40703: URL: https://github.com/apache/airflow/pull/40703#issuecomment-2293813085 I think that one caused problem https://github.com/apache/airflow/issues/41540#issuecomment-2293802672 - it does not seem backwards-compatible with the previous fab provider. -- This

Re: [I] Access control error after upgrading from version 2.9.3 to 2.10.0 [airflow]

2024-08-16 Thread via GitHub
FelipeRamos-neuro commented on issue #41540: URL: https://github.com/apache/airflow/issues/41540#issuecomment-2293802672 Some prints I forgot to add to the issue ![Captura de tela de 2024-08-16 12-41-16](https://github.com/user-attachments/assets/450b864b-d9c1-4fff-be3b-74f818d91d66)

(airflow) branch v2-10-test updated: Incorrect try number subtraction producing invalid span id for OTEL airflow (issue #41501) (#41502) (#41535)

2024-08-16 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-10-test by this push: new 30cfb7345a Incorrect try number subtrac

Re: [PR] Incorrect try number subtraction producing invalid span id for OTEL a… [airflow]

2024-08-16 Thread via GitHub
potiuk merged PR #41535: URL: https://github.com/apache/airflow/pull/41535 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] [WIP] Migrate core `EmailOperator` to SMTP provider [airflow]

2024-08-16 Thread via GitHub
eladkal commented on PR #30531: URL: https://github.com/apache/airflow/pull/30531#issuecomment-2293767340 @hussein-awala i think now that main branch is Airflow 3 this PR should be easier -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[I] Access control error after upgrading from version 2.9.3 to 2.10.0 [airflow]

2024-08-16 Thread via GitHub
FelipeRamos-neuro opened a new issue, #41540: URL: https://github.com/apache/airflow/issues/41540 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.10.0 ### What happened? After

Re: [PR] Delete experimental API [airflow]

2024-08-16 Thread via GitHub
vincbeck commented on PR #41434: URL: https://github.com/apache/airflow/pull/41434#issuecomment-2293763491 All green :) Any additional reviews? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Incorrect try number subtraction producing invalid span id for OTEL a… [airflow]

2024-08-16 Thread via GitHub
ephraimbuddy commented on PR #41535: URL: https://github.com/apache/airflow/pull/41535#issuecomment-2293763606 > cc: @utkarsharma2 @kaxil @ephraimbuddy -> should we mark both PRs with "Milestone 2.10.1" - the one in main and the backport one? WDYT? I think only the backport is enough

Re: [PR] add execute_sql API to fix ElasticSearchSQLHook [airflow]

2024-08-16 Thread via GitHub
Owen-CH-Leung commented on PR #41537: URL: https://github.com/apache/airflow/pull/41537#issuecomment-2293756831 I think the 3 failing tests are `Tests / Basic tests / Test Airflow release commands` , `Tests / Provider checks / Provider packages sdist build and install` , `Tests / Integratio

[PR] smtp email user and password deprecated config removal [airflow]

2024-08-16 Thread via GitHub
dirrao opened a new pull request, #41539: URL: https://github.com/apache/airflow/pull/41539 smtp email user and password deprecated config removal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Removed deprecated parameters from AzureKeyVaultBackend [airflow]

2024-08-16 Thread via GitHub
gopidesupavan commented on code in PR #41536: URL: https://github.com/apache/airflow/pull/41536#discussion_r1720018237 ## newsfragments/41536.significant.rst: ## @@ -0,0 +1,15 @@ +**Breaking Change** + +The ``get_conn_uri`` method has been removed from the ``AzureKeyVaultBackend

Re: [PR] Add support for executing multiple sql statements in AthenaOperator [defer support] [airflow]

2024-08-16 Thread via GitHub
gopidesupavan commented on code in PR #41511: URL: https://github.com/apache/airflow/pull/41511#discussion_r1720012373 ## airflow/providers/amazon/aws/operators/athena.py: ## @@ -126,63 +130,97 @@ def execute(self, context: Context) -> str | None: self.query_execution_c

Re: [PR] Removed deprecated parameters from AzureKeyVaultBackend [airflow]

2024-08-16 Thread via GitHub
gopidesupavan commented on code in PR #41536: URL: https://github.com/apache/airflow/pull/41536#discussion_r1720008280 ## newsfragments/41536.significant.rst: ## @@ -0,0 +1,15 @@ +**Breaking Change** + +The ``get_conn_uri`` method has been removed from the ``AzureKeyVaultBackend

Re: [PR] Add support for executing multiple sql statements in AthenaOperator [defer support] [airflow]

2024-08-16 Thread via GitHub
gopidesupavan commented on code in PR #41511: URL: https://github.com/apache/airflow/pull/41511#discussion_r1720006098 ## airflow/providers/amazon/aws/operators/athena.py: ## @@ -126,63 +130,97 @@ def execute(self, context: Context) -> str | None: self.query_execution_c

Re: [I] AttributeError: 'CeleryKubernetesExecutor' object has no attribute 'slots_occupied' [airflow]

2024-08-16 Thread via GitHub
potiuk commented on issue #41525: URL: https://github.com/apache/airflow/issues/41525#issuecomment-2293719334 > @potiuk what steps need to be followed for us to deprecate these old hybrid executors for airflow 3.0? Or is that not possible because they are in provider packages We can

Re: [PR] Separate FAB migration from Core Airflow migration [airflow]

2024-08-16 Thread via GitHub
potiuk commented on PR #41437: URL: https://github.com/apache/airflow/pull/41437#issuecomment-2293710232 Likely side-effect of other tests that have been somewhat masked or avoided before the change - where some setup/teardown removed the side-effect. You can reproduce the set of test

Re: [PR] Add support for executing multiple sql statements in AthenaOperator [defer support] [airflow]

2024-08-16 Thread via GitHub
vincbeck commented on code in PR #41511: URL: https://github.com/apache/airflow/pull/41511#discussion_r1719988542 ## airflow/providers/amazon/aws/operators/athena.py: ## @@ -126,63 +130,97 @@ def execute(self, context: Context) -> str | None: self.query_execution_contex

Re: [PR] Removed deprecated parameters from AzureKeyVaultBackend [airflow]

2024-08-16 Thread via GitHub
eladkal commented on code in PR #41536: URL: https://github.com/apache/airflow/pull/41536#discussion_r1719984678 ## newsfragments/41536.significant.rst: ## @@ -0,0 +1,15 @@ +**Breaking Change** + +The ``get_conn_uri`` method has been removed from the ``AzureKeyVaultBackend`` cl

Re: [I] AttributeError: 'CeleryKubernetesExecutor' object has no attribute 'slots_occupied' [airflow]

2024-08-16 Thread via GitHub
o-nikolas commented on issue #41525: URL: https://github.com/apache/airflow/issues/41525#issuecomment-2293700446 I'll look into fixing this issue But I encourage folks to use Multiple Executor Configuration instead of these old (and should be deprecated) hybrid executors. These are exactly

Re: [PR] Remove Database Isolation feature flag and run DB isolation tests [airflow]

2024-08-16 Thread via GitHub
ephraimbuddy commented on PR #41067: URL: https://github.com/apache/airflow/pull/41067#issuecomment-2293698324 > > @potiuk good to merge or still hold this back? > > I am struggling with it a bit - because it will likely make it more difficult for contributors to add their tests to ma

Re: [PR] add execute_sql API to fix ElasticSearchSQLHook [airflow]

2024-08-16 Thread via GitHub
eladkal commented on PR #41537: URL: https://github.com/apache/airflow/pull/41537#issuecomment-2293694267 Static checks are failing :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

(airflow) branch constraints-main updated: Updating constraints. Github run id:10420774982

2024-08-16 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 17d4fb2294 Updating con

Re: [I] Deprecation of operators should be prominently highlighted instead of the one line remark in the documentation [airflow]

2024-08-16 Thread via GitHub
kaxil commented on issue #41532: URL: https://github.com/apache/airflow/issues/41532#issuecomment-2293682944 I like your proposed solution @peter-gergely-horvath , would you like to help out with the PR? Might just be a change in this file: https://github.com/apache/airflow/blob/main

Re: [PR] Add support for executing multiple sql statements in AthenaOperator [defer support] [airflow]

2024-08-16 Thread via GitHub
gopidesupavan commented on code in PR #41511: URL: https://github.com/apache/airflow/pull/41511#discussion_r1719969696 ## airflow/providers/amazon/aws/operators/athena.py: ## @@ -126,63 +130,97 @@ def execute(self, context: Context) -> str | None: self.query_execution_c

Re: [PR] Add support for executing multiple sql statements in AthenaOperator [defer support] [airflow]

2024-08-16 Thread via GitHub
gopidesupavan commented on code in PR #41511: URL: https://github.com/apache/airflow/pull/41511#discussion_r1719969696 ## airflow/providers/amazon/aws/operators/athena.py: ## @@ -126,63 +130,97 @@ def execute(self, context: Context) -> str | None: self.query_execution_c

Re: [PR] Remove Database Isolation feature flag and run DB isolation tests [airflow]

2024-08-16 Thread via GitHub
potiuk commented on PR #41067: URL: https://github.com/apache/airflow/pull/41067#issuecomment-2293668414 > @potiuk good to merge or still hold this back? I am struggling with it a bit - because it will likely make it more difficult for contributors to add their tests to main - while w

Re: [I] wait_for_downstream doesn't seem to work as expected [airflow]

2024-08-16 Thread via GitHub
romsharon98 commented on issue #41526: URL: https://github.com/apache/airflow/issues/41526#issuecomment-2293656275 The behavior is as expected. It start a new DagRun because the default max_active_runs is 16. But it does not start the first task until the first and second task don't fi

Re: [PR] Incorrect try number subtraction producing invalid span id for OTEL a… [airflow]

2024-08-16 Thread via GitHub
potiuk commented on PR #41535: URL: https://github.com/apache/airflow/pull/41535#issuecomment-2293648020 cc: @utkarsharma2 @kaxil @ephraimbuddy -> should we mark both PRs with "Milestone 2.10" - the one in main and the backport one? WDYT? -- This is an automated message from the Apache Gi

Re: [PR] Added logging device and logging device options [airflow]

2024-08-16 Thread via GitHub
geraj1010 commented on code in PR #41416: URL: https://github.com/apache/airflow/pull/41416#discussion_r1719940960 ## airflow/providers/docker/operators/docker_swarm.py: ## @@ -116,6 +126,8 @@ def __init__( networks: list[str | types.NetworkAttachmentConfig] | None = No

[PR] add execute_sql API to fix ElasticSearchSQLHook [airflow]

2024-08-16 Thread via GitHub
Owen-CH-Leung opened a new pull request, #41537: URL: https://github.com/apache/airflow/pull/41537 Fixes #41486 The current `ElasticsearchSQLHook` is not working since it is still using `elasticsearch-dbapi` which has already been deprecated last year. This PR fixes the issue by add

Re: [PR] Added logging device and logging device options [airflow]

2024-08-16 Thread via GitHub
geraj1010 commented on PR #41416: URL: https://github.com/apache/airflow/pull/41416#issuecomment-2293616313 @Lee-W I'm not quite sure what kind of unit tests would be valid here, since this is an operator update. The operator doesn't return anything to check. I do have a test DAG if that co

Re: [PR] Added logging device and logging device options [airflow]

2024-08-16 Thread via GitHub
geraj1010 commented on code in PR #41416: URL: https://github.com/apache/airflow/pull/41416#discussion_r1719920651 ## airflow/providers/docker/operators/docker_swarm.py: ## @@ -116,6 +126,8 @@ def __init__( networks: list[str | types.NetworkAttachmentConfig] | None = No

Re: [I] AttributeError: 'CeleryKubernetesExecutor' object has no attribute 'slots_occupied' [airflow]

2024-08-16 Thread via GitHub
potiuk commented on issue #41525: URL: https://github.com/apache/airflow/issues/41525#issuecomment-2293586004 CC: @o-nikolas -> seems to be result of #40017 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] Airflow DAG throws GlueJobOperator is not JSON serializable [airflow]

2024-08-16 Thread via GitHub
mszpot-future-processing commented on issue #41306: URL: https://github.com/apache/airflow/issues/41306#issuecomment-2293567440 @Lee-W - working solution in below post: https://stackoverflow.com/questions/78842962/airflow-dag-throws-gluejoboperator-is-not-json-serializable It was a

Re: [I] AttributeError: 'CeleryKubernetesExecutor' object has no attribute 'slots_occupied' [airflow]

2024-08-16 Thread via GitHub
salam-abdul commented on issue #41525: URL: https://github.com/apache/airflow/issues/41525#issuecomment-2293562045 I'm getting a related error message on Airflow 2.10 and Python 3.11 `AttributeError: 'LocalKubernetesExecutor' object has no attribute 'slots_occupied'` -- This is an auto

Re: [PR] Airflow 3: airflow date utils date_range and days_ago deprecations removal [airflow]

2024-08-16 Thread via GitHub
potiuk merged PR #41496: URL: https://github.com/apache/airflow/pull/41496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Airflow 3: airflow date utils date_range and days_ago deprecations removal (#41496)

2024-08-16 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e35ae2db7c Airflow 3: airflow date utils date_range

(airflow) branch main updated: Util helper deprecated functions removal (#41520)

2024-08-16 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6ba2658aef Util helper deprecated functions remova

Re: [PR] Util helper deprecated functions removal [airflow]

2024-08-16 Thread via GitHub
potiuk merged PR #41520: URL: https://github.com/apache/airflow/pull/41520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] feat: openlineage listener captures hook-level lineage [airflow]

2024-08-16 Thread via GitHub
kacpermuda commented on code in PR #41482: URL: https://github.com/apache/airflow/pull/41482#discussion_r1719869241 ## airflow/providers/openlineage/extractors/manager.py: ## @@ -105,14 +106,25 @@ def extract_metadata(self, dagrun, task, complete: bool = False, task_instance=N

[PR] Removed deprecated parameters from AzureKeyVaultBackend [airflow]

2024-08-16 Thread via GitHub
gopidesupavan opened a new pull request, #41536: URL: https://github.com/apache/airflow/pull/41536 Removed deprecated parameters from AzureKeyVaultBackend --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apach

Re: [I] Airflow Provider for OpenTelemetry [airflow]

2024-08-16 Thread via GitHub
howardyoo commented on issue #37628: URL: https://github.com/apache/airflow/issues/37628#issuecomment-2293498396 Hi folks! As part of the items raised during the discussion in dev mailing list, I updated the document for this issue to further clarify what is the requirements and reasons beh

[PR] Incorrect try number subtraction producing invalid span id for OTEL a… [airflow]

2024-08-16 Thread via GitHub
potiuk opened a new pull request, #41535: URL: https://github.com/apache/airflow/pull/41535 …irflow (issue #41501) (#41502) * Fix for issue #39336 * removed unnecessary import (cherry picked from commit dd3c3a7a43102c967d76cdcfe1f2f8ebeef4e212)

(airflow) branch main updated: Incorrect try number subtraction producing invalid span id for OTEL airflow (issue #41501) (#41502)

2024-08-16 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new dd3c3a7a43 Incorrect try number subtraction produci

Re: [PR] Incorrect try number subtraction producing invalid span id for OTEL airflow (issue #41501) [airflow]

2024-08-16 Thread via GitHub
potiuk merged PR #41502: URL: https://github.com/apache/airflow/pull/41502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Incorrect try number subtraction producing invalid span id for OTEL airflow (issue #41501) [airflow]

2024-08-16 Thread via GitHub
potiuk commented on PR #41502: URL: https://github.com/apache/airflow/pull/41502#issuecomment-2293492313 > I wonder if this is related to the recent(-ish) try semantic changes? Yep. It's in the description - result of #39336 -- This is an automated message from the Apache Git Servi

Re: [I] Incorrect try number producing invalid span id for OTEL airflow [airflow]

2024-08-16 Thread via GitHub
potiuk closed issue #41501: Incorrect try number producing invalid span id for OTEL airflow URL: https://github.com/apache/airflow/issues/41501 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

  1   2   >