Re: [PR] Include removed providers when preparing packages [airflow]

2023-12-11 Thread via GitHub
eladkal commented on code in PR #36160: URL: https://github.com/apache/airflow/pull/36160#discussion_r1423551529 ## dev/README_RELEASE_PROVIDER_PACKAGES.md: ## @@ -164,16 +164,6 @@ Details about maintaining the SEMVER version are going to be discussed and imple breeze release-

Re: [I] Status of testing Providers that were prepared on December 08, 2023 [airflow]

2023-12-11 Thread via GitHub
eladkal closed issue #36117: Status of testing Providers that were prepared on December 08, 2023 URL: https://github.com/apache/airflow/issues/36117 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] Status of testing Providers that were prepared on December 08, 2023 [airflow]

2023-12-11 Thread via GitHub
eladkal commented on issue #36117: URL: https://github.com/apache/airflow/issues/36117#issuecomment-1851438066 Thank you everyone. Providers are released (excluding Google, ODBC, Databricks, Docker which we will have RC2 soon) I invite everyone to help improve providers for the next re

[PR] Update providers metadata 2023-12-12 [airflow]

2023-12-11 Thread via GitHub
eladkal opened a new pull request, #36184: URL: https://github.com/apache/airflow/pull/36184 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)

(airflow-site) branch add-documentation-2023-12-08 deleted (was b2e7d16813)

2023-12-11 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2023-12-08 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was b2e7d16813 remove google, databricks, odbc, docker providers from rc1 The revisions that

Re: [PR] Add documentation for packages - 2023-12-08 [airflow-site]

2023-12-11 Thread via GitHub
eladkal merged PR #905: URL: https://github.com/apache/airflow-site/pull/905 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] Remove duplicate dropdown entry in providers issue template [airflow]

2023-12-11 Thread via GitHub
potiuk commented on PR #36180: URL: https://github.com/apache/airflow/pull/36180#issuecomment-1851422802 Added pre-commit as quick fix (with some more fixes it turned out): https://github.com/apache/airflow/pull/36183 -- This is an automated message from the Apache Git Service. To respond

[PR] Add pre-commmit check for providers list in bug report is unique/sorted [airflow]

2023-12-11 Thread via GitHub
potiuk opened a new pull request, #36183: URL: https://github.com/apache/airflow/pull/36183 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)*

Re: [PR] Initial implementation of new airflow provider for Teradata with Teradata Hook, Teradata Operator and Teradata to Teradata Transfer Operator. [airflow]

2023-12-11 Thread via GitHub
SatishChGit closed pull request #35909: Initial implementation of new airflow provider for Teradata with Teradata Hook, Teradata Operator and Teradata to Teradata Transfer Operator. URL: https://github.com/apache/airflow/pull/35909 -- This is an automated message from the Apache Git Service.

[PR] Bugfix/logging for pausing [airflow]

2023-12-11 Thread via GitHub
romsharon98 opened a new pull request, #36182: URL: https://github.com/apache/airflow/pull/36182 Closes: #33713 Cherry picking @alephmelo --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airf

svn commit: r65990 [1/2] - /dev/airflow/providers/ /release/airflow/providers/

2023-12-11 Thread eladkal
Author: eladkal Date: Tue Dec 12 06:57:03 2023 New Revision: 65990 Log: Release Airflow Providers on 2023-12-12 Added: release/airflow/providers/apache_airflow_providers_airbyte-3.5.0-py3-none-any.whl - copied unchanged from r65989, dev/airflow/providers/apache_airflow_providers_airby

svn commit: r65990 [2/2] - /dev/airflow/providers/ /release/airflow/providers/

2023-12-11 Thread eladkal

Re: [I] Status of testing Providers that were prepared on December 08, 2023 [airflow]

2023-12-11 Thread via GitHub
eladkal commented on issue #36117: URL: https://github.com/apache/airflow/issues/36117#issuecomment-1851385174 Providers: Google, ODBC, Databricks, Docker are excluded from RC1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

(airflow) branch main updated: Added Datascan Profiling (#35696)

2023-12-11 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 47a9c8a4a4 Added Datascan Profiling (#35696) 47a9c8

Re: [PR] Added Datascan Profiling [airflow]

2023-12-11 Thread via GitHub
boring-cyborg[bot] commented on PR #35696: URL: https://github.com/apache/airflow/pull/35696#issuecomment-1851374534 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [PR] Added Datascan Profiling [airflow]

2023-12-11 Thread via GitHub
potiuk merged PR #35696: URL: https://github.com/apache/airflow/pull/35696 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Convert DagFileProcessor.execute_callbacks to Internal API [airflow]

2023-12-11 Thread via GitHub
dstandish commented on code in PR #28900: URL: https://github.com/apache/airflow/pull/28900#discussion_r1423410165 ## airflow/models/dag.py: ## @@ -1383,8 +1384,43 @@ def normalized_schedule_interval(self) -> ScheduleInterval: _schedule_interval = self.schedule_int

Re: [I] AIP-44 Migrate TaskInstance.refresh_from_db to Internal API [airflow]

2023-12-11 Thread via GitHub
dstandish commented on issue #29311: URL: https://github.com/apache/airflow/issues/29311#issuecomment-1851305789 Hi @mhenc I see you've marked this done. Is it done? Is work on it in progress? Are there any particular challenges in this one? -- This is an automated message from the Apach

Re: [PR] Remove duplicate dropdown entry in providers issue template [airflow]

2023-12-11 Thread via GitHub
josh-fell commented on PR #36180: URL: https://github.com/apache/airflow/pull/36180#issuecomment-1851302762 > we should also make sure that the list is sorted whenever we add a new entry going forward Agreed. The duplicated entry and sort of the list could be add in the "Validate pro

Re: [PR] SparkSubmit Connection Extras can be overridden [airflow]

2023-12-11 Thread via GitHub
pateash commented on PR #36151: URL: https://github.com/apache/airflow/pull/36151#issuecomment-1851297112 cc. @eladkal @hussein-awala -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Handling exit of kerberos sidecar container for k8s < 1.28 [airflow]

2023-12-11 Thread via GitHub
amoghrajesh commented on code in PR #35748: URL: https://github.com/apache/airflow/pull/35748#discussion_r1423386253 ## chart/templates/workers/worker-deployment.yaml: ## @@ -342,9 +347,13 @@ spec: {{- if $containerLifecycleHooksKerberosSidecar }} lifecycle

Re: [PR] Handling exit of kerberos sidecar container for k8s < 1.28 [airflow]

2023-12-11 Thread via GitHub
amoghrajesh commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1851276297 > Yes. No need to Add 2.8.0 in this case as we are not using any new feature added in kerberos command as I understand ? You are right. No new feature for airflow kerberos. Re

Re: [PR] Remove duplicate dropdown entry in providers issue template [airflow]

2023-12-11 Thread via GitHub
pateash commented on PR #36180: URL: https://github.com/apache/airflow/pull/36180#issuecomment-1851275984 thanks @josh-fell , we should also make sure that the list is sorted whenever we add a new entry going forward -- This is an automated message from the Apache Git Service. To respo

[PR] Improve document clarity. [airflow]

2023-12-11 Thread via GitHub
UncertainBadg3r opened a new pull request, #36181: URL: https://github.com/apache/airflow/pull/36181 This PR improves the clarity of the `CI.rst` document. --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apac

[PR] Remove duplicate dropdown entry in providers issue template [airflow]

2023-12-11 Thread via GitHub
josh-fell opened a new pull request, #36180: URL: https://github.com/apache/airflow/pull/36180 Because there is a duplicate option in the "Apache Airflow Provider(s)" dropdown, the "Airflow Providers Bug report" template is not available for users to select when logging a bug. -- Thi

Re: [I] dag_run.get_previous_dagrun() returning wrong dag_run with manual DAG triggering [airflow]

2023-12-11 Thread via GitHub
josh-fell commented on issue #36178: URL: https://github.com/apache/airflow/issues/36178#issuecomment-1851236835 Hi @ThiagoLuka, you mentioned this occurs in an older Airflow version, but I don't see one in the issue details you provided. What Airflow version are you experiencing this issue

Re: [I] Add operator to capture diagnostic tarball for Dataproc [airflow]

2023-12-11 Thread via GitHub
josh-fell commented on issue #36173: URL: https://github.com/apache/airflow/issues/36173#issuecomment-1851235144 Sounds cool. Feel free to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Syedahsn/aws executors ecs credentials refresh [airflow]

2023-12-11 Thread via GitHub
o-nikolas commented on code in PR #36179: URL: https://github.com/apache/airflow/pull/36179#discussion_r1423302337 ## airflow/providers/amazon/aws/executors/ecs/ecs_executor.py: ## @@ -288,6 +333,9 @@ def attempt_task_runs(self): _failure_reasons = [] t

Re: [PR] Add druid ingestion connection test [airflow]

2023-12-11 Thread via GitHub
github-actions[bot] closed pull request #33796: Add druid ingestion connection test URL: https://github.com/apache/airflow/pull/33796 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] "Celery command failed", but "Marking task as SUCCESS" [airflow]

2023-12-11 Thread via GitHub
github-actions[bot] commented on issue #35610: URL: https://github.com/apache/airflow/issues/35610#issuecomment-1851098214 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [PR] Feature add get polars df to dbapihook and bigqueryhook [airflow]

2023-12-11 Thread via GitHub
github-actions[bot] closed pull request #34679: Feature add get polars df to dbapihook and bigqueryhook URL: https://github.com/apache/airflow/pull/34679 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Support git-sync v4 in Helm Chart [airflow]

2023-12-11 Thread via GitHub
github-actions[bot] commented on PR #34731: URL: https://github.com/apache/airflow/pull/34731#issuecomment-1851098298 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] bump open api image to v7.0.1 [airflow]

2023-12-11 Thread via GitHub
github-actions[bot] closed pull request #34805: bump open api image to v7.0.1 URL: https://github.com/apache/airflow/pull/34805 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] EventsTimetable schedules past events even if catchup=False [airflow]

2023-12-11 Thread via GitHub
jscheffl commented on PR #36134: URL: https://github.com/apache/airflow/pull/36134#issuecomment-1851018689 @uranusjr As you are tagged as Codeowner, can you make another review before merge? Alternatively @Taragolis please make a second pass and then OK to merge? -- This is an automated m

[PR] Syedahsn/aws executors ecs credentials refresh [airflow]

2023-12-11 Thread via GitHub
syedahsn opened a new pull request, #36179: URL: https://github.com/apache/airflow/pull/36179 Currently, credentials are loaded once at the initialization of the ECS Executor, and those credentials are used to authenticate to the ECS API. If those credentials expire, the Executor will not a

Re: [I] dag_run.get_previous_dagrun() returning wrong dag_run with manual DAG triggering [airflow]

2023-12-11 Thread via GitHub
boring-cyborg[bot] commented on issue #36178: URL: https://github.com/apache/airflow/issues/36178#issuecomment-1851005833 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] dag_run.get_previous_dagrun() returning wrong dag_run with manual DAG triggering [airflow]

2023-12-11 Thread via GitHub
ThiagoLuka opened a new issue, #36178: URL: https://github.com/apache/airflow/issues/36178 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened I was running some tests to figure out how `data_interval_start ` and `data_interval_end

Re: [PR] Fix broken regex for allowed_deserialization_classes [airflow]

2023-12-11 Thread via GitHub
tobiaszorzetto commented on code in PR #36147: URL: https://github.com/apache/airflow/pull/36147#discussion_r1423167103 ## airflow/serialization/serde.py: ## @@ -360,7 +360,7 @@ def _register(): @functools.lru_cache(maxsize=None) def _get_patterns() -> list[Pattern]: patt

Re: [PR] Fix broken regex for allowed_deserialization_classes [airflow]

2023-12-11 Thread via GitHub
tobiaszorzetto commented on code in PR #36147: URL: https://github.com/apache/airflow/pull/36147#discussion_r1423167103 ## airflow/serialization/serde.py: ## @@ -360,7 +360,7 @@ def _register(): @functools.lru_cache(maxsize=None) def _get_patterns() -> list[Pattern]: patt

(airflow) branch main updated: Remove `is_authorized_cluster_activity` from auth manager (#36175)

2023-12-11 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 357355ac09 Remove `is_authorized_cluster_activity

Re: [PR] Remove `is_authorized_cluster_activity` from auth manager [airflow]

2023-12-11 Thread via GitHub
vincbeck merged PR #36175: URL: https://github.com/apache/airflow/pull/36175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [I] AIP-56 - FAB AM - Move FAB auth manager to new provider [airflow]

2023-12-11 Thread via GitHub
vincbeck commented on issue #32210: URL: https://github.com/apache/airflow/issues/32210#issuecomment-1850932842 Sounds good -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] AIP-56 - FAB AM - Move FAB auth manager to new provider [airflow]

2023-12-11 Thread via GitHub
potiuk commented on issue #32210: URL: https://github.com/apache/airflow/issues/32210#issuecomment-1850926319 then we need to take more nuanced approach and implement deprecation. I think we should do it in two stages - first add deprecation and extract everything to separate section and th

Re: [I] AIP-56 - FAB AM - Move FAB auth manager to new provider [airflow]

2023-12-11 Thread via GitHub
vincbeck commented on issue #32210: URL: https://github.com/apache/airflow/issues/32210#issuecomment-1850920840 But there are some configs in the section `webserver` which should stay in core Airflow -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [I] Deferrable operator tasks do not call `on_kill` method when fail or restarted [airflow]

2023-12-11 Thread via GitHub
potiuk commented on issue #36090: URL: https://github.com/apache/airflow/issues/36090#issuecomment-1850908399 Yes. Probably it should be changed. The problem is that this would have to bring the task from deferred state first to run it's on_kill method, but that would require someone to imp

Re: [I] AIP-56 - FAB AM - Move FAB auth manager to new provider [airflow]

2023-12-11 Thread via GitHub
potiuk commented on issue #32210: URL: https://github.com/apache/airflow/issues/32210#issuecomment-1850884013 And longer explanation -> both will produce the very same config entries, it's just the matter to not have code that handles the same section from both provider and core. But since:

Re: [PR] Add --forward-credentials flag to test commmands [airflow]

2023-12-11 Thread via GitHub
potiuk merged PR #36176: URL: https://github.com/apache/airflow/pull/36176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Upgrade bitnami/postgresql subchart to 13.2.24 [airflow]

2023-12-11 Thread via GitHub
dnskr commented on PR #36156: URL: https://github.com/apache/airflow/pull/36156#issuecomment-1850849587 Rebased onto `main` branch to include fix https://github.com/apache/airflow/pull/36169. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [I] Update Airflow Airbyte Provider to use the new Airbyte API [airflow]

2023-12-11 Thread via GitHub
potiuk commented on issue #35011: URL: https://github.com/apache/airflow/issues/35011#issuecomment-1850845816 Sure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

(airflow) branch main updated: Add helper function for CRUD operations on weaviate's schema and class objects (#35919)

2023-12-11 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5439b494b0 Add helper function for CRUD op

Re: [PR] Add helper function for CRUD operations on weaviate's schema and class objects [airflow]

2023-12-11 Thread via GitHub
ephraimbuddy merged PR #35919: URL: https://github.com/apache/airflow/pull/35919 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

Re: [I] Listeners for Datasets [airflow]

2023-12-11 Thread via GitHub
DjVinnii commented on issue #34327: URL: https://github.com/apache/airflow/issues/34327#issuecomment-1850822170 @cmarteepants > Like what kind of external processes could someone trigger on dataset events? We are going to use the dataset listeners in order to synchronize datas

[PR] Add --forward-credentials flag to test commmands [airflow]

2023-12-11 Thread via GitHub
potiuk opened a new pull request, #36176: URL: https://github.com/apache/airflow/pull/36176 The change #36131 made a subtle change to the behaviour of tests - they used to have credentials forwarded by default, but the intention was that that the credentials are only forwarded via explicit

Re: [I] AIP-56 - FAB AM - Move FAB auth manager to new provider [airflow]

2023-12-11 Thread via GitHub
vincbeck commented on issue #32210: URL: https://github.com/apache/airflow/issues/32210#issuecomment-1850804326 Resolved as part of #35926 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [I] Update Airflow Airbyte Provider to use the new Airbyte API [airflow]

2023-12-11 Thread via GitHub
Cir02 commented on issue #35011: URL: https://github.com/apache/airflow/issues/35011#issuecomment-1850739733 Hello there, i have face a similar issue using this provider, not working at all following this [configuration from airbyte](https://airbyte.com/tutorials/how-to-use-airflow-

(airflow) branch constraints-main updated: Updating constraints. Github run id:7171890351

2023-12-11 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 8fc9846db6 Updating con

[PR] Remove `is_authorized_cluster_activity` from auth manager [airflow]

2023-12-11 Thread via GitHub
vincbeck opened a new pull request, #36175: URL: https://github.com/apache/airflow/pull/36175 There is a duplicate way to check permissions access for cluster activity page, `is_authorized_view` is the correct one. --- **^ Add meaningful description abov

[PR] Enable native HPA for Airflow Workers [airflow]

2023-12-11 Thread via GitHub
pemiranda-clgx opened a new pull request, #36174: URL: https://github.com/apache/airflow/pull/36174 Add K8S native Horizontal Pod Autoscaler in Airflow Workers to support autoscaling based on external custom metrics. --- **^ Add meaningful description above**

Re: [PR] Create FAB provider and move FAB auth manager in it [airflow]

2023-12-11 Thread via GitHub
potiuk commented on PR #35926: URL: https://github.com/apache/airflow/pull/35926#issuecomment-1850674253 Wooh! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

(airflow) branch main updated (e37fe8f576 -> 1eca667e5f)

2023-12-11 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from e37fe8f576 Add `jsonpath_ng.ext.parse` support for `SqsSensor` (#36170) add 1eca667e5f Create FAB provider and m

Re: [PR] Create FAB provider and move FAB auth manager in it [airflow]

2023-12-11 Thread via GitHub
vincbeck merged PR #35926: URL: https://github.com/apache/airflow/pull/35926 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] SparkSubmit Operator fixes [airflow]

2023-12-11 Thread via GitHub
pateash commented on code in PR #36151: URL: https://github.com/apache/airflow/pull/36151#discussion_r1422965939 ## airflow/providers/apache/spark/hooks/spark_submit.py: ## @@ -204,8 +213,8 @@ def _resolve_connection(self) -> dict[str, Any]: # Determine optional y

(airflow) branch main updated: Add `jsonpath_ng.ext.parse` support for `SqsSensor` (#36170)

2023-12-11 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e37fe8f576 Add `jsonpath_ng.ext.parse` support fo

Re: [PR] Add jsonpath_ng.ext.parse support for SqsSensor [airflow]

2023-12-11 Thread via GitHub
vincbeck merged PR #36170: URL: https://github.com/apache/airflow/pull/36170 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] Add jsonpath_ng.ext.parse support for SqsSensor [airflow]

2023-12-11 Thread via GitHub
boring-cyborg[bot] commented on PR #36170: URL: https://github.com/apache/airflow/pull/36170#issuecomment-1850657667 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [PR] Create FAB provider and move FAB auth manager in it [airflow]

2023-12-11 Thread via GitHub
vincbeck commented on PR #35926: URL: https://github.com/apache/airflow/pull/35926#issuecomment-1850656418 CLI is green! (thanks @potiuk for the help!) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

(airflow) branch constraints-mai created (now 195abf8f71)

2023-12-11 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch constraints-mai in repository https://gitbox.apache.org/repos/asf/airflow.git at 195abf8f71 SparkSubmit: Adding propertyfiles option (#36164) No new revisions were added by this update.

Re: [I] Listeners for Datasets [airflow]

2023-12-11 Thread via GitHub
cmarteepants commented on issue #34327: URL: https://github.com/apache/airflow/issues/34327#issuecomment-1850598222 @DjVinnii, for docs/release notes purposes, high-level what is your use-case for this? -- This is an automated message from the Apache Git Service. To respond to the message

[I] Add operator to capture diagnostic tarball for Dataproc [airflow]

2023-12-11 Thread via GitHub
flacode opened a new issue, #36173: URL: https://github.com/apache/airflow/issues/36173 ### Description Add operator to capture the diagnostic tarball for a Dataproc cluster managed by Airflow. ### Use case/motivation The [diagnostic tarball](https://cloud.google.co

Re: [I] Add operator to capture diagnostic tarball for Dataproc [airflow]

2023-12-11 Thread via GitHub
boring-cyborg[bot] commented on issue #36173: URL: https://github.com/apache/airflow/issues/36173#issuecomment-1850586325 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Implement AthenaSQLHook [airflow]

2023-12-11 Thread via GitHub
vincbeck commented on code in PR #36171: URL: https://github.com/apache/airflow/pull/36171#discussion_r1422892078 ## airflow/providers/amazon/aws/hooks/athena_sql.py: ## @@ -0,0 +1,110 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licens

(airflow) 01/01: hooks poc

2023-12-11 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a commit to branch hooks-lineage in repository https://gitbox.apache.org/repos/asf/airflow.git commit 1d4c424fb03f97d4a7f974de33722b8c5f82b3b7 Author: Maciej Obuchowski AuthorDate: Fri Nov 3 16:49:02 2023 +010

(airflow) branch hooks-lineage updated (6ae8fb11d9 -> 1d4c424fb0)

2023-12-11 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a change to branch hooks-lineage in repository https://gitbox.apache.org/repos/asf/airflow.git discard 6ae8fb11d9 hooks poc add 7e1f67b4b8 Fix a bug in method name used in `GlacierToGCSOperator` (#35978

(airflow) 02/02: datasets from hooks

2023-12-11 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a commit to branch hooks-lineage in repository https://gitbox.apache.org/repos/asf/airflow.git commit 351f7fe2271812ea6ba43c2ebce65c3f163854fa Author: Maciej Obuchowski AuthorDate: Mon Dec 11 18:41:54 2023 +01

(airflow) 01/01: hooks poc

2023-12-11 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a commit to branch hooks-lineage in repository https://gitbox.apache.org/repos/asf/airflow.git commit 6ae8fb11d924c1a863e3445d803beb2d85d31f4b Author: Maciej Obuchowski AuthorDate: Fri Nov 3 16:49:02 2023 +010

(airflow) branch hooks-lineage updated (351f7fe227 -> 6ae8fb11d9)

2023-12-11 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a change to branch hooks-lineage in repository https://gitbox.apache.org/repos/asf/airflow.git discard 351f7fe227 datasets from hooks discard 64e522eead hooks poc new 6ae8fb11d9 hooks poc This update a

(airflow) 01/02: hooks poc

2023-12-11 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a commit to branch hooks-lineage in repository https://gitbox.apache.org/repos/asf/airflow.git commit 64e522eead3dcf6a3b35e4bed96363080cd01a7a Author: Maciej Obuchowski AuthorDate: Fri Nov 3 16:49:02 2023 +010

Re: [PR] Adding property files option in the Spark Submit command [airflow]

2023-12-11 Thread via GitHub
eladkal closed pull request #34894: Adding property files option in the Spark Submit command URL: https://github.com/apache/airflow/pull/34894 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] feat: SQLExecuteQueryOperator sql_callable [airflow]

2023-12-11 Thread via GitHub
potiuk commented on PR #35844: URL: https://github.com/apache/airflow/pull/35844#issuecomment-1850480565 > 2. we add to the baseoperator the possibility to add a callable to custom the templating logic I would very much prefer this -- This is an automated message from the Apache Gi

[PR] Implement AthenaSQLHook [airflow]

2023-12-11 Thread via GitHub
flolas opened a new pull request, #36171: URL: https://github.com/apache/airflow/pull/36171 In this Pull Request, we're introducing the `AthenaSQLHook`, an implementation leveraging PyAthena. This development opens doors for us to interact with Athena using the familiar DBApiHook operators.

Re: [PR] Fix broken regex for allowed_deserialization_classes [airflow]

2023-12-11 Thread via GitHub
potiuk commented on code in PR #36147: URL: https://github.com/apache/airflow/pull/36147#discussion_r1422794159 ## airflow/serialization/serde.py: ## @@ -360,7 +360,7 @@ def _register(): @functools.lru_cache(maxsize=None) def _get_patterns() -> list[Pattern]: patterns = c

Re: [I] Adding property files option in the Spark Submit command [airflow]

2023-12-11 Thread via GitHub
potiuk closed issue #34838: Adding property files option in the Spark Submit command URL: https://github.com/apache/airflow/issues/34838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

(airflow) branch main updated: SparkSubmit: Adding propertyfiles option (#36164)

2023-12-11 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 195abf8f71 SparkSubmit: Adding propertyfiles option

Re: [PR] SparkSubmit: Adding propertyfiles option [airflow]

2023-12-11 Thread via GitHub
potiuk merged PR #36164: URL: https://github.com/apache/airflow/pull/36164 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Create FAB provider and move FAB auth manager in it [airflow]

2023-12-11 Thread via GitHub
potiuk commented on PR #35926: URL: https://github.com/apache/airflow/pull/35926#issuecomment-1850399011 Nope. It was https://github.com/apache/airflow/pull/36168 - rebase and it should be gone (and my PR also added different behaviour when constraint generation will fail - it will fail at

Re: [PR] Create FAB provider and move FAB auth manager in it [airflow]

2023-12-11 Thread via GitHub
vincbeck commented on PR #35926: URL: https://github.com/apache/airflow/pull/35926#issuecomment-1850364946 When doing the merge, I removed [this line](https://github.com/apache/airflow/pull/35926/commits/8f1b089af16f06cd372304d185ba0565ec32b3f5#diff-9a3861f568f7701950cbeaec9b46fbafb8f39ccbfa

[PR] Add jsonpath_ng.ext.parse support for SqsSensor [airflow]

2023-12-11 Thread via GitHub
cliebBS opened a new pull request, #36170: URL: https://github.com/apache/airflow/pull/36170 This PR extends the `message_filtering` parameter to the `SqsSensor` to add a third type of filtering: `jsonpath-ext`. The existing JSONPath filtering functionality in `SqsSensor` uses the `jsonpat

Re: [PR] Create FAB provider and move FAB auth manager in it [airflow]

2023-12-11 Thread via GitHub
vincbeck commented on PR #35926: URL: https://github.com/apache/airflow/pull/35926#issuecomment-1850356286 The build prod image fails again with the latest changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Allow changing the unfinished ti state [airflow]

2023-12-11 Thread via GitHub
kimyen commented on code in PR #36078: URL: https://github.com/apache/airflow/pull/36078#discussion_r1422701491 ## tests/integration/executors/test_celery_executor.py: ## @@ -146,6 +147,9 @@ def fake_execute_command(command): executor._process_tasks(task_tuple

Re: [PR] SparkSubmit: Adding propertyfiles option [airflow]

2023-12-11 Thread via GitHub
eladkal commented on code in PR #36164: URL: https://github.com/apache/airflow/pull/36164#discussion_r1422658466 ## tests/system/providers/apache/hive/example_twitter_README.md: ## @@ -33,7 +33,7 @@ 1. Save Summary to MySQL ***Screenshot:*** -http://i.imgur.com/rRpSO12.png";

Re: [I] "AirflowException: Invalid arguments were passed to GlueJobOperator" when setting update_config=True [airflow]

2023-12-11 Thread via GitHub
andrew-parsons-janus commented on issue #35637: URL: https://github.com/apache/airflow/issues/35637#issuecomment-1850309032 This was embarrassing. Sorry for creating noise! I was indeed running an old version of `apache-airflow-providers-amazon` without realizing it. -- This is an automat

Re: [I] "AirflowException: Invalid arguments were passed to GlueJobOperator" when setting update_config=True [airflow]

2023-12-11 Thread via GitHub
andrew-parsons-janus closed issue #35637: "AirflowException: Invalid arguments were passed to GlueJobOperator" when setting update_config=True URL: https://github.com/apache/airflow/issues/35637 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Add overrides to template fields of Google Cloud Run Jobs Execute Operator [airflow]

2023-12-11 Thread via GitHub
boring-cyborg[bot] commented on PR #36133: URL: https://github.com/apache/airflow/pull/36133#issuecomment-1850306924 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [I] Add overrides in the template field for the Google Cloud Run Jobs Execute operator [airflow]

2023-12-11 Thread via GitHub
potiuk closed issue #36132: Add overrides in the template field for the Google Cloud Run Jobs Execute operator URL: https://github.com/apache/airflow/issues/36132 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Add overrides to template fields of Google Cloud Run Jobs Execute Operator [airflow]

2023-12-11 Thread via GitHub
potiuk commented on PR #36133: URL: https://github.com/apache/airflow/pull/36133#issuecomment-1850306512 Nope. I fixed it already earlier today in #36162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

(airflow) branch main updated: Add overrides to template fields of Google Cloud Run Jobs Execute Operator (#36133)

2023-12-11 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 3dddfb4a4a Add overrides to template fields of Goog

Re: [PR] Add overrides to template fields of Google Cloud Run Jobs Execute Operator [airflow]

2023-12-11 Thread via GitHub
potiuk merged PR #36133: URL: https://github.com/apache/airflow/pull/36133 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] Task is not able to be run - k8s Celery workers with stale resources [airflow]

2023-12-11 Thread via GitHub
potiuk commented on issue #34013: URL: https://github.com/apache/airflow/issues/34013#issuecomment-1850303636 > I have been seeing this error appearing after upgrading to 2.7.3, also using the Celery Executor on K8s. However, I don't see that pattern when looking at resource consumption. I

Re: [PR] Pin markdownlint-cli to latest version and fix problems it finds [airflow]

2023-12-11 Thread via GitHub
potiuk merged PR #36169: URL: https://github.com/apache/airflow/pull/36169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Pin markdownlint-cli to latest version and fix problems it finds (#36169)

2023-12-11 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new df23df5315 Pin markdownlint-cli to latest version a

  1   2   >