Re: [PR] Reenable CI using Azure Pipelines [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #368: URL: https://github.com/apache/incubator-xtable/pull/368#issuecomment-1978157990 ## CI report: * 384249100e7ad5b006a110be3d085043e2dd12fd Azure: [SUCCESS](https://dev.azure.com/apache-xtable-ci-org/bb368b1b-7955-479d-8cbf-689ff9aa23eb/_build/result

Re: [PR] test [incubator-xtable]

2024-03-04 Thread via GitHub
jcamachor closed pull request #370: test URL: https://github.com/apache/incubator-xtable/pull/370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commit

Re: [PR] test [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #370: URL: https://github.com/apache/incubator-xtable/pull/370#issuecomment-1978152658 ## CI report: * 109bd5866b115a27ccbd5c4018126ef5ac77c16d UNKNOWN Bot commands @xtable-bot supports the following commands: - `@xtable-bot run az

[PR] test [incubator-xtable]

2024-03-04 Thread via GitHub
jcamachor opened a new pull request, #370: URL: https://github.com/apache/incubator-xtable/pull/370 ## *Important Read* - *Please ensure the GitHub issue is mentioned at the beginning of the PR* ## What is the purpose of the pull request *(For example: This pull request imple

Re: [PR] Reenable CI using Azure Pipelines [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #368: URL: https://github.com/apache/incubator-xtable/pull/368#issuecomment-1978137399 ## CI report: * 4f142a9bd5ceb6fc56bd16e14574f5e12abce11a Azure: [SUCCESS](https://dev.azure.com/apache-xtable-ci-org/bb368b1b-7955-479d-8cbf-689ff9aa23eb/_build/result

Re: [PR] Reenable CI using Azure Pipelines [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #368: URL: https://github.com/apache/incubator-xtable/pull/368#issuecomment-1978135838 ## CI report: * 4f142a9bd5ceb6fc56bd16e14574f5e12abce11a Azure: [SUCCESS](https://dev.azure.com/apache-xtable-ci-org/bb368b1b-7955-479d-8cbf-689ff9aa23eb/_build/result

Re: [PR] Reenable CI using Azure Pipelines [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #368: URL: https://github.com/apache/incubator-xtable/pull/368#issuecomment-1978133541 ## CI report: * 4f142a9bd5ceb6fc56bd16e14574f5e12abce11a Azure: [SUCCESS](https://dev.azure.com/apache-xtable-ci-org/bb368b1b-7955-479d-8cbf-689ff9aa23eb/_build/result

Re: [PR] Reenable CI using Azure Pipelines [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #368: URL: https://github.com/apache/incubator-xtable/pull/368#issuecomment-1978131267 ## CI report: * 4f142a9bd5ceb6fc56bd16e14574f5e12abce11a Azure: [PENDING](https://dev.azure.com/apache-xtable-ci-org/bb368b1b-7955-479d-8cbf-689ff9aa23eb/_build/result

Re: [PR] Reenable CI using Azure Pipelines [incubator-xtable]

2024-03-04 Thread via GitHub
jcamachor commented on PR #368: URL: https://github.com/apache/incubator-xtable/pull/368#issuecomment-1978130296 @xtable-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Reenable CI using Azure Pipelines [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #368: URL: https://github.com/apache/incubator-xtable/pull/368#issuecomment-1978123719 ## CI report: * 4f142a9bd5ceb6fc56bd16e14574f5e12abce11a Azure: [SUCCESS](https://dev.azure.com/apache-xtable-ci-org/bb368b1b-7955-479d-8cbf-689ff9aa23eb/_build/result

Re: [PR] Reenable CI using Azure Pipelines [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #368: URL: https://github.com/apache/incubator-xtable/pull/368#issuecomment-1978108029 ## CI report: * 4f142a9bd5ceb6fc56bd16e14574f5e12abce11a Azure: [PENDING](https://dev.azure.com/apache-xtable-ci-org/bb368b1b-7955-479d-8cbf-689ff9aa23eb/_build/result

Re: [PR] License header check [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #364: URL: https://github.com/apache/incubator-xtable/pull/364#issuecomment-1978107931 ## CI report: * 41763f3601dcdd9acb268fd4e8a4a772c7b85032 UNKNOWN Bot commands @xtable-bot supports the following commands: - `@xtable-bot run az

Re: [PR] Docs and Website XTable Rename [incubator-xtable]

2024-03-04 Thread via GitHub
kywe665 commented on code in PR #352: URL: https://github.com/apache/incubator-xtable/pull/352#discussion_r1512237572 ## website/static/images/destination1.svg: ## @@ -1,3 +1,19 @@ + Review Comment: I went to traceback the history on this image and actually this image and a

Re: [PR] Reenable CI using Azure Pipelines [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #368: URL: https://github.com/apache/incubator-xtable/pull/368#issuecomment-1978104950 ## CI report: * 4f142a9bd5ceb6fc56bd16e14574f5e12abce11a UNKNOWN Bot commands @xtable-bot supports the following commands: - `@xtable-bot run az

Re: [PR] Docs and Website XTable Rename [incubator-xtable]

2024-03-04 Thread via GitHub
xtable-bot commented on PR #352: URL: https://github.com/apache/incubator-xtable/pull/352#issuecomment-1978104821 ## CI report: * b129b3401604c8913d23596d1b620c1263c0af07 UNKNOWN Bot commands @xtable-bot supports the following commands: - `@xtable-bot run az

Re: [PR] Docs and Website XTable Rename [incubator-xtable]

2024-03-04 Thread via GitHub
kywe665 commented on code in PR #352: URL: https://github.com/apache/incubator-xtable/pull/352#discussion_r1512216885 ## website/blog/OneTable-is-now-Apache-XTable.md: ## @@ -0,0 +1,67 @@ +--- +title: "OneTable is now “Apache XTable (Incubating)”" +excerpt: "XTable is now Incuba

Re: [PR] Docs and Website XTable Rename [incubator-xtable]

2024-03-04 Thread via GitHub
kywe665 commented on code in PR #352: URL: https://github.com/apache/incubator-xtable/pull/352#discussion_r1512216207 ## website/static/images/xtable-svg.svg: ## @@ -0,0 +1,70 @@ +http://www.w3.org/2000/svg";> Review Comment: Thanks for the good idea, I just started the vote

Re: [I] Enable Direct Writing to OneTable Format from Hudi Delta Streamer [incubator-xtable]

2024-03-04 Thread via GitHub
soumilshah1995 commented on issue #351: URL: https://github.com/apache/incubator-xtable/issues/351#issuecomment-1977747328 ![Screenshot 2024-03-04 at 7 47 52  PM](https://github.com/apache/incubator-xtable/assets/39345855/9d46b860-4579-45a1-99e4-a8a33a5696ab) Videos coming soon

Re: [I] AWS Glue with Iceberg Source and Delta Target [incubator-xtable]

2024-03-04 Thread via GitHub
ForeverAngry commented on issue #354: URL: https://github.com/apache/incubator-xtable/issues/354#issuecomment-1977738016 @the-other-tim-brown yeah I misspoke in my comment, you are correct, I'm using the hour transform. -- This is an automated message from the Apache Git Service. To resp

Re: [I] Enable Direct Writing to OneTable Format from Hudi Delta Streamer [incubator-xtable]

2024-03-04 Thread via GitHub
soumilshah1995 commented on issue #351: URL: https://github.com/apache/incubator-xtable/issues/351#issuecomment-1977732640 @sagarlakshmipathy it looks like code worked finally sagar told me remove partition path and after removing partition path it works im curious to know why

Re: [I] AWS Glue Hudi to ICEBERG Tables Fails [incubator-xtable]

2024-03-04 Thread via GitHub
soumilshah1995 commented on issue #362: URL: https://github.com/apache/incubator-xtable/issues/362#issuecomment-1977731278 Roger that ill try this case and update soon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] AWS Glue Hudi to ICEBERG Tables Fails [incubator-xtable]

2024-03-04 Thread via GitHub
the-other-tim-brown commented on issue #362: URL: https://github.com/apache/incubator-xtable/issues/362#issuecomment-1977626106 @soumilshah1995 you would need to incorporate that schema into your writer. One way I've set the schema on the writer in the past is with the [hoodie.write.schema

(incubator-xtable) branch main updated: Fix data file extraction from Hudi restore metadata with log files (#332)

2024-03-04 Thread timbrown
This is an automated email from the ASF dual-hosted git repository. timbrown pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/incubator-xtable.git The following commit(s) were added to refs/heads/main by this push: new 7577727 Fix data file extraction from H

Re: [PR] Fix data file extraction from Hudi restore metadata with log files [incubator-xtable]

2024-03-04 Thread via GitHub
the-other-tim-brown merged PR #332: URL: https://github.com/apache/incubator-xtable/pull/332 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-uns

Re: [I] AWS Glue with Iceberg Source and Delta Target [incubator-xtable]

2024-03-04 Thread via GitHub
the-other-tim-brown commented on issue #354: URL: https://github.com/apache/incubator-xtable/issues/354#issuecomment-1977612046 @ForeverAngry when you say you have an "identity transformation" on the column, what do you mean by that? It sounds like you are using the `hour` transform from t

Re: [I] Enable Direct Writing to OneTable Format from Hudi Delta Streamer [incubator-xtable]

2024-03-04 Thread via GitHub
soumilshah1995 commented on issue #351: URL: https://github.com/apache/incubator-xtable/issues/351#issuecomment-1977611127 Update just spoke with sagar on call adding exercise files for references https://github.com/soumilshah1995/apache-hudi-delta-streamer-labs/tree/main/E2

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-04 Thread via GitHub
the-other-tim-brown commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1977142750 I have not worked with Glue before. Can you help me understand where it differs from regular Scala code? There is no support for python at this time but tha

Re: [I] AWS Glue with Iceberg Source and Delta Target [incubator-xtable]

2024-03-04 Thread via GitHub
ForeverAngry commented on issue #354: URL: https://github.com/apache/incubator-xtable/issues/354#issuecomment-1977086367 > @ForeverAngry what is the error message you are seeing and how are you creating this docker image? Well, it was for sure an error with the image. But the error I

Re: [I] Enable Direct Writing to OneTable Format from Hudi Delta Streamer [incubator-xtable]

2024-03-04 Thread via GitHub
soumilshah1995 commented on issue #351: URL: https://github.com/apache/incubator-xtable/issues/351#issuecomment-1976635118 I wanted to reach out regarding the issue I've been encountering despite trying all the approaches you previously mentioned. It seems I'm still facing some challenges.

Re: [I] [Support] Request: Ability to Invoke Scala Code for Operations in oneTable [incubator-xtable]

2024-03-04 Thread via GitHub
soumilshah1995 commented on issue #353: URL: https://github.com/apache/incubator-xtable/issues/353#issuecomment-1976608176 Hello @the-other-tim-brown, Thank you for your response. I understand that we can add Jar files, but I was wondering if you have any examples of glue code with o

Re: [I] AWS Glue Hudi to ICEBERG Tables Fails [incubator-xtable]

2024-03-04 Thread via GitHub
soumilshah1995 commented on issue #362: URL: https://github.com/apache/incubator-xtable/issues/362#issuecomment-1976601709 While writing data do I need to specify schema ``` schema = StructType([ StructField("customer_id", StringType(), nullable=False), StructField("

Re: [I] AWS Glue Hudi to ICEBERG Tables Fails [incubator-xtable]

2024-03-04 Thread via GitHub
soumilshah1995 commented on issue #362: URL: https://github.com/apache/incubator-xtable/issues/362#issuecomment-1976599431 ello, I hope this message finds you well. I wanted to share with you the progress I've made in writing data into Hudi using PySpark. You can find the code imple

Re: [I] Enable Direct Writing to OneTable Format from Hudi Delta Streamer [incubator-xtable]

2024-03-04 Thread via GitHub
soumilshah1995 commented on issue #351: URL: https://github.com/apache/incubator-xtable/issues/351#issuecomment-1976591572 I did use that in this example im having same issue # Spark Submit ``` spark-submit \ --class org.apache.hudi.utilities.streamer.HoodieStreamer \

Re: [PR] License header check [incubator-xtable]

2024-03-04 Thread via GitHub
wuchunfu commented on PR #364: URL: https://github.com/apache/incubator-xtable/pull/364#issuecomment-1976569864 > @wuchunfu Spotless is not limited to Java as far as I know. Indeed the current configuration of the plugin only deals with Java files but this can change I believe. > > T

Re: [PR] Docs and Website XTable Rename [incubator-xtable]

2024-03-04 Thread via GitHub
zabetak commented on code in PR #352: URL: https://github.com/apache/incubator-xtable/pull/352#discussion_r1510890894 ## website/blog/OneTable-is-now-Apache-XTable.md: ## @@ -0,0 +1,67 @@ +--- +title: "OneTable is now “Apache XTable (Incubating)”" +excerpt: "XTable is now Incuba

Re: [PR] License header check [incubator-xtable]

2024-03-04 Thread via GitHub
zabetak commented on PR #364: URL: https://github.com/apache/incubator-xtable/pull/364#issuecomment-1976134692 @wuchunfu Spotless is not limited to Java as far as I know. Indeed the current configuration of the plugin only deals with Java files but this can change I believe. The curr

Re: [PR] License header check [incubator-xtable]

2024-03-04 Thread via GitHub
wuchunfu commented on PR #364: URL: https://github.com/apache/incubator-xtable/pull/364#issuecomment-1976043800 > The project uses spotless-maven-plugin to enforce/check the LICENSE header as part of the build. Why do we need a separate CI action and additional tooling? @zabetak Beca