Re: [PR] [HUDI-7826] Make column nullable when setNullForMissingColumns is true [hudi]

2024-06-11 Thread via GitHub
nsivabalan merged PR #11381: URL: https://github.com/apache/hudi/pull/11381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apa

(hudi) branch master updated (2a56065c854 -> 51ef7097cb4)

2024-06-11 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 2a56065c854 [HUDI-7853] Fix missing serDe properties post migration from hiveSync to glueSync (#11404) add 51ef

Re: [PR] fix the error message for glue sync tool [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11430: URL: https://github.com/apache/hudi/pull/11430#issuecomment-2160007456 ## CI report: * 5b0d785428f5c559037c63fe76040f23bc741f91 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

[I] [SUPPORT] [hudi]

2024-06-11 Thread via GitHub
zaminhassnain06 opened a new issue, #11431: URL: https://github.com/apache/hudi/issues/11431 Hi, I tried to update my hudi version from 0.6.0 to 0.11. I updated it gradually version by version starting from 0.6 to 0.9 and then from 0.9 to 0.10 and finally from 0.10 to 0.11. I am running

Re: [PR] fix the error message for glue sync tool [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11430: URL: https://github.com/apache/hudi/pull/11430#issuecomment-2160126378 ## CI report: * 5b0d785428f5c559037c63fe76040f23bc741f91 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [I] org.apache.hudi.exception.HoodieException: org.apache.avro.AvroTypeException: Cannot encode decimal with precision 14 as max precision 13 [hudi]

2024-06-11 Thread via GitHub
danny0405 commented on issue #11335: URL: https://github.com/apache/hudi/issues/11335#issuecomment-2160184451 I see quite a few columns are declared as precision 13, do we have schema evolution on them? ```java |-- target_amount: decimal(13,4) (nullable = true) |-- overwritten_t

Re: [I] org.apache.hudi.exception.HoodieException: org.apache.avro.AvroTypeException: Cannot encode decimal with precision 14 as max precision 13 [hudi]

2024-06-11 Thread via GitHub
KSubramanyaH commented on issue #11335: URL: https://github.com/apache/hudi/issues/11335#issuecomment-2160234304 @danny0405 No, It has not evolved . But schema evolution is not required as incoming data is compatible with 13,4 We are loading 2 hudi tables with same schema .

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11385: URL: https://github.com/apache/hudi/pull/11385#issuecomment-2160555670 ## CI report: * eb8310a8cf80af1c769e0faa558bfdba8e2d188b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11385: URL: https://github.com/apache/hudi/pull/11385#issuecomment-2160572092 ## CI report: * eb8310a8cf80af1c769e0faa558bfdba8e2d188b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [I] [SUPPORT] Spark Read Hudi Tables with WARN Message [hudi]

2024-06-11 Thread via GitHub
michael1991 commented on issue #10828: URL: https://github.com/apache/hudi/issues/10828#issuecomment-2160641880 I use spark3.5+hudi0.15.0 to write and read same table, this warning message could be seen too. -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11385: URL: https://github.com/apache/hudi/pull/11385#issuecomment-2160675828 ## CI report: * f82b7a52ffad611530363dbec1b546d017851a3a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7269] Fallback to key based merge if positions are missing from log block [hudi]

2024-06-11 Thread via GitHub
codope merged PR #11415: URL: https://github.com/apache/hudi/pull/11415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

(hudi) branch master updated: [HUDI-7269] Fallback to key based merge if positions are missing from log block (#11415)

2024-06-11 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 5143a982bca [HUDI-7269] Fallback to key based merg

Re: [I] [SUPPORT] Spark Read Hudi Tables with WARN Message [hudi]

2024-06-11 Thread via GitHub
michael1991 closed issue #10828: [SUPPORT] Spark Read Hudi Tables with WARN Message URL: https://github.com/apache/hudi/issues/10828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] [HUDI-7384] Secondary index support [hudi]

2024-06-11 Thread via GitHub
bhat-vinay commented on PR #10625: URL: https://github.com/apache/hudi/pull/10625#issuecomment-2160742331 > Hi @bhat-vinay! Is this design of secondary index through MDT is the only one to be implemented or there plans to make some other Index Types? As I remember there was RFC for Lucene I

Re: [I] [SUPPORT] Spark Read Hudi Tables with WARN Message [hudi]

2024-06-11 Thread via GitHub
michael1991 commented on issue #10828: URL: https://github.com/apache/hudi/issues/10828#issuecomment-2160744749 > @michael1991 Not sure if this one is related to warning. Hi @ad1happy2go , I resolved this warning message by setting `"hoodie.datasource.write.partitionpath.urlencode" ->

[jira] [Closed] (HUDI-7269) Fallback to key-based merging if there is no positions in log header

2024-06-11 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7269. - Resolution: Done > Fallback to key-based merging if there is no positions in log header >

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() [hudi]

2024-06-11 Thread via GitHub
wombatu-kun commented on PR #11385: URL: https://github.com/apache/hudi/pull/11385#issuecomment-2160893423 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11385: URL: https://github.com/apache/hudi/pull/11385#issuecomment-2160938984 ## CI report: * f82b7a52ffad611530363dbec1b546d017851a3a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-6787] DO NOT MERGE. Test hive3 in ghactions bootstrap [hudi]

2024-06-11 Thread via GitHub
jonvex closed pull request #11399: [HUDI-6787] DO NOT MERGE. Test hive3 in ghactions bootstrap URL: https://github.com/apache/hudi/pull/11399 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] current state [hudi]

2024-06-11 Thread via GitHub
jonvex closed pull request #11094: current state URL: https://github.com/apache/hudi/pull/11094 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-

[PR] [DOCS] Update show_fsview_latest para [hudi]

2024-06-11 Thread via GitHub
Zouxxyy opened a new pull request, #11432: URL: https://github.com/apache/hudi/pull/11432 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performanc

Re: [PR] [HUDI-6787] DO NOT MERGE. Test hive3 in ghactions [hudi]

2024-06-11 Thread via GitHub
jonvex closed pull request #11398: [HUDI-6787] DO NOT MERGE. Test hive3 in ghactions URL: https://github.com/apache/hudi/pull/11398 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] [HUDI-1234] DO NOT MERGE use fg reader in cdc test [hudi]

2024-06-11 Thread via GitHub
jonvex closed pull request #11401: [HUDI-1234] DO NOT MERGE use fg reader in cdc test URL: https://github.com/apache/hudi/pull/11401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11385: URL: https://github.com/apache/hudi/pull/11385#issuecomment-2160957592 ## CI report: * f82b7a52ffad611530363dbec1b546d017851a3a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

[jira] [Closed] (HUDI-6792) Integrate FileGroupReader with NewHoodieParquetFileFormat for Spark Incremental Query

2024-06-11 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler closed HUDI-6792. - Resolution: Duplicate > Integrate FileGroupReader with NewHoodieParquetFileFormat for Spark > Inc

(hudi) 01/01: Create release branch for version 1.0.0-beta2

2024-06-11 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch release-1.0.0-beta2 in repository https://gitbox.apache.org/repos/asf/hudi.git commit ea2cb5c5d74f086a8469881dac85556cd6683b14 Author: Sagar Sumit AuthorDate: Tue Jun 11 09:23:59 2024 -0700

(hudi) branch release-1.0.0-beta2 created (now ea2cb5c5d74)

2024-06-11 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a change to branch release-1.0.0-beta2 in repository https://gitbox.apache.org/repos/asf/hudi.git at ea2cb5c5d74 Create release branch for version 1.0.0-beta2 This branch includes the following new commits:

[PR] [MINOR] Update release guide [hudi]

2024-06-11 Thread via GitHub
codope opened a new pull request, #11433: URL: https://github.com/apache/hudi/pull/11433 ### Change Logs Minor release guide update ### Impact none ### Risk level (write none, low medium or high below) none ### Documentation Update _Describe an

Re: [PR] [HUDI-7384] Secondary index support [hudi]

2024-06-11 Thread via GitHub
codope commented on PR #10625: URL: https://github.com/apache/hudi/pull/10625#issuecomment-2161277447 Hi @skyshineb , we do plan to add more index types. If you are interested in contributing to lucene based secondary index, I can help you to get started with multi-modal indexing framework.

Re: [PR] [HUDI-7384] Secondary index support [hudi]

2024-06-11 Thread via GitHub
codope closed pull request #10625: [HUDI-7384] Secondary index support URL: https://github.com/apache/hudi/pull/10625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[jira] [Updated] (HUDI-7841) RLI and secondary index should consider only pruned partitions for file skipping

2024-06-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7841: - Labels: pull-request-available (was: ) > RLI and secondary index should consider only pruned part

[PR] [HUDI-7841] RLI and secondary index should consider only pruned partitions for file skipping [hudi]

2024-06-11 Thread via GitHub
lokeshj1703 opened a new pull request, #11434: URL: https://github.com/apache/hudi/pull/11434 ### Change Logs Even though RLI scans only matching files, it tries to get those candidate files by iterating over all files from file index. See - https://github.com/apache/hudi/blob/f4be74

Re: [PR] [MINOR] Update release guide [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11433: URL: https://github.com/apache/hudi/pull/11433#issuecomment-2161308857 ## CI report: * 34e7b5605dabbae4e81fed0d8c2f32e7f1b61261 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-7841] RLI and secondary index should consider only pruned partitions for file skipping [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11434: URL: https://github.com/apache/hudi/pull/11434#issuecomment-2161308932 ## CI report: * 9aadf246c5e26b4fe5c48a6b152ef9632d7e586f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [MINOR] Update release guide [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11433: URL: https://github.com/apache/hudi/pull/11433#issuecomment-2161321539 ## CI report: * 34e7b5605dabbae4e81fed0d8c2f32e7f1b61261 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7841] RLI and secondary index should consider only pruned partitions for file skipping [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11434: URL: https://github.com/apache/hudi/pull/11434#issuecomment-2161321599 ## CI report: * 9aadf246c5e26b4fe5c48a6b152ef9632d7e586f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [MINOR] Update release guide [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11433: URL: https://github.com/apache/hudi/pull/11433#issuecomment-2161485869 ## CI report: * 34e7b5605dabbae4e81fed0d8c2f32e7f1b61261 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [I] [SUPPORT] Hudi Application getting stuck when Async cleaner is spawned [hudi]

2024-06-11 Thread via GitHub
JuanAmayaBT commented on issue #7364: URL: https://github.com/apache/hudi/issues/7364#issuecomment-2161690376 any news on this? I am using **hudi 0.14.1 on aws glue** and getting from time to time the following error that seems to be related to this issue: Error waiting for async clean se

Re: [PR] [HUDI-6806] Support Spark 3.5.0 [hudi]

2024-06-11 Thread via GitHub
Gatsby-Lee commented on PR #9717: URL: https://github.com/apache/hudi/pull/9717#issuecomment-2161690698 > none what is the release date for Hudi 0.15.0? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] [SUPPORT] org.apache.hudi.common.fs.HoodieWrapperFileSystem cannot be cast to org.apache.hudi.common.fs.HoodieWrapperFileSystem [hudi]

2024-06-11 Thread via GitHub
ankit0811 commented on issue #5457: URL: https://github.com/apache/hudi/issues/5457#issuecomment-2161707842 @danny0405 I am still seeing this exception when I try writing a COW type of table flink `1.15.2` hudi `1.14.1` This is the stack trace ``` Caused by: java.l

Re: [PR] [HUDI-6806] Support Spark 3.5.0 [hudi]

2024-06-11 Thread via GitHub
CTTY commented on PR #9717: URL: https://github.com/apache/hudi/pull/9717#issuecomment-2161730341 Hi @Gatsby-Lee , it should be released last week: https://github.com/apache/hudi/tree/release-0.15.0 -- This is an automated message from the Apache Git Service. To respond to the message, pl

[PR] [HUDI-3304][branch-0.x] Add support for selective partial update [hudi]

2024-06-11 Thread via GitHub
CTTY opened a new pull request, #11435: URL: https://github.com/apache/hudi/pull/11435 ### Change Logs See #9979 ### Impact None ### Risk level (write none, low medium or high below) Medium ### Documentation Update _Describe any necessary docu

Re: [PR] [HUDI-7854] Bump AWS SDK v2 version to 2.25.69 [hudi]

2024-06-11 Thread via GitHub
CTTY commented on PR #11426: URL: https://github.com/apache/hudi/pull/11426#issuecomment-2161761573 LGTM, EMR currently uses `2.25.48` but I don't think there are major API changes between these versions -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] [HUDI-3304][branch-0.x] Add support for selective partial update [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11435: URL: https://github.com/apache/hudi/pull/11435#issuecomment-2161795063 ## CI report: * d41071520eb5b1b1fd9daa24ea3a87710b45bbeb UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-3304][branch-0.x] Add support for selective partial update [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11435: URL: https://github.com/apache/hudi/pull/11435#issuecomment-2161802719 ## CI report: * d41071520eb5b1b1fd9daa24ea3a87710b45bbeb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-6806] Support Spark 3.5.0 [hudi]

2024-06-11 Thread via GitHub
Gatsby-Lee commented on PR #9717: URL: https://github.com/apache/hudi/pull/9717#issuecomment-2161824217 > Hi @Gatsby-Lee , it was released last week: https://github.com/apache/hudi/tree/release-0.15.0 Oh.. so, the doc has not been updated yet. -- This is an automated message from th

[I] [SUPPORT] hoodie.cleaner.commits.retained Not Respected, Warning to Increase to 20 [hudi]

2024-06-11 Thread via GitHub
soumilshah1995 opened a new issue, #11436: URL: https://github.com/apache/hudi/issues/11436 Issue Description: I'm encountering an issue with Hudi configurations related to commit retention and cleaning. Despite explicitly setting hoodie.cleaner.commits.retained to 5, I'm receiving a war

Re: [I] [SUPPORT] hoodie.cleaner.commits.retained Setting Overridden, Warning to Increase to 20 [hudi]

2024-06-11 Thread via GitHub
soumilshah1995 commented on issue #11436: URL: https://github.com/apache/hudi/issues/11436#issuecomment-2161831683 Slack Thread https://apache-hudi.slack.com/archives/C4D716NPQ/p1718110790091359 -- This is an automated message from the Apache Git Service. To respond to the message

Re: [I] [SUPPORT] Unable to Use DynamoDB Based Lock with Hudi PySpark Job Locally [hudi]

2024-06-11 Thread via GitHub
soumilshah1995 closed issue #11391: [SUPPORT] Unable to Use DynamoDB Based Lock with Hudi PySpark Job Locally URL: https://github.com/apache/hudi/issues/11391 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Created] (HUDI-7856) 1.0 Migration

2024-06-11 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7856: - Summary: 1.0 Migration Key: HUDI-7856 URL: https://issues.apache.org/jira/browse/HUDI-7856 Project: Apache Hudi Issue Type: Epic Reporter: Sagar Sumit

[jira] [Created] (HUDI-7857) Handline timeline instants without completion time

2024-06-11 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7857: - Summary: Handline timeline instants without completion time Key: HUDI-7857 URL: https://issues.apache.org/jira/browse/HUDI-7857 Project: Apache Hudi Issue Type: Ta

[jira] [Created] (HUDI-7858) Write archived timeline as LSM timeline

2024-06-11 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7858: - Summary: Write archived timeline as LSM timeline Key: HUDI-7858 URL: https://issues.apache.org/jira/browse/HUDI-7858 Project: Apache Hudi Issue Type: Task

[jira] [Created] (HUDI-7860) Branch out bridge release

2024-06-11 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7860: - Summary: Branch out bridge release Key: HUDI-7860 URL: https://issues.apache.org/jira/browse/HUDI-7860 Project: Apache Hudi Issue Type: Task Reporter:

[jira] [Updated] (HUDI-7858) Write archived timeline as LSM timeline

2024-06-11 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7858: -- Fix Version/s: 1.0.0 > Write archived timeline as LSM timeline > ---

[jira] [Created] (HUDI-7859) Rename instant files to be consistent with 0.x naming format

2024-06-11 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7859: - Summary: Rename instant files to be consistent with 0.x naming format Key: HUDI-7859 URL: https://issues.apache.org/jira/browse/HUDI-7859 Project: Apache Hudi Iss

[jira] [Updated] (HUDI-7859) Rename instant files to be consistent with 0.x naming format

2024-06-11 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7859: -- Fix Version/s: 1.0.0 > Rename instant files to be consistent with 0.x naming format > --

[jira] [Updated] (HUDI-7857) Handline timeline instants without completion time

2024-06-11 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7857: -- Fix Version/s: 1.0.0 > Handline timeline instants without completion time >

Re: [I] [SUPPORT] - Partial update of the MOR table after compaction with Hudi Streamer [hudi]

2024-06-11 Thread via GitHub
soumilshah1995 commented on issue #11348: URL: https://github.com/apache/hudi/issues/11348#issuecomment-2161843209 I was reading thread would this helphttps://soumilshah1995.blogspot.com/2023/05/hands-on-lab-unleashing-efficiency-and.html ``` +---+--+--

[jira] [Created] (HUDI-7861) Pull MDT schema changes to bridge release

2024-06-11 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7861: - Summary: Pull MDT schema changes to bridge release Key: HUDI-7861 URL: https://issues.apache.org/jira/browse/HUDI-7861 Project: Apache Hudi Issue Type: Task

[jira] [Updated] (HUDI-7860) Branch out bridge release

2024-06-11 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7860: -- Fix Version/s: 0.16.0 > Branch out bridge release > - > > Key: H

[jira] [Updated] (HUDI-7860) Branch out bridge release

2024-06-11 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7860: -- Fix Version/s: 1.0.0 > Branch out bridge release > - > > Key: HU

Re: [I] [SUPPORT]Performance degrade for migrating from Hudi 0.7 to Hudi 0.14 [hudi]

2024-06-11 Thread via GitHub
soumilshah1995 commented on issue #11274: URL: https://github.com/apache/hudi/issues/11274#issuecomment-2161845692 Slack Thread https://apache-hudi.slack.com/archives/C4D716NPQ/p1718122404452279 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [HUDI-3304][branch-0.x] Add support for selective partial update [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11435: URL: https://github.com/apache/hudi/pull/11435#issuecomment-2161865614 ## CI report: * d41071520eb5b1b1fd9daa24ea3a87710b45bbeb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

[jira] [Created] (HUDI-7862) Implement LegacyArchivedMetaEntryWriter to rewrite archived timeline

2024-06-11 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7862: - Summary: Implement LegacyArchivedMetaEntryWriter to rewrite archived timeline Key: HUDI-7862 URL: https://issues.apache.org/jira/browse/HUDI-7862 Project: Apache Hudi

[jira] [Updated] (HUDI-7857) Handle timeline instants without completion time

2024-06-11 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7857: -- Summary: Handle timeline instants without completion time (was: Handline timeline instants without comp

[jira] [Created] (HUDI-7863) Ensure full compaction before downgrade

2024-06-11 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7863: - Summary: Ensure full compaction before downgrade Key: HUDI-7863 URL: https://issues.apache.org/jira/browse/HUDI-7863 Project: Apache Hudi Issue Type: Task

Re: [I] [SUPPORT] org.apache.hudi.common.fs.HoodieWrapperFileSystem cannot be cast to org.apache.hudi.common.fs.HoodieWrapperFileSystem [hudi]

2024-06-11 Thread via GitHub
danny0405 commented on issue #5457: URL: https://github.com/apache/hudi/issues/5457#issuecomment-2161871846 @ankit0811 How did you load the hudi flink bundle jar from the classpath then? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [DOCS] Update show_fsview_latest para [hudi]

2024-06-11 Thread via GitHub
danny0405 merged PR #11432: URL: https://github.com/apache/hudi/pull/11432 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

(hudi) branch asf-site updated: [DOCS] Update show_fsview_latest param (#11432)

2024-06-11 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 73d78d413c9 [DOCS] Update show_fsview_lates

[jira] [Assigned] (HUDI-7857) Handle timeline instants without completion time

2024-06-11 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit reassigned HUDI-7857: - Assignee: Danny Chen > Handle timeline instants without completion time > ---

Re: [I] org.apache.hudi.exception.HoodieException: org.apache.avro.AvroTypeException: Cannot encode decimal with precision 14 as max precision 13 [hudi]

2024-06-11 Thread via GitHub
danny0405 commented on issue #11335: URL: https://github.com/apache/hudi/issues/11335#issuecomment-2161874142 > Cannot encode decimal with precision 14 as max precision 13 But from the error msg there looks like a decimal with precision 14. -- This is an automated message from the A

[jira] [Updated] (HUDI-7857) Handle timeline instants without completion time

2024-06-11 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit updated HUDI-7857: -- Sprint: 2024/06/03-16 > Handle timeline instants without completion time > -

Re: [PR] [MINOR] Update release guide [hudi]

2024-06-11 Thread via GitHub
danny0405 merged PR #11433: URL: https://github.com/apache/hudi/pull/11433 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

(hudi) branch master updated: [MINOR] Update release guide (#11433)

2024-06-11 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8eed6aefbc4 [MINOR] Update release guide (#1143

[jira] [Created] (HUDI-7864) Audit all beta2 configs are enabled by default

2024-06-11 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7864: - Summary: Audit all beta2 configs are enabled by default Key: HUDI-7864 URL: https://issues.apache.org/jira/browse/HUDI-7864 Project: Apache Hudi Issue Type: Task

[jira] [Closed] (HUDI-7864) Audit all beta2 configs are enabled by default

2024-06-11 Thread Sagar Sumit (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-7864. - Resolution: Done > Audit all beta2 configs are enabled by default > --

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11385: URL: https://github.com/apache/hudi/pull/11385#issuecomment-2161950406 ## CI report: * f82b7a52ffad611530363dbec1b546d017851a3a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11385: URL: https://github.com/apache/hudi/pull/11385#issuecomment-2161998072 ## CI report: * f82b7a52ffad611530363dbec1b546d017851a3a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11385: URL: https://github.com/apache/hudi/pull/11385#issuecomment-2162047237 ## CI report: * 18ebe4bfd8d145c003ff01ef6341360c836c25f4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [I] [SUPPORT] org.apache.hudi.common.fs.HoodieWrapperFileSystem cannot be cast to org.apache.hudi.common.fs.HoodieWrapperFileSystem [hudi]

2024-06-11 Thread via GitHub
ankit0811 commented on issue #5457: URL: https://github.com/apache/hudi/issues/5457#issuecomment-2162078937 this is part of the pom (compile time) Should this be added explicitly to the class path? -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [I] [SUPPORT] org.apache.hudi.common.fs.HoodieWrapperFileSystem cannot be cast to org.apache.hudi.common.fs.HoodieWrapperFileSystem [hudi]

2024-06-11 Thread via GitHub
danny0405 commented on issue #5457: URL: https://github.com/apache/hudi/issues/5457#issuecomment-2162081264 Then how do you submit the Flink application, the way Flink App jar loaded is very relevent with the classloader loadings. -- This is an automated message from the Apache Git Servic

Re: [I] [SUPPORT] - Partial update of the MOR table after compaction with Hudi Streamer [hudi]

2024-06-11 Thread via GitHub
kirillklimenko commented on issue #11348: URL: https://github.com/apache/hudi/issues/11348#issuecomment-2162113629 > I have not tried streamer route I assume should be similar @soumilshah1995, I can confirm that `org.apache.hudi.common.model.PartialUpdateAvroPayload` works as expected

Re: [PR] [HUDI-7830] Add predicate filter pruning for snapshot queries in hudi related sources [hudi]

2024-06-11 Thread via GitHub
nsivabalan commented on code in PR #11396: URL: https://github.com/apache/hudi/pull/11396#discussion_r1635826968 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/QueryInfo.java: ## @@ -33,6 +37,7 @@ public class QueryInfo { private final String previo

[PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() // to branch-0.x [hudi]

2024-06-11 Thread via GitHub
wombatu-kun opened a new pull request, #11437: URL: https://github.com/apache/hudi/pull/11437 ### Change Logs In HoodieTableMetaClient remove getBasePathV2() and return StoragePath from getBasePath(). (This PR is to branch-0.x as @yihua asked me here https://github.com/apache/

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() [hudi]

2024-06-11 Thread via GitHub
wombatu-kun commented on PR #11385: URL: https://github.com/apache/hudi/pull/11385#issuecomment-2162235303 > Let me know if you prefer to address the `toString()` calls in this PR. Also, could you raise another PR against `branch-0.x` with the same changes? @yihua raised PR against br

Re: [PR] [HUDI-7747] In MetaClient remove getBasePathV2() and return StoragePath from getBasePath() // to branch-0.x [hudi]

2024-06-11 Thread via GitHub
hudi-bot commented on PR #11437: URL: https://github.com/apache/hudi/pull/11437#issuecomment-2162249676 ## CI report: * b45450dde0105f17f667aab9e812df84629ec1b2 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th