[GitHub] [hudi] hudi-bot commented on pull request #8994: [HUDI-6390] Fixes partition specs generation

hudi-bot commented on PR #8994: URL: https://github.com/apache/hudi/pull/8994#issuecomment-1594199233 ## CI report: * 12040380d519a659b09f32b9b289632f596951bc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8963: [DO NOT MERGE] Test fix for RLI

hudi-bot commented on PR #8963: URL: https://github.com/apache/hudi/pull/8963#issuecomment-1594199098 ## CI report: * ff68526b7d0964122a5ce9642fe2ec756e6f16ed UNKNOWN * e6b3c779baf7bd88567539755739469a2773c852 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] wombatu-kun opened a new pull request, #8994: [HUDI-6390] Fixes partition specs generation

wombatu-kun opened a new pull request, #8994: URL: https://github.com/apache/hudi/pull/8994 ### Change Logs Flink COW table reads partition spec from file instead of file path (when the table is not in hive style partitioning), while In MOR table partition specs generation was correc

[GitHub] [hudi] stream2000 commented on pull request #8983: [HUDI-6381] Support bucket bulk insert for CoW table

stream2000 commented on PR #8983: URL: https://github.com/apache/hudi/pull/8983#issuecomment-1594171643 @danny0405 Could you also have a look at this~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[jira] [Closed] (HUDI-6380) Add API to fetch transformed schema in Transformer

[ https://issues.apache.org/jira/browse/HUDI-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Sumit closed HUDI-6380. - Fix Version/s: 0.14.0 Resolution: Fixed > Add API to fetch transformed schema in Transformer >

[GitHub] [hudi] codope commented on a diff in pull request #8982: [HUDI-6380] Add API to fetch transformed schema in Transformer

codope commented on code in PR #8982: URL: https://github.com/apache/hudi/pull/8982#discussion_r1231821022 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/transform/ChainedTransformer.java: ## @@ -112,6 +118,30 @@ private void validateIdentifier(String id, Set identif

[hudi] branch master updated (c892ed0131d -> 4436a48d3f1)

This is an automated email from the ASF dual-hosted git repository. codope pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from c892ed0131d [HUDI-6387] Fix Spark CDC when delete all records from a filegroup (#8990) add 4436a48d3f1 [HUDI-6380]

[GitHub] [hudi] codope merged pull request #8982: [HUDI-6380] Add API to fetch transformed schema in Transformer

codope merged PR #8982: URL: https://github.com/apache/hudi/pull/8982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.or

[GitHub] [hudi] codope commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

codope commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1231819563 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -147,84 +156,55 @@ protected HoodieBackedTableMetadat

[hudi] branch master updated: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup (#8990)

This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new c892ed0131d [HUDI-6387] Fix Spark CDC when delete a

[GitHub] [hudi] yihua merged pull request #8990: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup

yihua merged PR #8990: URL: https://github.com/apache/hudi/pull/8990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

[GitHub] [hudi] lokeshj1703 commented on a diff in pull request #8982: [HUDI-6380] Add API to fetch transformed schema in Transformer

lokeshj1703 commented on code in PR #8982: URL: https://github.com/apache/hudi/pull/8982#discussion_r1231818382 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/transform/ChainedTransformer.java: ## @@ -112,6 +118,30 @@ private void validateIdentifier(String id, Set id

[jira] [Updated] (HUDI-6391) Fix SqlQueryEqualityPreCommitValidator

[ https://issues.apache.org/jira/browse/HUDI-6391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6391: - Labels: pull-request-available (was: ) > Fix SqlQueryEqualityPreCommitValidator > ---

[GitHub] [hudi] yihua opened a new pull request, #8993: [HUDI-6391][WIP] Fix SqlQueryEqualityPreCommitValidator

yihua opened a new pull request, #8993: URL: https://github.com/apache/hudi/pull/8993 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance i

[jira] [Created] (HUDI-6391) Fix SqlQueryEqualityPreCommitValidator

Ethan Guo created HUDI-6391: --- Summary: Fix SqlQueryEqualityPreCommitValidator Key: HUDI-6391 URL: https://issues.apache.org/jira/browse/HUDI-6391 Project: Apache Hudi Issue Type: Bug Re

[GitHub] [hudi] hudi-bot commented on pull request #8985: [HUDI-6383] Hudi bucket index bulk insert into need partition sort to improve speed

hudi-bot commented on PR #8985: URL: https://github.com/apache/hudi/pull/8985#issuecomment-1594153630 ## CI report: * a43543318b8898d0989de8099fed0d805241e59e UNKNOWN * 9c01aac1a5b26cdbb3e7465c9a306370aeb0f5f3 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-

[GitHub] [hudi] hudi-bot commented on pull request #8982: [HUDI-6380] Add API to fetch transformed schema in Transformer

hudi-bot commented on PR #8982: URL: https://github.com/apache/hudi/pull/8982#issuecomment-1594153539 ## CI report: * f275d85204507349b1b18fc87406fd7d2ffc0d04 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1787

[GitHub] [hudi] codope commented on a diff in pull request #8982: [HUDI-6380] Add API to fetch transformed schema in Transformer

codope commented on code in PR #8982: URL: https://github.com/apache/hudi/pull/8982#discussion_r1231809984 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/transform/ChainedTransformer.java: ## @@ -112,6 +118,30 @@ private void validateIdentifier(String id, Set identif

[GitHub] [hudi] hudi-bot commented on pull request #8985: [HUDI-6383] Hudi bucket index bulk insert into need partition sort to improve speed

hudi-bot commented on PR #8985: URL: https://github.com/apache/hudi/pull/8985#issuecomment-1594147053 ## CI report: * a43543318b8898d0989de8099fed0d805241e59e UNKNOWN * 1694414779d7694e09e7071c96665513c383607b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8982: [HUDI-6380] Add API to fetch transformed schema in Transformer

hudi-bot commented on PR #8982: URL: https://github.com/apache/hudi/pull/8982#issuecomment-1594146977 ## CI report: * f275d85204507349b1b18fc87406fd7d2ffc0d04 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] eyjian closed issue #8992: [SUPPORT] Holder DFSClient_NONMAPREDUCE_2077643444_34 does not have any open files

eyjian closed issue #8992: [SUPPORT] Holder DFSClient_NONMAPREDUCE_2077643444_34 does not have any open files URL: https://github.com/apache/hudi/issues/8992 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hudi] eyjian commented on issue #8971: [SUPPORT] is UNDER_RECOVERY but not UNDER_CONSTRUCTION

eyjian commented on issue #8971: URL: https://github.com/apache/hudi/issues/8971#issuecomment-1594100739 > Hi @eyjian thanks for the question. Is the Hudi table on HDFS? Could you share the job configs and `.hoodie/` folder which contains the timeline, markers, etc., to help with triaging?

[GitHub] [hudi] hudi-bot commented on pull request #8983: [HUDI-6381] Support bucket bulk insert for CoW table

hudi-bot commented on PR #8983: URL: https://github.com/apache/hudi/pull/8983#issuecomment-1594091922 ## CI report: * 9f221b1d0b439167f8c1a77041e865ee9d78a3ad UNKNOWN * 8a2cc2138e0d9497ed9fb6e753a1d9876d57da48 UNKNOWN * 9e45d312c7b7499f914687fffd0e50958796cb4d Azure: [SUCCES

[GitHub] [hudi] hudi-bot commented on pull request #8967: [HUDI-6378] allow to delete twice for an empty table

hudi-bot commented on PR #8967: URL: https://github.com/apache/hudi/pull/8967#issuecomment-1594091871 ## CI report: * 96b14a14446288bae5070db221f8d0ea04e98d8f UNKNOWN * 0224bbe7f2108c417b67a5368a22b9994c33e574 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Created] (HUDI-6390) Flink COW table reads partition spec from file instead of file path

Vova Kolmakov created HUDI-6390: --- Summary: Flink COW table reads partition spec from file instead of file path Key: HUDI-6390 URL: https://issues.apache.org/jira/browse/HUDI-6390 Project: Apache Hudi

[GitHub] [hudi] beyond1920 commented on a diff in pull request #4913: [HUDI-1517] create marker file for every log file

beyond1920 commented on code in PR #4913: URL: https://github.com/apache/hudi/pull/4913#discussion_r1231757821 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java: ## @@ -273,4 +280,31 @@ protected static Option toAvroRecord(HoodieRecord re

[GitHub] [hudi] beyond1920 commented on a diff in pull request #4913: [HUDI-1517] create marker file for every log file

beyond1920 commented on code in PR #4913: URL: https://github.com/apache/hudi/pull/4913#discussion_r1231757821 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java: ## @@ -273,4 +280,31 @@ protected static Option toAvroRecord(HoodieRecord re

[GitHub] [hudi] beyond1920 commented on a diff in pull request #4913: [HUDI-1517] create marker file for every log file

beyond1920 commented on code in PR #4913: URL: https://github.com/apache/hudi/pull/4913#discussion_r1231757821 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java: ## @@ -273,4 +280,31 @@ protected static Option toAvroRecord(HoodieRecord re

[GitHub] [hudi] beyond1920 commented on a diff in pull request #4913: [HUDI-1517] create marker file for every log file

beyond1920 commented on code in PR #4913: URL: https://github.com/apache/hudi/pull/4913#discussion_r1231757821 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java: ## @@ -273,4 +280,31 @@ protected static Option toAvroRecord(HoodieRecord re

[GitHub] [hudi] xuzifu666 commented on a diff in pull request #8985: [HUDI-6383] Hudi bucket index bulk insert into need partition sort to improve speed

xuzifu666 commented on code in PR #8985: URL: https://github.com/apache/hudi/pull/8985#discussion_r1231756101 ## hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/BucketPartitionUtils.scala: ## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [hudi] hudi-bot commented on pull request #8985: [HUDI-6383] Hudi bucket index bulk insert into need partition sort to improve speed

hudi-bot commented on PR #8985: URL: https://github.com/apache/hudi/pull/8985#issuecomment-1594056619 ## CI report: * a43543318b8898d0989de8099fed0d805241e59e UNKNOWN * 1694414779d7694e09e7071c96665513c383607b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Closed] (HUDI-6384) The commit action type of inflight instant for replacecommit should be replacecommit

[ https://issues.apache.org/jira/browse/HUDI-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6384. Resolution: Fixed Fixed via master branch: bf2b712c1c48981437624fa883ff25e564a25fd0 > The commit action typ

[hudi] branch master updated: [HUDI-6384] The commit action type of inflight instant for replacecommit should be replacecommit (#8987)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new bf2b712c1c4 [HUDI-6384] The commit action type

[GitHub] [hudi] danny0405 commented on a diff in pull request #8985: [HUDI-6383] Hudi bucket index bulk insert into need partition sort to improve speed

danny0405 commented on code in PR #8985: URL: https://github.com/apache/hudi/pull/8985#discussion_r1231713717 ## hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/BucketPartitionUtils.scala: ## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [hudi] hudi-bot commented on pull request #8987: [HUDI-6384] The commit action type of inflight instant for replacecommit should be replacecommit

hudi-bot commented on PR #8987: URL: https://github.com/apache/hudi/pull/8987#issuecomment-1594049165 ## CI report: * 11db9770db47fb150cfd2fcf0eaaf7047f26aca4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1788

[GitHub] [hudi] hudi-bot commented on pull request #8985: [HUDI-6383] Hudi bucket index bulk insert into need partition sort to improve speed

hudi-bot commented on PR #8985: URL: https://github.com/apache/hudi/pull/8985#issuecomment-1594049059 ## CI report: * a43543318b8898d0989de8099fed0d805241e59e UNKNOWN * 1694414779d7694e09e7071c96665513c383607b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] KnightChess commented on a diff in pull request #8985: [HUDI-6383] Hudi bucket index bulk insert into need partition sort to improve speed

KnightChess commented on code in PR #8985: URL: https://github.com/apache/hudi/pull/8985#discussion_r1231744115 ## hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/BucketPartitionUtils.scala: ## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [hudi] danny0405 commented on a diff in pull request #8668: [HUDI-3639] Add Proper Incremental Records FIltering support into Hudi's custom RDD

danny0405 commented on code in PR #8668: URL: https://github.com/apache/hudi/pull/8668#discussion_r1231736743 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala: ## @@ -116,7 +125,33 @@ class HoodieMergeOnReadRDD(@transient sc: S

[hudi] branch master updated: [MINOR] Add database name to flink operator uid and operator name (#8961)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 5fadd162826 [MINOR] Add database name to flink

[GitHub] [hudi] danny0405 merged pull request #8961: [MINOR] Add database name to flink operator uid and operator name

danny0405 merged PR #8961: URL: https://github.com/apache/hudi/pull/8961 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] boneanxs commented on a diff in pull request #8668: [HUDI-3639] Add Proper Incremental Records FIltering support into Hudi's custom RDD

boneanxs commented on code in PR #8668: URL: https://github.com/apache/hudi/pull/8668#discussion_r1231728451 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala: ## @@ -116,7 +125,33 @@ class HoodieMergeOnReadRDD(@transient sc: Sp

[GitHub] [hudi] SteNicholas commented on a diff in pull request #8987: [HUDI-6384] The commit action type of inflight instant for replacecommit should be replacecommit

SteNicholas commented on code in PR #8987: URL: https://github.com/apache/hudi/pull/8987#discussion_r1231127038 ## hudi-common/src/main/java/org/apache/hudi/common/model/WriteOperationType.java: ## @@ -68,6 +68,10 @@ public enum WriteOperationType { * Convert string value to

[GitHub] [hudi] boneanxs commented on a diff in pull request #8668: [HUDI-3639] Add Proper Incremental Records FIltering support into Hudi's custom RDD

boneanxs commented on code in PR #8668: URL: https://github.com/apache/hudi/pull/8668#discussion_r1231728451 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala: ## @@ -116,7 +125,33 @@ class HoodieMergeOnReadRDD(@transient sc: Sp

[GitHub] [hudi] hudi-bot commented on pull request #8983: [HUDI-6381] Support bucket bulk insert for CoW table

hudi-bot commented on PR #8983: URL: https://github.com/apache/hudi/pull/8983#issuecomment-1594018027 ## CI report: * 9f221b1d0b439167f8c1a77041e865ee9d78a3ad UNKNOWN * 8a2cc2138e0d9497ed9fb6e753a1d9876d57da48 UNKNOWN * 81da2aac34f7c81013e797c03ec645097f37c5cb Azure: [SUCCES

[GitHub] [hudi] boneanxs commented on a diff in pull request #8668: [HUDI-3639] Add Proper Incremental Records FIltering support into Hudi's custom RDD

boneanxs commented on code in PR #8668: URL: https://github.com/apache/hudi/pull/8668#discussion_r1231725467 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala: ## @@ -116,7 +125,33 @@ class HoodieMergeOnReadRDD(@transient sc: Sp

[GitHub] [hudi] danny0405 commented on a diff in pull request #8668: [HUDI-3639] Add Proper Incremental Records FIltering support into Hudi's custom RDD

danny0405 commented on code in PR #8668: URL: https://github.com/apache/hudi/pull/8668#discussion_r1231723303 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala: ## @@ -55,7 +55,7 @@ case class MergeOnReadIncrementalRel

[GitHub] [hudi] hudi-bot commented on pull request #8983: [HUDI-6381] Support bucket bulk insert for CoW table

hudi-bot commented on PR #8983: URL: https://github.com/apache/hudi/pull/8983#issuecomment-1594013822 ## CI report: * 9f221b1d0b439167f8c1a77041e865ee9d78a3ad UNKNOWN * 8a2cc2138e0d9497ed9fb6e753a1d9876d57da48 UNKNOWN * 81da2aac34f7c81013e797c03ec645097f37c5cb Azure: [SUCCES

[GitHub] [hudi] hudi-bot commented on pull request #8967: [HUDI-6378] allow to delete twice for an empty table

hudi-bot commented on PR #8967: URL: https://github.com/apache/hudi/pull/8967#issuecomment-1594013793 ## CI report: * 96b14a14446288bae5070db221f8d0ea04e98d8f UNKNOWN * 7a009c2ca0befdd73b7c14a382a76ff14d7d16b7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] boneanxs commented on a diff in pull request #8668: [HUDI-3639] Add Proper Incremental Records FIltering support into Hudi's custom RDD

boneanxs commented on code in PR #8668: URL: https://github.com/apache/hudi/pull/8668#discussion_r1231721948 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadIncrementalRelation.scala: ## @@ -55,7 +55,7 @@ case class MergeOnReadIncrementalRela

[GitHub] [hudi] boneanxs commented on a diff in pull request #6045: [WIP][HUDI-4345] Enable partition pruning and projection for incremental query on COW table

boneanxs commented on code in PR #6045: URL: https://github.com/apache/hudi/pull/6045#discussion_r1231718172 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/CopyOnWriteIncrementalRelation.scala: ## @@ -0,0 +1,196 @@ +/* + * Licensed to the Apache Softwa

[GitHub] [hudi] danny0405 commented on a diff in pull request #8985: [HUDI-6383] Hudi bucket index bulk insert into need partition sort to improve speed

danny0405 commented on code in PR #8985: URL: https://github.com/apache/hudi/pull/8985#discussion_r1231713717 ## hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/BucketPartitionUtils.scala: ## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [hudi] danny0405 commented on a diff in pull request #8944: [HUDI-6359]Spark offline compaction/clustering will never rollback when both requested and inflight states exist

danny0405 commented on code in PR #8944: URL: https://github.com/apache/hudi/pull/8944#discussion_r1231711906 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieClusteringJob.java: ## @@ -209,8 +209,7 @@ private int doCluster(JavaSparkContext jsc) throws Exception

[GitHub] [hudi] danny0405 commented on pull request #8668: [HUDI-3639] Add Proper Incremental Records FIltering support into Hudi's custom RDD

danny0405 commented on PR #8668: URL: https://github.com/apache/hudi/pull/8668#issuecomment-1594000489 > > can we write a test case for the mor inc read scenario? > > I tried to write several UTs, but they are not very easy to reproduce, and I am still working hard. Thanks for

[jira] [Closed] (HUDI-6350) AWS Hive sync: allow to enable/disable MDT on athena

[ https://issues.apache.org/jira/browse/HUDI-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6350. Fix Version/s: 0.14.0 Resolution: Fixed Fixed via master branch: 43b4852a1b1fdf5ef28acc44f1c5a1268ee6

[hudi] branch master updated: [HUDI-6350] Allow athena to use the metadata table (#8929)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 43b4852a1b1 [HUDI-6350] Allow athena to use the

[GitHub] [hudi] danny0405 merged pull request #8929: [HUDI-6350] Allow athena to use the metadata table

danny0405 merged PR #8929: URL: https://github.com/apache/hudi/pull/8929 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] eyjian opened a new issue, #8992: [SUPPORT] Holder DFSClient_NONMAPREDUCE_2077643444_34 does not have any open files

eyjian opened a new issue, #8992: URL: https://github.com/apache/hudi/issues/8992 Hudi: 0.12.1 Spark: 3.1 ``` Caused by: java.sql.SQLException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.hudi.exception.HoodieRollbackException: Failed to roll

[GitHub] [hudi] danny0405 commented on pull request #8729: [HUDI-6223] Support Incremental read via Spark SQL table valued function

danny0405 commented on PR #8729: URL: https://github.com/apache/hudi/pull/8729#issuecomment-1593981495 Hi, @kazdy , are there any updates here, the 0.14.0 release code freeze is due to the next weekend. -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [hudi] hudi-bot commented on pull request #8967: [HUDI-6378] allow to delete twice for an empty table

hudi-bot commented on PR #8967: URL: https://github.com/apache/hudi/pull/8967#issuecomment-1593978328 ## CI report: * 96b14a14446288bae5070db221f8d0ea04e98d8f UNKNOWN * 7a009c2ca0befdd73b7c14a382a76ff14d7d16b7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] zinking commented on issue #6428: [BUG] S3 Deltastreamer: Block has already been inflated

zinking commented on issue #6428: URL: https://github.com/apache/hudi/issues/6428#issuecomment-159394 > @kasured to unblock the processing, could you try disabling and deleting the metadata table by setting `hoodie.metadata.enable=false` in Hudi configs? This automatically deletes the m

[GitHub] [hudi] lvyanquan commented on a diff in pull request #8967: [HUDI-6378] allow to delete twice for an empty table

lvyanquan commented on code in PR #8967: URL: https://github.com/apache/hudi/pull/8967#discussion_r1231700047 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java: ## @@ -1255,10 +1255,14 @@ protected void setWriteSchemaForDeletes(Ho

[GitHub] [hudi] danny0405 commented on a diff in pull request #8990: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup

danny0405 commented on code in PR #8990: URL: https://github.com/apache/hudi/pull/8990#discussion_r1231697487 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestCDCForSparkSQL.scala: ## @@ -44,6 +44,47 @@ class TestCDCForSparkSQL extends HoodieSpark

[GitHub] [hudi] danny0405 commented on a diff in pull request #8990: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup

danny0405 commented on code in PR #8990: URL: https://github.com/apache/hudi/pull/8990#discussion_r1231697257 ## hudi-common/src/main/java/org/apache/hudi/common/table/cdc/HoodieCDCExtractor.java: ## @@ -265,7 +265,7 @@ private HoodieCDCFileSplit parseWriteStat( n

[GitHub] [hudi] danny0405 commented on a diff in pull request #8987: [HUDI-6384] The commit action type of inflight instant for replacecommit should be replacecommit

danny0405 commented on code in PR #8987: URL: https://github.com/apache/hudi/pull/8987#discussion_r1231690955 ## hudi-common/src/main/java/org/apache/hudi/common/model/WriteOperationType.java: ## @@ -68,6 +68,10 @@ public enum WriteOperationType { * Convert string value to W

[GitHub] [hudi] danny0405 commented on a diff in pull request #8987: [HUDI-6384] The commit action type of inflight instant for replacecommit should be replacecommit

danny0405 commented on code in PR #8987: URL: https://github.com/apache/hudi/pull/8987#discussion_r1231692148 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java: ## @@ -223,7 +223,7 @@ public boolean commitStats(String instantTime,

[jira] [Closed] (HUDI-6366) fix flink offline table service rerun issue

[ https://issues.apache.org/jira/browse/HUDI-6366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6366. Fix Version/s: 0.14.0 Resolution: Fixed Fixed via master branch: cc33f53c794a3295044daacaaed900c77274

[GitHub] [hudi] hudi-bot commented on pull request #8991: [HUDI-6389] Fix instant time check against the active timeline in meta sync

hudi-bot commented on PR #8991: URL: https://github.com/apache/hudi/pull/8991#issuecomment-1593902393 ## CI report: * da5187e1e40a9499c83ad6382c5ae70a9c9617bf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1790

[GitHub] [hudi] hudi-bot commented on pull request #8990: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup

hudi-bot commented on PR #8990: URL: https://github.com/apache/hudi/pull/8990#issuecomment-1593856361 ## CI report: * 46c421b0666395d3fb84efb9777d96c8cebe83b0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1789

[GitHub] [hudi] hudi-bot commented on pull request #8963: [DO NOT MERGE] Test fix for RLI

hudi-bot commented on PR #8963: URL: https://github.com/apache/hudi/pull/8963#issuecomment-1593852315 ## CI report: * ff68526b7d0964122a5ce9642fe2ec756e6f16ed UNKNOWN * e6b3c779baf7bd88567539755739469a2773c852 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8875: [HUDI-6311] Spark SQL Insert Into should use bulk insert by default for out of the box experience

hudi-bot commented on PR #8875: URL: https://github.com/apache/hudi/pull/8875#issuecomment-1593852184 ## CI report: * 8b79bae989ef9e71935ee3824f886c6da1c3e2c9 UNKNOWN * 3218a3e822008884f8fe00e922006716a337ae2a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] xiedeyantu commented on issue #8979: [SUPPORT] setup_demo.sh can support MacOS M1

xiedeyantu commented on issue #8979: URL: https://github.com/apache/hudi/issues/8979#issuecomment-1593834219 > @xiedeyantu Did you tried using `--mac-aarch64`, Are you still facing this error with that flag? yes -- This is an automated message from the Apache Git Service. To respon

[GitHub] [hudi] hudi-bot commented on pull request #8991: [HUDI-6389] Fix instant time check against the active timeline in meta sync

hudi-bot commented on PR #8991: URL: https://github.com/apache/hudi/pull/8991#issuecomment-1593819851 ## CI report: * da5187e1e40a9499c83ad6382c5ae70a9c9617bf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1790

[GitHub] [hudi] hudi-bot commented on pull request #8991: [HUDI-6389] Fix instant time check against the active timeline in meta sync

hudi-bot commented on PR #8991: URL: https://github.com/apache/hudi/pull/8991#issuecomment-1593814265 ## CI report: * da5187e1e40a9499c83ad6382c5ae70a9c9617bf UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

hudi-bot commented on PR #8758: URL: https://github.com/apache/hudi/pull/8758#issuecomment-1593813756 ## CI report: * b74960431e2fa1b7aace8c9455ecc6270523bce6 UNKNOWN * f5c279925d939278a81afee538a99c64ebd7fcf0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] yihua commented on a diff in pull request #7561: [HUDI-5477] Optimize timeline loading in Hudi sync client

yihua commented on code in PR #7561: URL: https://github.com/apache/hudi/pull/7561#discussion_r1231573133 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimelineUtils.java: ## @@ -210,11 +210,30 @@ public static HoodieDefaultTimeline getTimeline(HoodieTableM

[jira] [Updated] (HUDI-6389) Fix instant time check against the active timeline in meta sync

[ https://issues.apache.org/jira/browse/HUDI-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-6389: - Labels: pull-request-available (was: ) > Fix instant time check against the active timeline in me

[GitHub] [hudi] yihua opened a new pull request, #8991: [HUDI-6389] Fix instant time check against the active timeline in meta sync

yihua opened a new pull request, #8991: URL: https://github.com/apache/hudi/pull/8991 ### Change Logs #7561 introduced a bug where in a case, the partition changes may be missed in the meta sync, e.g., the following active timeline: ``` ts3.rollback, ts50.commit, ts51.commit

[GitHub] [hudi] hudi-bot commented on pull request #8988: Bump snappy-java from 1.1.8.3 to 1.1.10.1

hudi-bot commented on PR #8988: URL: https://github.com/apache/hudi/pull/8988#issuecomment-1593761871 ## CI report: * a668d776058940757480edcc228972d3389d5988 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1789

[GitHub] [hudi] hudi-bot commented on pull request #8990: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup

hudi-bot commented on PR #8990: URL: https://github.com/apache/hudi/pull/8990#issuecomment-1593702927 ## CI report: * 7a1da85eaf1994379dfa22a94579d09433cde477 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=178

[GitHub] [hudi] hudi-bot commented on pull request #8929: [HUDI-6350] Allow athena to use the metadata table

hudi-bot commented on PR #8929: URL: https://github.com/apache/hudi/pull/8929#issuecomment-1593702638 ## CI report: * 2a0b024a8c3a088ce44be9328ace1bad8648eea2 UNKNOWN * 6db6c357d7d1c27fc171da68f59216e2c1df772e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Updated] (HUDI-6389) Fix instant time check against the active timeline in meta sync

[ https://issues.apache.org/jira/browse/HUDI-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6389: Priority: Blocker (was: Major) > Fix instant time check against the active timeline in meta sync >

[jira] [Assigned] (HUDI-6389) Fix instant time check against the active timeline in meta sync

[ https://issues.apache.org/jira/browse/HUDI-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-6389: --- Assignee: Ethan Guo > Fix instant time check against the active timeline in meta sync > -

[jira] [Updated] (HUDI-6389) Fix instant time check against the active timeline in meta sync

[ https://issues.apache.org/jira/browse/HUDI-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6389: Fix Version/s: 0.14.0 > Fix instant time check against the active timeline in meta sync > --

[jira] [Updated] (HUDI-6389) Fix instant time check against the active timeline in meta sync

[ https://issues.apache.org/jira/browse/HUDI-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6389: Description: [https://github.com/apache/hudi/pull/7561/files/f61d9480f00b3f79d71044e0392ed6e2f10af4ad#r12036

[jira] [Created] (HUDI-6389) Fix instant time check against the active timeline in meta sync

Ethan Guo created HUDI-6389: --- Summary: Fix instant time check against the active timeline in meta sync Key: HUDI-6389 URL: https://issues.apache.org/jira/browse/HUDI-6389 Project: Apache Hudi Issu

[GitHub] [hudi] hudi-bot commented on pull request #8990: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup

hudi-bot commented on PR #8990: URL: https://github.com/apache/hudi/pull/8990#issuecomment-1593656272 ## CI report: * 7a1da85eaf1994379dfa22a94579d09433cde477 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1789

[GitHub] [hudi] hudi-bot commented on pull request #8987: [HUDI-6384] The commit action type of inflight instant for replacecommit should be replacecommit

hudi-bot commented on PR #8987: URL: https://github.com/apache/hudi/pull/8987#issuecomment-1593626393 ## CI report: * 11db9770db47fb150cfd2fcf0eaaf7047f26aca4 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1788

[GitHub] [hudi] hudi-bot commented on pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

hudi-bot commented on PR #8758: URL: https://github.com/apache/hudi/pull/8758#issuecomment-1593625496 ## CI report: * 5abf0d726c3a04d2e46bb9652ee82ab1ab6655df Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1786

[GitHub] [hudi] hudi-bot commented on pull request #8668: [HUDI-3639] Add Proper Incremental Records FIltering support into Hudi's custom RDD

hudi-bot commented on PR #8668: URL: https://github.com/apache/hudi/pull/8668#issuecomment-1593625101 ## CI report: * 0e677a1d959139b8805fc35a177bcf2db954a43e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1788

[GitHub] [hudi] hudi-bot commented on pull request #8990: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup

hudi-bot commented on PR #8990: URL: https://github.com/apache/hudi/pull/8990#issuecomment-1593579591 ## CI report: * 7a1da85eaf1994379dfa22a94579d09433cde477 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1789

[GitHub] [hudi] hudi-bot commented on pull request #8875: [HUDI-6311] Spark SQL Insert Into should use bulk insert by default for out of the box experience

hudi-bot commented on PR #8875: URL: https://github.com/apache/hudi/pull/8875#issuecomment-1593579114 ## CI report: * 8b79bae989ef9e71935ee3824f886c6da1c3e2c9 UNKNOWN * 9dc35d6c4dfcc8b98efd9a35da70f862950a09d9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

hudi-bot commented on PR #8758: URL: https://github.com/apache/hudi/pull/8758#issuecomment-1593578653 ## CI report: * 5abf0d726c3a04d2e46bb9652ee82ab1ab6655df Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1786

[GitHub] [hudi] hudi-bot commented on pull request #8990: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup

hudi-bot commented on PR #8990: URL: https://github.com/apache/hudi/pull/8990#issuecomment-1593569903 ## CI report: * 7a1da85eaf1994379dfa22a94579d09433cde477 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #8875: [HUDI-6311] Spark SQL Insert Into should use bulk insert by default for out of the box experience

hudi-bot commented on PR #8875: URL: https://github.com/apache/hudi/pull/8875#issuecomment-1593569478 ## CI report: * 8b79bae989ef9e71935ee3824f886c6da1c3e2c9 UNKNOWN * 9dc35d6c4dfcc8b98efd9a35da70f862950a09d9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

hudi-bot commented on PR #8758: URL: https://github.com/apache/hudi/pull/8758#issuecomment-1593569126 ## CI report: * 5abf0d726c3a04d2e46bb9652ee82ab1ab6655df Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1786

[GitHub] [hudi] prashantwason commented on pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

prashantwason commented on PR #8758: URL: https://github.com/apache/hudi/pull/8758#issuecomment-1593563939 @xushiyan @danny0405 @codope @nsivabalan I have addressed all feedback. PTAL and resolve the open comments. -- This is an automated message from the Apache Git Service. To respond t

[GitHub] [hudi] yihua commented on a diff in pull request #8990: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup

yihua commented on code in PR #8990: URL: https://github.com/apache/hudi/pull/8990#discussion_r1231414550 ## hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestCDCForSparkSQL.scala: ## @@ -44,6 +44,47 @@ class TestCDCForSparkSQL extends HoodieSparkSqlT

[GitHub] [hudi] hudi-bot commented on pull request #8875: [HUDI-6311] Spark SQL Insert Into should use bulk insert by default for out of the box experience

hudi-bot commented on PR #8875: URL: https://github.com/apache/hudi/pull/8875#issuecomment-1593559892 ## CI report: * 8b79bae989ef9e71935ee3824f886c6da1c3e2c9 UNKNOWN * 9dc35d6c4dfcc8b98efd9a35da70f862950a09d9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] yihua commented on pull request #8990: [HUDI-6387] Fix Spark CDC when delete all records from a filegroup

yihua commented on PR #8990: URL: https://github.com/apache/hudi/pull/8990#issuecomment-1593559413 @danny0405 @YannByron could you review too? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] prashantwason commented on a diff in pull request #8758: [HUDI-53] Implementation of record_index - a HUDI index based on the metadata table.

prashantwason commented on code in PR #8758: URL: https://github.com/apache/hudi/pull/8758#discussion_r1231403862 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/index/SparkMetadataTableRecordIndex.java: ## @@ -0,0 +1,253 @@ +/* + * Licensed to the Apache Software

  1   2   3   >