Re: [PR] [HUDI-7144] Build storage partition stats index and use it for data skipping [hudi]

2024-04-24 Thread via GitHub
codope commented on code in PR #10352: URL: https://github.com/apache/hudi/pull/10352#discussion_r1578973431 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -224,6 +224,10 @@ private void enablePartitions() {

Re: [PR] [HUDI-7144] Build storage partition stats index and use it for data skipping [hudi]

2024-04-24 Thread via GitHub
codope commented on code in PR #10352: URL: https://github.com/apache/hudi/pull/10352#discussion_r1578976365 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/upgrade/TestUpgradeDowngrade.java: ## @@ -557,7 +558,8 @@ public void testDowngradeSixToFiveShouldDe

[jira] [Created] (HUDI-7665) Upgrade Table Version

2024-04-24 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7665: - Summary: Upgrade Table Version Key: HUDI-7665 URL: https://issues.apache.org/jira/browse/HUDI-7665 Project: Apache Hudi Issue Type: Task Reporter: Saga

[jira] [Updated] (HUDI-7664) Remove Hadoop dependency from hudi-io module

2024-04-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7664: - Labels: hoodie-storage pull-request-available (was: hoodie-storage) > Remove Hadoop dependency fr

[PR] [HUDI-7664] Remove Hadoop dependency from hudi-io module [hudi]

2024-04-24 Thread via GitHub
yihua opened a new pull request, #11089: URL: https://github.com/apache/hudi/pull/11089 ### Change Logs This PR removes usage of Hadoop classes and the Hadoop dependency from `hudi-io` module. ### Impact Makes `hudi-io` module Hadoop indepedent. ### Risk level

Re: [PR] [HUDI-7588][DO NOT MERGE][Stacked on others] Replace hadoop Configuration with StorageConfiguration in meta client [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11071: URL: https://github.com/apache/hudi/pull/11071#issuecomment-2076487365 ## CI report: * a9972fa230cade524432f51566a8e6345b39afb9 UNKNOWN * 28166f6623f712b3b29ee3410552b3788d2f66b1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7144] Build storage partition stats index and use it for data skipping [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10352: URL: https://github.com/apache/hudi/pull/10352#issuecomment-2076486121 ## CI report: * ab97c0d984a8336f781cb9a646c0a22421e7ace0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7144] Build storage partition stats index and use it for data skipping [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10352: URL: https://github.com/apache/hudi/pull/10352#issuecomment-2076475604 ## CI report: * ab97c0d984a8336f781cb9a646c0a22421e7ace0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7588][DO NOT MERGE][Stacked on others] Replace hadoop Configuration with StorageConfiguration in meta client [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11071: URL: https://github.com/apache/hudi/pull/11071#issuecomment-2076477487 ## CI report: * a9972fa230cade524432f51566a8e6345b39afb9 UNKNOWN * 28166f6623f712b3b29ee3410552b3788d2f66b1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

[jira] [Updated] (HUDI-7664) Remove Hadoop dependency from hudi-io module

2024-04-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7664: Epic Link: HUDI-6243 > Remove Hadoop dependency from hudi-io module > --

[jira] [Updated] (HUDI-7664) Remove Hadoop dependency from hudi-io module

2024-04-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7664: Fix Version/s: 0.15.0 1.0.0 > Remove Hadoop dependency from hudi-io module >

[jira] [Updated] (HUDI-7664) Remove Hadoop dependency from hudi-io module

2024-04-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7664: Priority: Blocker (was: Major) > Remove Hadoop dependency from hudi-io module > ---

[jira] [Assigned] (HUDI-7664) Remove Hadoop dependency from hudi-io module

2024-04-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-7664: --- Assignee: Ethan Guo > Remove Hadoop dependency from hudi-io module >

[jira] [Updated] (HUDI-7664) Remove Hadoop dependency from hudi-io module

2024-04-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7664: Labels: hoodie-storage (was: ) > Remove Hadoop dependency from hudi-io module > ---

[jira] [Created] (HUDI-7664) Remove Hadoop dependency from hudi-io module

2024-04-24 Thread Ethan Guo (Jira)
Ethan Guo created HUDI-7664: --- Summary: Remove Hadoop dependency from hudi-io module Key: HUDI-7664 URL: https://issues.apache.org/jira/browse/HUDI-7664 Project: Apache Hudi Issue Type: Improvement

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
danny0405 commented on code in PR #11073: URL: https://github.com/apache/hudi/pull/11073#discussion_r1578943411 ## pom.xml: ## @@ -2040,6 +2040,27 @@ + +org.jacoco +jacoco-maven-plugin + +

Re: [PR] [HUDI-7588][DO NOT MERGE][Stacked on others] Replace hadoop Configuration with StorageConfiguration in meta client [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11071: URL: https://github.com/apache/hudi/pull/11071#issuecomment-2076467738 ## CI report: * a9972fa230cade524432f51566a8e6345b39afb9 UNKNOWN * 28166f6623f712b3b29ee3410552b3788d2f66b1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
danny0405 commented on code in PR #11073: URL: https://github.com/apache/hudi/pull/11073#discussion_r1578937570 ## pom.xml: ## @@ -2040,6 +2040,27 @@ + +org.jacoco +jacoco-maven-plugin + +

Re: [PR] [HUDI-7588][DO NOT MERGE][Stacked on others] Replace hadoop Configuration with StorageConfiguration in meta client [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11071: URL: https://github.com/apache/hudi/pull/11071#issuecomment-2076418347 ## CI report: * a9972fa230cade524432f51566a8e6345b39afb9 UNKNOWN * 5fe34eea207b1be8790842c5ec687527c9aa99c7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7652] Add new `HoodieMergeKey` API to support simple and composite keys [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11077: URL: https://github.com/apache/hudi/pull/11077#issuecomment-2076418403 ## CI report: * 7ee2125aa1fa318c177a689282996d510a75ac52 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7511] Fixing offset range calculation for kafka [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10875: URL: https://github.com/apache/hudi/pull/10875#issuecomment-2076417986 ## CI report: * 31f4aea64312ba3a72ca9116aa1f0d913661f658 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7652] Add new `HoodieMergeKey` API to support simple and composite keys [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11077: URL: https://github.com/apache/hudi/pull/11077#issuecomment-2076411707 ## CI report: * 19a23e39e15d2818d28956959dc00f09bc51 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7588][DO NOT MERGE][Stacked on others] Replace hadoop Configuration with StorageConfiguration in meta client [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11071: URL: https://github.com/apache/hudi/pull/11071#issuecomment-2076411654 ## CI report: * a9972fa230cade524432f51566a8e6345b39afb9 UNKNOWN * 5fe34eea207b1be8790842c5ec687527c9aa99c7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7645] Optimize BQ sync tool for MDT [hudi]

2024-04-24 Thread via GitHub
bvaradar commented on code in PR #11065: URL: https://github.com/apache/hudi/pull/11065#discussion_r1578897695 ## hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/util/ManifestFileWriter.java: ## @@ -85,16 +85,22 @@ public synchronized void writeManifestFile(

(hudi) branch master updated (9d689fd68f5 -> 835d47367e3)

2024-04-24 Thread vbalaji
This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 9d689fd68f5 [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse (#11067) add 835d47367e3 [HUDI-764

Re: [PR] [HUDI-7645] Optimize BQ sync tool for MDT [hudi]

2024-04-24 Thread via GitHub
bvaradar merged PR #11065: URL: https://github.com/apache/hudi/pull/11065 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

Re: [PR] [HUDI-7652] Add new `HoodieMergeKey` API to support simple and composite keys [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11077: URL: https://github.com/apache/hudi/pull/11077#issuecomment-2076404467 ## CI report: * 19a23e39e15d2818d28956959dc00f09bc51 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11073: URL: https://github.com/apache/hudi/pull/11073#issuecomment-2076394737 ## CI report: * 39c44a33eaae3bc17270cec93536ce727daacd98 UNKNOWN * d329d241fef55c438e72343e4564671df92befff UNKNOWN * 37cf7585527469ea75ee8b5d972910837273852d Azure: [SUCC

(hudi) branch master updated: [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse (#11067)

2024-04-24 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 9d689fd68f5 [HUDI-7648] Refactor MetadataPartitionT

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
yihua merged PR #11067: URL: https://github.com/apache/hudi/pull/11067 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.o

Re: [PR] [HUDI-7629] Safely rename HoodieFileStatus [hudi]

2024-04-24 Thread via GitHub
wombatu-kun closed pull request #11055: [HUDI-7629] Safely rename HoodieFileStatus URL: https://github.com/apache/hudi/pull/11055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [HUDI-7511] Fixing offset range calculation for kafka [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10875: URL: https://github.com/apache/hudi/pull/10875#issuecomment-2076345614 ## CI report: * fc1ac188627e04bca134e0da817d0f3777331483 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-7511] Fixing offset range calculation for kafka [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10875: URL: https://github.com/apache/hudi/pull/10875#issuecomment-2076340927 ## CI report: * fc1ac188627e04bca134e0da817d0f3777331483 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11067: URL: https://github.com/apache/hudi/pull/11067#issuecomment-2076335844 ## CI report: * 35086fa718fcc505d1a975763efe5426b7b8e71a UNKNOWN * 773d4c62a4754ac9b2029dc4d4789caba01f372f UNKNOWN * 8eed8247656c6d231c838dd4650ac6fbffd047dc Azure: [SUCC

Re: [PR] [HUDI-7511] Fixing offset range calculation for kafka [hudi]

2024-04-24 Thread via GitHub
bvaradar commented on code in PR #10875: URL: https://github.com/apache/hudi/pull/10875#discussion_r1578824134 ## hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java: ## @@ -174,6 +174,13 @@ public static OffsetRange[] computeOffsetRanges(

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11073: URL: https://github.com/apache/hudi/pull/11073#issuecomment-2076291982 ## CI report: * 39c44a33eaae3bc17270cec93536ce727daacd98 UNKNOWN * d329d241fef55c438e72343e4564671df92befff UNKNOWN * bcd9ef9989ece8bd4b5b7406ce5cc85b7b223131 Azure: [SUCC

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11073: URL: https://github.com/apache/hudi/pull/11073#issuecomment-2076287701 ## CI report: * 39c44a33eaae3bc17270cec93536ce727daacd98 UNKNOWN * d329d241fef55c438e72343e4564671df92befff UNKNOWN * bcd9ef9989ece8bd4b5b7406ce5cc85b7b223131 Azure: [SUCC

(hudi) branch master updated (f8e559f5e6f -> fb022491b2b)

2024-04-24 Thread vbalaji
This is an automated email from the ASF dual-hosted git repository. vbalaji pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from f8e559f5e6f [HUDI-7660] Fix excessive object creation in RowDataKeyGen (#11084) add fb022491b2b [HUDI-7235] Fix c

Re: [PR] [HUDI-7235] Fix checkpoint bug for S3/GCS Incremental Source [hudi]

2024-04-24 Thread via GitHub
bvaradar merged PR #10336: URL: https://github.com/apache/hudi/pull/10336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[jira] [Assigned] (HUDI-7646) Consistent naming in Compaction service

2024-04-24 Thread Geser Dugarov (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geser Dugarov reassigned HUDI-7646: --- Assignee: Geser Dugarov > Consistent naming in Compaction service > -

[jira] [Comment Edited] (HUDI-7596) Enable Jacoco code coverage report across multiple modules

2024-04-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838411#comment-17838411 ] Danny Chen edited comment on HUDI-7596 at 4/25/24 3:05 AM: --- The

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11067: URL: https://github.com/apache/hudi/pull/11067#issuecomment-2076253054 ## CI report: * a3049b2f019185ffb23ec7f84c7ce602836b1b06 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11067: URL: https://github.com/apache/hudi/pull/11067#issuecomment-2076247948 ## CI report: * a3049b2f019185ffb23ec7f84c7ce602836b1b06 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11067: URL: https://github.com/apache/hudi/pull/11067#issuecomment-2076211105 ## CI report: * a3049b2f019185ffb23ec7f84c7ce602836b1b06 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11073: URL: https://github.com/apache/hudi/pull/11073#issuecomment-2076211396 ## CI report: * 39c44a33eaae3bc17270cec93536ce727daacd98 UNKNOWN * d329d241fef55c438e72343e4564671df92befff UNKNOWN * bcd9ef9989ece8bd4b5b7406ce5cc85b7b223131 Azure: [SUCC

Re: [PR] [HUDI-5823] Partition ttl management [hudi]

2024-04-24 Thread via GitHub
xicm commented on code in PR #9723: URL: https://github.com/apache/hudi/pull/9723#discussion_r1578761500 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/ttl/strategy/KeepByTimeStrategy.java: ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Softwar

[jira] [Updated] (HUDI-7663) Cannot discover new partitons when i using stream reading by flink1.1.6.1-hudi13.1

2024-04-24 Thread weitianpei (Jira)
: 96h > Remaining Estimate: 96h > > !image-2024-04-25-09-53-22-731.png! > I am reading hudi multiple-stage table by flink1.16.1-hudi1.13.1's steam > reading feature. > Today is 2024-04-25,but my program cannot read any new parquet in partition > 20240425. > It just r

[jira] [Reopened] (HUDI-7663) Cannot discover new partitons when i using stream reading by flink1.1.6.1-hudi13.1

2024-04-24 Thread weitianpei (Jira)
Estimate: 96h > > !image-2024-04-25-09-53-22-731.png! > I am reading hudi multiple-stage table by flink1.16.1-hudi1.13.1's steam > reading feature. > Today is 2024-04-25,but my program cannot read any new parquet in partition > 20240425. > It just read the data from p

Re: [PR] [HUDI-5823] Partition ttl management [hudi]

2024-04-24 Thread via GitHub
xicm commented on code in PR #9723: URL: https://github.com/apache/hudi/pull/9723#discussion_r1578761500 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/ttl/strategy/KeepByTimeStrategy.java: ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Softwar

[jira] [Resolved] (HUDI-7663) Cannot discover new partitons when i using stream reading by flink1.1.6.1-hudi13.1

2024-04-24 Thread weitianpei (Jira)
Estimate: 96h > > !image-2024-04-25-09-53-22-731.png! > I am reading hudi multiple-stage table by flink1.16.1-hudi1.13.1's steam > reading feature. > Today is 2024-04-25,but my program cannot read any new parquet in partition > 20240425. > It just read the data from p

[jira] [Created] (HUDI-7663) Cannot discover new partitons when i using stream reading by flink1.1.6.1-hudi13.1

2024-04-24 Thread weitianpei (Jira)
! I am reading hudi multiple-stage table by flink1.16.1-hudi1.13.1's steam reading feature. Today is 2024-04-25,but my program cannot read any new parquet in partition 20240425. It just read the data from partition 20240424,since I start my program at yesterday's 2Pm. It is

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
codope commented on code in PR #11067: URL: https://github.com/apache/hudi/pull/11067#discussion_r1578755511 ## hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java: ## @@ -18,27 +18,77 @@ package org.apache.hudi.metadata; +import org.apache.hudi.co

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
codope commented on code in PR #11067: URL: https://github.com/apache/hudi/pull/11067#discussion_r1578755311 ## hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java: ## @@ -18,27 +18,77 @@ package org.apache.hudi.metadata; +import org.apache.hudi.co

Re: [PR] [HUDI-6645] Relax the restriction for Spark MDT rollback [hudi]

2024-04-24 Thread via GitHub
xuzifu666 closed pull request #9364: [HUDI-6645] Relax the restriction for Spark MDT rollback URL: https://github.com/apache/hudi/pull/9364 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] [HUDI-7627] ParquetSchema clip case-sensetive need be configurable [hudi]

2024-04-24 Thread via GitHub
xuzifu666 closed pull request #11040: [HUDI-7627] ParquetSchema clip case-sensetive need be configurable URL: https://github.com/apache/hudi/pull/11040 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11067: URL: https://github.com/apache/hudi/pull/11067#issuecomment-2076174773 ## CI report: * a3049b2f019185ffb23ec7f84c7ce602836b1b06 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
codope commented on code in PR #11067: URL: https://github.com/apache/hudi/pull/11067#discussion_r1578752114 ## hudi-common/src/test/java/org/apache/hudi/metadata/TestMetadataPartitionType.java: ## @@ -0,0 +1,122 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[jira] [Updated] (HUDI-7596) Enable Jacoco code coverage report across multiple modules

2024-04-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7596: Status: Patch Available (was: In Progress) > Enable Jacoco code coverage report across multiple modules > -

[jira] [Closed] (HUDI-7651) Add util methods for creating meta client

2024-04-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo closed HUDI-7651. --- Resolution: Fixed > Add util methods for creating meta client > - > >

[jira] [Closed] (HUDI-7650) Remove FileSystem argument in TestHelpers methods

2024-04-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo closed HUDI-7650. --- Resolution: Fixed > Remove FileSystem argument in TestHelpers methods > --

[jira] [Created] (HUDI-7662) Expose a config to enable disable functional index

2024-04-24 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7662: - Summary: Expose a config to enable disable functional index Key: HUDI-7662 URL: https://issues.apache.org/jira/browse/HUDI-7662 Project: Apache Hudi Issue Type: Ta

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11073: URL: https://github.com/apache/hudi/pull/11073#issuecomment-2076143184 ## CI report: * 39c44a33eaae3bc17270cec93536ce727daacd98 UNKNOWN * d329d241fef55c438e72343e4564671df92befff UNKNOWN * 44dcb4360dfd39b9f3b2d651280b3b2cc413d6f6 Azure: [SUCC

Re: [PR] [MINOR] Streamer test setup performance [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10806: URL: https://github.com/apache/hudi/pull/10806#issuecomment-2076142114 ## CI report: * e0414708ebbd734156c0383cb4e5dbfe5ff4151a UNKNOWN * 11c19fa8fd39ed058a4e3487c99c793610b61564 UNKNOWN * 767330a2fd7698adec4b00fe1c4d0ad321b5403f Azure: [SUCC

[jira] [Closed] (HUDI-3939) Website Contributing code to the project (newbie JIRAs) links wrong.

2024-04-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-3939. Resolution: Fixed https://github.com/apache/hudi/pull/11087 > Website Contributing code to the project (new

[jira] [Comment Edited] (HUDI-3939) Website Contributing code to the project (newbie JIRAs) links wrong.

2024-04-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840609#comment-17840609 ] Raymond Xu edited comment on HUDI-3939 at 4/25/24 1:12 AM: --- [htt

[jira] [Created] (HUDI-7661) Update docs to show how a new index implementation can be added

2024-04-24 Thread Sagar Sumit (Jira)
Sagar Sumit created HUDI-7661: - Summary: Update docs to show how a new index implementation can be added Key: HUDI-7661 URL: https://issues.apache.org/jira/browse/HUDI-7661 Project: Apache Hudi

[jira] [Closed] (HUDI-5180) Get Involved on the website has broken links

2024-04-24 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu closed HUDI-5180. Resolution: Fixed > Get Involved on the website has broken links > -

(hudi) branch asf-site updated: [MINOR] Fix link (#11088)

2024-04-24 Thread xushiyan
This is an automated email from the ASF dual-hosted git repository. xushiyan pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 218ea203e83 [MINOR] Fix link (#11088) 218ea2

Re: [PR] [MINOR] Fix link [hudi]

2024-04-24 Thread via GitHub
xushiyan merged PR #11088: URL: https://github.com/apache/hudi/pull/11088 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11073: URL: https://github.com/apache/hudi/pull/11073#issuecomment-2076104015 ## CI report: * 39c44a33eaae3bc17270cec93536ce727daacd98 UNKNOWN * d329d241fef55c438e72343e4564671df92befff UNKNOWN * 44dcb4360dfd39b9f3b2d651280b3b2cc413d6f6 Azure: [SUCC

Re: [PR] [HUDI-7588][DO NOT MERGE][Stacked on others] Replace hadoop Configuration with StorageConfiguration in meta client [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11071: URL: https://github.com/apache/hudi/pull/11071#issuecomment-2076103986 ## CI report: * a9972fa230cade524432f51566a8e6345b39afb9 UNKNOWN * 5fe34eea207b1be8790842c5ec687527c9aa99c7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7588][DO NOT MERGE][Stacked on others] Replace hadoop Configuration with StorageConfiguration in meta client [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11071: URL: https://github.com/apache/hudi/pull/11071#issuecomment-2076096114 ## CI report: * a9972fa230cade524432f51566a8e6345b39afb9 UNKNOWN * 457ef5ece294d2ca965fd9b9ef527a46b3ceced3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [MINOR] Streamer test setup performance [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10806: URL: https://github.com/apache/hudi/pull/10806#issuecomment-2076093163 ## CI report: * e0414708ebbd734156c0383cb4e5dbfe5ff4151a UNKNOWN * 11c19fa8fd39ed058a4e3487c99c793610b61564 UNKNOWN * 0cf069ca9e238b396b3ab9afd2779432190aafb8 Azure: [FAIL

Re: [PR] [MINOR] Optimization function MergeOnReadTableState#getRequiredPositions [hudi]

2024-04-24 Thread via GitHub
danny0405 commented on code in PR #11031: URL: https://github.com/apache/hudi/pull/11031#discussion_r1578662024 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java: ## @@ -86,12 +84,14 @@ public DynamicTableSource createDynamicTableSou

[jira] [Updated] (HUDI-7660) Fix excessive object creation in RowDataKeyGen

2024-04-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7660: - Fix Version/s: 0.15.0 1.0.0 > Fix excessive object creation in RowDataKeyGen >

[jira] [Resolved] (HUDI-7660) Fix excessive object creation in RowDataKeyGen

2024-04-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-7660. -- > Fix excessive object creation in RowDataKeyGen > -- > >

[jira] [Closed] (HUDI-7660) Fix excessive object creation in RowDataKeyGen

2024-04-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7660. Resolution: Fixed Fixed via master branch: f8e559f5e6f75ba5967c96efbcb39dd53365624f > Fix excessive object

(hudi) branch master updated: [HUDI-7660] Fix excessive object creation in RowDataKeyGen (#11084)

2024-04-24 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new f8e559f5e6f [HUDI-7660] Fix excessive object cr

Re: [PR] [HUDI-7660] Fix excessive object creation in RowDataKeyGen [hudi]

2024-04-24 Thread via GitHub
danny0405 merged PR #11084: URL: https://github.com/apache/hudi/pull/11084 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

Re: [PR] [HUDI-7575] avoid repeated fetching of pending replace instants [hudi]

2024-04-24 Thread via GitHub
danny0405 commented on code in PR #10976: URL: https://github.com/apache/hudi/pull/10976#discussion_r1578657154 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieDefaultTimeline.java: ## @@ -540,10 +542,10 @@ private Option getLastOrFirstPendingClusterIns

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
danny0405 commented on code in PR #11067: URL: https://github.com/apache/hudi/pull/11067#discussion_r1578656443 ## hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java: ## @@ -18,27 +18,77 @@ package org.apache.hudi.metadata; +import org.apache.hudi

Re: [PR] [HUDI-7575] avoid repeated fetching of pending replace instants [hudi]

2024-04-24 Thread via GitHub
the-other-tim-brown commented on code in PR #10976: URL: https://github.com/apache/hudi/pull/10976#discussion_r1578655857 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieDefaultTimeline.java: ## @@ -540,10 +542,10 @@ private Option getLastOrFirstPending

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
danny0405 commented on code in PR #11067: URL: https://github.com/apache/hudi/pull/11067#discussion_r1578654891 ## hudi-common/src/test/java/org/apache/hudi/metadata/TestMetadataPartitionType.java: ## @@ -0,0 +1,122 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] [HUDI-7575] avoid repeated fetching of pending replace instants [hudi]

2024-04-24 Thread via GitHub
danny0405 commented on code in PR #10976: URL: https://github.com/apache/hudi/pull/10976#discussion_r1578654464 ## hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieDefaultTimeline.java: ## @@ -540,10 +542,10 @@ private Option getLastOrFirstPendingClusterIns

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
danny0405 commented on code in PR #11073: URL: https://github.com/apache/hudi/pull/11073#discussion_r1578649906 ## pom.xml: ## @@ -2040,6 +2040,27 @@ + +org.jacoco +jacoco-maven-plugin + +

[jira] [Comment Edited] (HUDI-7596) Enable Jacoco code coverage report across multiple modules

2024-04-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838411#comment-17838411 ] Danny Chen edited comment on HUDI-7596 at 4/24/24 11:52 PM: Th

Re: [PR] [HUDI-7588][DO NOT MERGE][Stacked on others] Replace hadoop Configuration with StorageConfiguration in meta client [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11071: URL: https://github.com/apache/hudi/pull/11071#issuecomment-2076054031 ## CI report: * a9972fa230cade524432f51566a8e6345b39afb9 UNKNOWN * 457ef5ece294d2ca965fd9b9ef527a46b3ceced3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7588][DO NOT MERGE][Stacked on others] Replace hadoop Configuration with StorageConfiguration in meta client [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #11071: URL: https://github.com/apache/hudi/pull/11071#issuecomment-2076048103 ## CI report: * a9972fa230cade524432f51566a8e6345b39afb9 UNKNOWN * 5adbdeaf42bb58c40ec587c93da81406bc9222be Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [MINOR] Streamer test setup performance [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10806: URL: https://github.com/apache/hudi/pull/10806#issuecomment-2076047624 ## CI report: * e0414708ebbd734156c0383cb4e5dbfe5ff4151a UNKNOWN * 11c19fa8fd39ed058a4e3487c99c793610b61564 UNKNOWN * 0cf069ca9e238b396b3ab9afd2779432190aafb8 Azure: [FAIL

Re: [PR] [HUDI-7575] avoid repeated fetching of pending replace instants [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10976: URL: https://github.com/apache/hudi/pull/10976#issuecomment-2076027839 ## CI report: * fa5db11f49dbf6e5caa9713d8860d076f86ac8e6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
yihua commented on code in PR #11073: URL: https://github.com/apache/hudi/pull/11073#discussion_r1578615084 ## pom.xml: ## @@ -2040,6 +2040,27 @@ + +org.jacoco +jacoco-maven-plugin + +

Re: [PR] [HUDI-7596] Enable Jacoco code coverage report across multiple modules [hudi]

2024-04-24 Thread via GitHub
yihua commented on code in PR #11073: URL: https://github.com/apache/hudi/pull/11073#discussion_r1578613938 ## pom.xml: ## @@ -2040,6 +2040,27 @@ + +org.jacoco +jacoco-maven-plugin + +

(hudi) branch asf-site updated: [HUDI-5180][HUDI-3939] Fix links and clarify JIRA self-service (#11087)

2024-04-24 Thread bhavanisudha
This is an automated email from the ASF dual-hosted git repository. bhavanisudha pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 89b9767372a [HUDI-5180][HUDI-3939] Fix l

Re: [PR] [HUDI-5180][HUDI-3939] Fix links and clarify JIRA self-service [hudi]

2024-04-24 Thread via GitHub
bhasudha merged PR #11087: URL: https://github.com/apache/hudi/pull/11087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[jira] [Closed] (HUDI-7640) Uses UUID as temporary file suffix for HoodieWrapperFileSystem.createImmutableFileInPath

2024-04-24 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo closed HUDI-7640. --- Resolution: Fixed > Uses UUID as temporary file suffix for > HoodieWrapperFileSystem.createImmutableFileInPat

Re: [PR] [HUDI-7648] Refactor MetadataPartitionType so as to enahance reuse [hudi]

2024-04-24 Thread via GitHub
yihua commented on code in PR #11067: URL: https://github.com/apache/hudi/pull/11067#discussion_r1578602042 ## hudi-common/src/main/java/org/apache/hudi/metadata/MetadataPartitionType.java: ## @@ -18,27 +18,77 @@ package org.apache.hudi.metadata; +import org.apache.hudi.com

Re: [PR] [HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2075956260 ## CI report: * 6df21d87a37588010ba16bf62bcc2d8a1974ddf7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

[jira] [Updated] (HUDI-5180) Get Involved on the website has broken links

2024-04-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5180: - Labels: pull-request-available (was: ) > Get Involved on the website has broken links > -

Re: [PR] [HUDI-7575] avoid repeated fetching of pending replace instants [hudi]

2024-04-24 Thread via GitHub
hudi-bot commented on PR #10976: URL: https://github.com/apache/hudi/pull/10976#issuecomment-2075899432 ## CI report: * 641e4e1885d174370cc7a4e438cc67a486a36b04 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

(hudi) branch master updated: [MINOR] Remove unused util methods in LogReaderUtils (#11086)

2024-04-24 Thread jonvex
This is an automated email from the ASF dual-hosted git repository. jonvex pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 7f0a88ce905 [MINOR] Remove unused util methods in

  1   2   >