Re: [PR] [HUDI-7402] Align MDT cleaner configs with the data table [hudi]

hudi-bot commented on PR #10655: URL: https://github.com/apache/hudi/pull/10655#issuecomment-1938188026 ## CI report: * 0e0c687bca218557ea379ef01c78db8c8f07b38e Azure:

Re: [PR] [HUDI-7391] HoodieMetadataMetrics should use Metrics instance for metrics registry [hudi]

hudi-bot commented on PR #10635: URL: https://github.com/apache/hudi/pull/10635#issuecomment-1938187888 ## CI report: * adc183a351b8f15d671c0c6eefd1f999bed54774 Azure:

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

lokeshj1703 commented on code in PR #10641: URL: https://github.com/apache/hudi/pull/10641#discussion_r1485825887 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java: ## @@ -147,6 +148,10 @@ public static HoodieWriteConfig

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

lokeshj1703 commented on code in PR #10641: URL: https://github.com/apache/hudi/pull/10641#discussion_r1485824879 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -324,7 +324,7 @@ private boolean

Re: [PR] [HUDI-7402] Align MDT cleaner configs with the data table [hudi]

hudi-bot commented on PR #10655: URL: https://github.com/apache/hudi/pull/10655#issuecomment-1938180834 ## CI report: * 0e0c687bca218557ea379ef01c78db8c8f07b38e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-7391] HoodieMetadataMetrics should use Metrics instance for metrics registry [hudi]

hudi-bot commented on PR #10635: URL: https://github.com/apache/hudi/pull/10635#issuecomment-1938180722 ## CI report: * adc183a351b8f15d671c0c6eefd1f999bed54774 Azure:

Re: [PR] [MINOR] Fix tests crashing surefire JVM [hudi]

hudi-bot commented on PR #10654: URL: https://github.com/apache/hudi/pull/10654#issuecomment-1938174074 ## CI report: * 0884ca6be51707611508549df6fdd4f094ae8f5b UNKNOWN * e69bc9cb1027a2e9dfb096de7434a040c2c3d882 Azure:

Re: [PR] [HUDI-7391] HoodieMetadataMetrics should use Metrics instance for metrics registry [hudi]

lokeshj1703 commented on code in PR #10635: URL: https://github.com/apache/hudi/pull/10635#discussion_r1485819383 ## hudi-common/src/main/java/org/apache/hudi/metrics/MetricsReporterFactory.java: ## @@ -66,28 +64,29 @@ public static Option createReporter(HoodieWriteConfig

[jira] [Updated] (HUDI-7402) Align MDT cleaner configs with the data table

[ https://issues.apache.org/jira/browse/HUDI-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7402: - Labels: pull-request-available (was: ) > Align MDT cleaner configs with the data table >

[PR] [HUDI-7402] Align MDT cleaner configs with the data table [hudi]

codope opened a new pull request, #10655: URL: https://github.com/apache/hudi/pull/10655 ### Change Logs Metadata table should retain at least as much history as data table. Follow similar policy as data table and set retention to 1.2x for metadata table. ### Impact

Re: [PR] [HUDI-7391] HoodieMetadataMetrics should use Metrics instance for metrics registry [hudi]

lokeshj1703 commented on code in PR #10635: URL: https://github.com/apache/hudi/pull/10635#discussion_r1485805111 ## hudi-common/src/main/java/org/apache/hudi/metrics/MetricsReporterFactory.java: ## @@ -66,28 +64,29 @@ public static Option createReporter(HoodieWriteConfig

Re: [PR] [MINOR] Fix tests crashing surefire JVM [hudi]

hudi-bot commented on PR #10654: URL: https://github.com/apache/hudi/pull/10654#issuecomment-1938132988 ## CI report: * 0884ca6be51707611508549df6fdd4f094ae8f5b UNKNOWN * e69bc9cb1027a2e9dfb096de7434a040c2c3d882 Azure:

Re: [PR] [MINOR] Fix tests crashing surefire JVM [hudi]

hudi-bot commented on PR #10654: URL: https://github.com/apache/hudi/pull/10654#issuecomment-1938128479 ## CI report: * 0884ca6be51707611508549df6fdd4f094ae8f5b UNKNOWN * e69bc9cb1027a2e9dfb096de7434a040c2c3d882 UNKNOWN Bot commands @hudi-bot supports the

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

nsivabalan commented on code in PR #10641: URL: https://github.com/apache/hudi/pull/10641#discussion_r1485785454 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java: ## @@ -147,6 +148,10 @@ public static HoodieWriteConfig

[jira] [Created] (HUDI-7402) Align MDT cleaner configs with the data table

Sagar Sumit created HUDI-7402: - Summary: Align MDT cleaner configs with the data table Key: HUDI-7402 URL: https://issues.apache.org/jira/browse/HUDI-7402 Project: Apache Hudi Issue Type: Task

Re: [PR] [MINOR] Fix tests crashing surefire JVM [hudi]

hudi-bot commented on PR #10654: URL: https://github.com/apache/hudi/pull/10654#issuecomment-1938097699 ## CI report: * 0884ca6be51707611508549df6fdd4f094ae8f5b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [MINOR] Fix tests crashing surefire JVM [hudi]

hudi-bot commented on PR #10654: URL: https://github.com/apache/hudi/pull/10654#issuecomment-1938093450 ## CI report: * 0884ca6be51707611508549df6fdd4f094ae8f5b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [MINOR] Fix tests crashing surefire JVM [hudi]

linliu-code commented on PR #10654: URL: https://github.com/apache/hudi/pull/10654#issuecomment-1938089758 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] [MINOR] Fix tests for DFSSource [hudi]

linliu-code opened a new pull request, #10654: URL: https://github.com/apache/hudi/pull/10654 ### Change Logs We saw the JVM of surefire crashed for many tests in hudi-utilities. Since there are several different tests crashed the JVM, the cause of the crashing should be general.

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

bvaradar commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1485731251 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java: ## @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [HUDI-7104] Fixing cleaner savepoint interplay to fix edge case with incremental cleaning [hudi]

hudi-bot commented on PR #10651: URL: https://github.com/apache/hudi/pull/10651#issuecomment-1938026546 ## CI report: * a62585ad202a405af8fc8cf22833e622fe8a262c Azure:

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937970560 ## CI report: * 9add9dd6030aba044c2996be099be8b336720b00 Azure:

Re: [PR] [HUDI-7395] Fix computation for metrics in HoodieMetadataMetrics [hudi]

danny0405 commented on code in PR #10641: URL: https://github.com/apache/hudi/pull/10641#discussion_r1485703297 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -324,7 +324,7 @@ private boolean

Re: [PR] [HUDI-7104] Fixing cleaner savepoint interplay to fix edge case with incremental cleaning [hudi]

hudi-bot commented on PR #10651: URL: https://github.com/apache/hudi/pull/10651#issuecomment-1937944346 ## CI report: * 265cef9b5d5d96a561dec33111c37b87a5710a0f Azure:

[jira] [Updated] (HUDI-7104) Cleaner could miss to clean up some files w/ savepoint interplay

[ https://issues.apache.org/jira/browse/HUDI-7104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7104: - Labels: pull-request-available (was: ) > Cleaner could miss to clean up some files w/ savepoint

Re: [PR] [HUDI-7104] Fixing cleaner savepoint interplay to fix edge case with incremental cleaning [hudi]

hudi-bot commented on PR #10651: URL: https://github.com/apache/hudi/pull/10651#issuecomment-1937940854 ## CI report: * 265cef9b5d5d96a561dec33111c37b87a5710a0f Azure:

[jira] [Closed] (HUDI-7379) hudi-aws-bundle included jackson-module-afterburner without relocating jackson-databind

[ https://issues.apache.org/jira/browse/HUDI-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7379. Resolution: Fixed Fixed via master branch: 1b12705a50248520c856e3d8f040166a78d3fda8 > hudi-aws-bundle

[jira] [Updated] (HUDI-7379) hudi-aws-bundle included jackson-module-afterburner without relocating jackson-databind

[ https://issues.apache.org/jira/browse/HUDI-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7379: - Fix Version/s: 1.0.0 0.14.2 > hudi-aws-bundle included jackson-module-afterburner

(hudi) branch master updated (95a0df398c7 -> 1b12705a502)

This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 95a0df398c7 [HUDI-7397] Adding support to purge pending clustering instant (#10645) add 1b12705a502

Re: [PR] [HUDI-7379] Exclude jackson-module-afterburner from hudi-aws module [hudi]

danny0405 merged PR #10618: URL: https://github.com/apache/hudi/pull/10618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] [SUPPORT] Setting hoodie.datasource.insert.dup.policy to drop still upserts the record in 0.14 [hudi]

keerthiskating commented on issue #10650: URL: https://github.com/apache/hudi/issues/10650#issuecomment-1937919633 > @keerthiskating This setting is only applicable when operation type is insert. > > > >

Re: [I] [SUPPORT] Setting hoodie.datasource.insert.dup.policy to drop still upserts the record in 0.14 [hudi]

keerthiskating commented on issue #10650: URL: https://github.com/apache/hudi/issues/10650#issuecomment-1937919552 Any idea how do I achieve this when doing upsert operation? I want hudi to ignore records that already exist in hudi table and not update those record's commit time. --

Re: [PR] lazy init secondary view in FS view [hudi]

hudi-bot commented on PR #10652: URL: https://github.com/apache/hudi/pull/10652#issuecomment-1937913087 ## CI report: * 51ff1e0f5fe7c211c13961fb2a7709a456519fb3 Azure:

Re: [PR] [HUDI-7381] Fix compaction write stats and metrics for create and upsert time [hudi]

hudi-bot commented on PR #10619: URL: https://github.com/apache/hudi/pull/10619#issuecomment-1937913033 ## CI report: * c89b06f099656f289eba408248bd9e2dbd702c61 Azure:

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937901327 ## CI report: * f1d46fb9831136f1899698f8366bba87ab2c166b Azure:

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937899226 ## CI report: * 43d691ee54a9a9e5dc9b60cd297f6385d8e89bd4 Azure:

Re: [PR] try test payload compression [hudi]

the-other-tim-brown closed pull request #10653: try test payload compression URL: https://github.com/apache/hudi/pull/10653 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] try test payload compression [hudi]

hudi-bot commented on PR #10653: URL: https://github.com/apache/hudi/pull/10653#issuecomment-1937883083 ## CI report: * 98b41a03effd534052dd0f001353d28280f8b196 Azure:

Re: [PR] lazy init secondary view in FS view [hudi]

hudi-bot commented on PR #10652: URL: https://github.com/apache/hudi/pull/10652#issuecomment-1937883069 ## CI report: * 0822180197de325aca9d835d9d4b5cb13b431a8b Azure:

Re: [PR] try test payload compression [hudi]

hudi-bot commented on PR #10653: URL: https://github.com/apache/hudi/pull/10653#issuecomment-1937880666 ## CI report: * 98b41a03effd534052dd0f001353d28280f8b196 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] lazy init secondary view in FS view [hudi]

hudi-bot commented on PR #10652: URL: https://github.com/apache/hudi/pull/10652#issuecomment-1937880651 ## CI report: * 0822180197de325aca9d835d9d4b5cb13b431a8b Azure:

Re: [PR] [HUDI-7381] Fix compaction write stats and metrics for create and upsert time [hudi]

hudi-bot commented on PR #10619: URL: https://github.com/apache/hudi/pull/10619#issuecomment-1937880591 ## CI report: * c89b06f099656f289eba408248bd9e2dbd702c61 Azure:

Re: [PR] lazy init secondary view in FS view [hudi]

hudi-bot commented on PR #10652: URL: https://github.com/apache/hudi/pull/10652#issuecomment-1937878388 ## CI report: * 0822180197de325aca9d835d9d4b5cb13b431a8b Azure:

Re: [PR] [HUDI-7381] Fix compaction write stats and metrics for create and upsert time [hudi]

hudi-bot commented on PR #10619: URL: https://github.com/apache/hudi/pull/10619#issuecomment-1937878326 ## CI report: * c89b06f099656f289eba408248bd9e2dbd702c61 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [DNM] Draft for savepoint cleaner fix [hudi]

hudi-bot commented on PR #10651: URL: https://github.com/apache/hudi/pull/10651#issuecomment-1937878376 ## CI report: * 265cef9b5d5d96a561dec33111c37b87a5710a0f Azure:

[PR] try test payload compression [hudi]

the-other-tim-brown opened a new pull request, #10653: URL: https://github.com/apache/hudi/pull/10653 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or

Re: [PR] [DNM] Draft for savepoint cleaner fix [hudi]

hudi-bot commented on PR #10651: URL: https://github.com/apache/hudi/pull/10651#issuecomment-1937866016 ## CI report: * 265cef9b5d5d96a561dec33111c37b87a5710a0f Azure:

Re: [PR] lazy init secondary view in FS view [hudi]

hudi-bot commented on PR #10652: URL: https://github.com/apache/hudi/pull/10652#issuecomment-1937866028 ## CI report: * 0822180197de325aca9d835d9d4b5cb13b431a8b UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937865981 ## CI report: * 43d691ee54a9a9e5dc9b60cd297f6385d8e89bd4 Azure:

[PR] lazy init secondary view in FS view [hudi]

the-other-tim-brown opened a new pull request, #10652: URL: https://github.com/apache/hudi/pull/10652 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or

Re: [PR] [DNM] Draft for savepoint cleaner fix [hudi]

hudi-bot commented on PR #10651: URL: https://github.com/apache/hudi/pull/10651#issuecomment-1937864512 ## CI report: * 265cef9b5d5d96a561dec33111c37b87a5710a0f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937864462 ## CI report: * 43d691ee54a9a9e5dc9b60cd297f6385d8e89bd4 Azure:

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937862640 ## CI report: * 43d691ee54a9a9e5dc9b60cd297f6385d8e89bd4 Azure:

[PR] [DNM] Draft for savepoint cleaner fix [hudi]

nsivabalan opened a new pull request, #10651: URL: https://github.com/apache/hudi/pull/10651 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937848198 ## CI report: * fb0c3abcb071cc96a5686c076f6d385ed60bf398 Azure:

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937837059 ## CI report: * d7ec2e657446125700d28ccb9ac8e4ff0e420b9a Azure:

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937835360 ## CI report: * d7ec2e657446125700d28ccb9ac8e4ff0e420b9a Azure:

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937833581 ## CI report: * d7ec2e657446125700d28ccb9ac8e4ff0e420b9a Azure:

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937818772 ## CI report: * b3713e198af9dc3f3cffa66094a420bb7748cb02 Azure:

Re: [I] [SUPPORT] Setting hoodie.datasource.insert.dup.policy to drop still upserts the record in 0.14 [hudi]

ad1happy2go commented on issue #10650: URL: https://github.com/apache/hudi/issues/10650#issuecomment-1937803794 @keerthiskating This setting is only applicable when operation type is insert. https://github.com/apache/hudi/assets/63430370/9cbb376e-356a-4d43-9aa8-956d9794d371;>

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937803734 ## CI report: * b3713e198af9dc3f3cffa66094a420bb7748cb02 Azure:

[I] [SUPPORT] Setting hoodie.datasource.insert.dup.policy to drop still upserts the record in 0.14 [hudi]

keerthiskating opened a new issue, #10650: URL: https://github.com/apache/hudi/issues/10650 **Describe the problem you faced** If my incoming dataset already has a record which already exists in the hudi table, hudi is still updating the commit time and treating it as update even

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937792236 ## CI report: * b3713e198af9dc3f3cffa66094a420bb7748cb02 Azure:

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1937790311 ## CI report: * b3713e198af9dc3f3cffa66094a420bb7748cb02 Azure:

Re: [PR] [DOCS] Update about recent Athena upgrade query failures [hudi]

codope commented on PR #10649: URL: https://github.com/apache/hudi/pull/10649#issuecomment-1937783325 https://github.com/apache/hudi/assets/16440354/a5c9e784-d962-4e5c-a8aa-b9950697c431;> -- This is an automated message from the Apache Git Service. To respond to the message, please

[PR] [DOCS] Update about recent Athena upgrade query failures [hudi]

codope opened a new pull request, #10649: URL: https://github.com/apache/hudi/pull/10649 ### Change Logs Wtih recent Athena queries, Hudi MOR tables are not queryable. Update in the docs until fixed. ### Impact Docs update. ### Risk level (write none, low medium

Re: [PR] [HUDI-7362] Fix hudi partition base path scheme to s3 [hudi]

codope commented on PR #10596: URL: https://github.com/apache/hudi/pull/10596#issuecomment-1937646490 _rt queries are failing with cast exception only for timestamp fields. That will be fixed separately. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [MINOR] Enable code coverage in Azure [hudi]

hudi-bot commented on PR #10646: URL: https://github.com/apache/hudi/pull/10646#issuecomment-1937476581 ## CI report: * 85e8b3b7f90e304e289cea1136ea6baf9b81933e Azure:

Re: [PR] [HUDI-7381] Fix compaction write stats and metrics for create and upsert time [hudi]

hudi-bot commented on PR #10619: URL: https://github.com/apache/hudi/pull/10619#issuecomment-1937474509 ## CI report: * c89b06f099656f289eba408248bd9e2dbd702c61 Azure: