Re: [I] [SUPPORT] MOR hudi 0.14, Bloom Filters are not being used on query time [hudi]

2024-02-09 Thread via GitHub
parisni commented on issue #10511: URL: https://github.com/apache/hudi/issues/10511#issuecomment-1936917209 Hi @bk-mz thanks for the interest in parquet bloom filter. We have [an open documentation](https://github.com/apache/hudi/pull/9056/files) about bloom filters which states: > S

Re: [PR] [HUDI-6902] Containerize the Azure CI [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10512: URL: https://github.com/apache/hudi/pull/10512#issuecomment-1936882566 ## CI report: * 0e5a63db2337ae435f17eb956460e22caeea65b3 UNKNOWN * 4d759f3b4d6629e738b9b1afe4157c514d6df182 UNKNOWN * a70247f32679a6441cea131e946acce6fd09523e UNKNOWN *

Re: [PR] [HUDI-6497] Replace FileSystem, Path, and FileStatus usage in `hudi-common` module [hudi]

2024-02-09 Thread via GitHub
vinothchandar commented on code in PR #10591: URL: https://github.com/apache/hudi/pull/10591#discussion_r1484910496 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java: ## @@ -736,14 +745,15 @@ private void initializeFile

Re: [PR] [HUDI-6902] Containerize the Azure CI [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10512: URL: https://github.com/apache/hudi/pull/10512#issuecomment-1936842596 ## CI report: * 0e5a63db2337ae435f17eb956460e22caeea65b3 UNKNOWN * 4d759f3b4d6629e738b9b1afe4157c514d6df182 UNKNOWN * a70247f32679a6441cea131e946acce6fd09523e UNKNOWN *

Re: [PR] [HUDI-6902] Containerize the Azure CI [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10512: URL: https://github.com/apache/hudi/pull/10512#issuecomment-1936840311 ## CI report: * 0e5a63db2337ae435f17eb956460e22caeea65b3 UNKNOWN * 4d759f3b4d6629e738b9b1afe4157c514d6df182 UNKNOWN * a70247f32679a6441cea131e946acce6fd09523e UNKNOWN *

[jira] [Created] (HUDI-7400) Fix debezium test failures

2024-02-09 Thread Lin Liu (Jira)
Lin Liu created HUDI-7400: - Summary: Fix debezium test failures Key: HUDI-7400 URL: https://issues.apache.org/jira/browse/HUDI-7400 Project: Apache Hudi Issue Type: Bug Reporter: Lin Liu

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1936838305 ## CI report: * b3713e198af9dc3f3cffa66094a420bb7748cb02 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-7362] Fix hudi partition base path scheme to s3 [hudi]

2024-02-09 Thread via GitHub
nsivabalan commented on PR #10596: URL: https://github.com/apache/hudi/pull/10596#issuecomment-1936819026 hey @codope : is there any more fix required for rt queries? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] [HUDI-7391] HoodieMetadataMetrics should use Metrics instance for metrics registry [hudi]

2024-02-09 Thread via GitHub
nsivabalan commented on code in PR #10635: URL: https://github.com/apache/hudi/pull/10635#discussion_r1484934049 ## hudi-common/src/main/java/org/apache/hudi/metrics/MetricsReporterFactory.java: ## @@ -66,28 +64,29 @@ public static Option createReporter(HoodieWriteConfig config

Re: [PR] [HUDI-7391] HoodieMetadataMetrics should use Metrics instance for metrics registry [hudi]

2024-02-09 Thread via GitHub
nsivabalan commented on PR #10635: URL: https://github.com/apache/hudi/pull/10635#issuecomment-1936815515 hey @prashantwason : can you review this patch please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] [HUDI-7381] Fix compaction write stats and metrics for create and upsert time [hudi]

2024-02-09 Thread via GitHub
nsivabalan commented on code in PR #10619: URL: https://github.com/apache/hudi/pull/10619#discussion_r1484932647 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/HoodieCompactor.java: ## @@ -239,18 +240,25 @@ public List compact(HoodieCompacti

Re: [PR] [HUDI-6902] Containerize the Azure CI [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10512: URL: https://github.com/apache/hudi/pull/10512#issuecomment-1936797088 ## CI report: * 0e5a63db2337ae435f17eb956460e22caeea65b3 UNKNOWN * 4d759f3b4d6629e738b9b1afe4157c514d6df182 UNKNOWN * a70247f32679a6441cea131e946acce6fd09523e UNKNOWN *

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1936777444 ## CI report: * d29fd4e583a76828e6cbaffe24e8618db868e6ce Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

[jira] [Updated] (HUDI-7399) Add IT tests for hudi-aws sync

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7399: - Labels: pull-request-available (was: ) > Add IT tests for hudi-aws sync > ---

Re: [PR] [HUDI-7399] hudi-aws sync integration testing [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1936774571 ## CI report: * d29fd4e583a76828e6cbaffe24e8618db868e6ce Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [MINOR] Enable code coverage in Azure [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10646: URL: https://github.com/apache/hudi/pull/10646#issuecomment-1936771383 ## CI report: * eb130bb64daadb66dbabbb01afa9a1ff85380eb3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

[jira] [Created] (HUDI-7399) Add IT tests for hudi-aws sync

2024-02-09 Thread nicolas paris (Jira)
nicolas paris created HUDI-7399: --- Summary: Add IT tests for hudi-aws sync Key: HUDI-7399 URL: https://issues.apache.org/jira/browse/HUDI-7399 Project: Apache Hudi Issue Type: Test R

Re: [PR] POC hudi-aws integration testing w/ moto [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1936738188 ## CI report: * d29fd4e583a76828e6cbaffe24e8618db868e6ce Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-6497] Replace FileSystem, Path, and FileStatus usage in `hudi-common` module [hudi]

2024-02-09 Thread via GitHub
vinothchandar commented on code in PR #10591: URL: https://github.com/apache/hudi/pull/10591#discussion_r1484485806 ## hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/HoodieDeltaStreamerWrapper.java: ## @@ -83,7 +83,7 @@ public Pair>> fetchSource() t StreamSyn

Re: [PR] [HUDI-6902] Containerize the Azure CI [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10512: URL: https://github.com/apache/hudi/pull/10512#issuecomment-1936703020 ## CI report: * 0e5a63db2337ae435f17eb956460e22caeea65b3 UNKNOWN * 4d759f3b4d6629e738b9b1afe4157c514d6df182 UNKNOWN * a70247f32679a6441cea131e946acce6fd09523e UNKNOWN *

Re: [PR] POC hudi-aws integration testing w/ moto [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1936703202 ## CI report: * 2afd22c6dde395184b61244c69a49400e74dc8d0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] POC hudi-aws integration testing w/ moto [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10614: URL: https://github.com/apache/hudi/pull/10614#issuecomment-1936696847 ## CI report: * 2afd22c6dde395184b61244c69a49400e74dc8d0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-6902] Containerize the Azure CI [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10512: URL: https://github.com/apache/hudi/pull/10512#issuecomment-1936696618 ## CI report: * 0e5a63db2337ae435f17eb956460e22caeea65b3 UNKNOWN * 4d759f3b4d6629e738b9b1afe4157c514d6df182 UNKNOWN * a70247f32679a6441cea131e946acce6fd09523e UNKNOWN *

Re: [PR] [MINOR] Docker push [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10637: URL: https://github.com/apache/hudi/pull/10637#issuecomment-1936690735 ## CI report: * 5f2b3d3db40b81d64d113179c4ee681d3486aa34 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [MINOR] Docker push [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10637: URL: https://github.com/apache/hudi/pull/10637#issuecomment-1936649420 ## CI report: * 2474180f419af653e607ceb3f5d7362d8ffe75db Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [MINOR] Enable code coverage in Azure [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10646: URL: https://github.com/apache/hudi/pull/10646#issuecomment-1936641792 ## CI report: * eb130bb64daadb66dbabbb01afa9a1ff85380eb3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [MINOR] Docker push [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10637: URL: https://github.com/apache/hudi/pull/10637#issuecomment-1936641727 ## CI report: * dfb8e8df1e7bc3723206d6601df68aff9e440a35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [MINOR] Enable code coverage in Azure [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10646: URL: https://github.com/apache/hudi/pull/10646#issuecomment-1936633867 ## CI report: * eb130bb64daadb66dbabbb01afa9a1ff85380eb3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [MINOR] Docker push [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10637: URL: https://github.com/apache/hudi/pull/10637#issuecomment-1936633800 ## CI report: * dfb8e8df1e7bc3723206d6601df68aff9e440a35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-6902] Containerize the Azure CI [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10512: URL: https://github.com/apache/hudi/pull/10512#issuecomment-1936625574 ## CI report: * 0e5a63db2337ae435f17eb956460e22caeea65b3 UNKNOWN * 4d759f3b4d6629e738b9b1afe4157c514d6df182 UNKNOWN * a70247f32679a6441cea131e946acce6fd09523e UNKNOWN *

Re: [PR] [MINOR] Docker push [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10637: URL: https://github.com/apache/hudi/pull/10637#issuecomment-1936625884 ## CI report: * dfb8e8df1e7bc3723206d6601df68aff9e440a35 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

[PR] [MINOR] Enable code coverage [hudi]

2024-02-09 Thread via GitHub
linliu-code opened a new pull request, #10646: URL: https://github.com/apache/hudi/pull/10646 ### Change Logs Enable code covery. ### Impact More tool to improve code quality. ### Risk level (write none, low medium or high below) None. ### Contributor

Re: [PR] [MINOR] Docker push [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10637: URL: https://github.com/apache/hudi/pull/10637#issuecomment-1936575834 ## CI report: * 5db9059f261a00681db0b0c68ea54b2a1bdd71b9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [MINOR] Docker push [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10637: URL: https://github.com/apache/hudi/pull/10637#issuecomment-1936567744 ## CI report: * 5db9059f261a00681db0b0c68ea54b2a1bdd71b9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [MINOR] Docker push [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10637: URL: https://github.com/apache/hudi/pull/10637#issuecomment-1936504094 ## CI report: * 5db9059f261a00681db0b0c68ea54b2a1bdd71b9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-6902] Containerize the Azure CI [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10512: URL: https://github.com/apache/hudi/pull/10512#issuecomment-1936503648 ## CI report: * 0e5a63db2337ae435f17eb956460e22caeea65b3 UNKNOWN * 4d759f3b4d6629e738b9b1afe4157c514d6df182 UNKNOWN * a70247f32679a6441cea131e946acce6fd09523e UNKNOWN *

Re: [PR] [HUDI-6902] Containerize the Azure CI [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10512: URL: https://github.com/apache/hudi/pull/10512#issuecomment-1936494186 ## CI report: * 0e5a63db2337ae435f17eb956460e22caeea65b3 UNKNOWN * 4d759f3b4d6629e738b9b1afe4157c514d6df182 UNKNOWN * a70247f32679a6441cea131e946acce6fd09523e UNKNOWN *

Re: [PR] [HUDI-7397] Adding support to purge pending clustering instant [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10645: URL: https://github.com/apache/hudi/pull/10645#issuecomment-1936494683 ## CI report: * 6b46abe0e1f4ca6e7979aba464a16ec5564f4448 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [MINOR] Docker push [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10637: URL: https://github.com/apache/hudi/pull/10637#issuecomment-1936494632 ## CI report: * 9ca9a1f0f76a2f8592cdcbbed8fdbbaea43ae437 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-7397] Adding support to purge pending clustering instant [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10645: URL: https://github.com/apache/hudi/pull/10645#issuecomment-1936324414 ## CI report: * 6b46abe0e1f4ca6e7979aba464a16ec5564f4448 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22

Re: [PR] [HUDI-7397] Adding support to purge pending clustering instant [hudi]

2024-02-09 Thread via GitHub
hudi-bot commented on PR #10645: URL: https://github.com/apache/hudi/pull/10645#issuecomment-1936315180 ## CI report: * 6b46abe0e1f4ca6e7979aba464a16ec5564f4448 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[jira] [Created] (HUDI-7398) clarify clustering strategy for java client

2024-02-09 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-7398: Summary: clarify clustering strategy for java client Key: HUDI-7398 URL: https://issues.apache.org/jira/browse/HUDI-7398 Project: Apache Hudi Issue Type: Improvement

[jira] [Updated] (HUDI-7397) Add support to purge a clustering instant

2024-02-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7397: - Labels: pull-request-available (was: ) > Add support to purge a clustering instant >

[PR] [HUDI-7397] Adding support to purge pending clustering instant [hudi]

2024-02-09 Thread via GitHub
nsivabalan opened a new pull request, #10645: URL: https://github.com/apache/hudi/pull/10645 ### Change Logs - Adding support to purge pending clustering instant. ### Impact If a user made some mistake on clustering params and wishes to completely purge a pending cluste

[jira] [Created] (HUDI-7397) Add support to purge a clustering instant

2024-02-09 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-7397: - Summary: Add support to purge a clustering instant Key: HUDI-7397 URL: https://issues.apache.org/jira/browse/HUDI-7397 Project: Apache Hudi Issue T

Re: [PR] [MINOR] fix typo [hudi]

2024-02-09 Thread via GitHub
lxliyou001 commented on PR #10634: URL: https://github.com/apache/hudi/pull/10634#issuecomment-1935973639 Hi @bvaradar ,thank you for reviewing my PR, it is my first time participating in the Hudi project, I will contribute more efforts to this project in the future. -- This is an automa

[I] [FeatureRequest] Inquiry Regarding Hudi Exporter with SQL Transformer for Data Filtering [hudi]

2024-02-09 Thread via GitHub
soumilshah1995 opened a new issue, #10644: URL: https://github.com/apache/hudi/issues/10644 Hello, I'm reaching out to inquire about the Hudi exporter service. I've had some experience working with it, but I'm particularly interested in whether we support the integration of SQL trans