[GitHub] [hudi] xiarixiaoyao commented on pull request #7173: [HUDI-5189] Make HiveAvroSerializer compatible with hive3

2022-12-01 Thread GitBox
xiarixiaoyao commented on PR #7173: URL: https://github.com/apache/hudi/pull/7173#issuecomment-1334873151 @cdmikechen could you pls help review this pr, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [hudi] hudi-bot commented on pull request #7362: [HUDI-5315] The record size is dynamically estimated when the table i…

2022-12-01 Thread GitBox
hudi-bot commented on PR #7362: URL: https://github.com/apache/hudi/pull/7362#issuecomment-1334858886 ## CI report: * 645ff88bce4491c51ebfbfa8a460918890fa0697 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1341

[GitHub] [hudi] hudi-bot commented on pull request #7345: [HUDI-3378] RFC46 rebase

2022-12-01 Thread GitBox
hudi-bot commented on PR #7345: URL: https://github.com/apache/hudi/pull/7345#issuecomment-1334858843 ## CI report: * e5f1dba84479f08417f25f53a79f6dae4425ba23 UNKNOWN * 5bbbd489b2c50a91bee2a5193e4d56c4048c897d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-

[GitHub] [hudi] hudi-bot commented on pull request #7339: [HUDI-5301] Spark SQL queries support setting parameters through set

2022-12-01 Thread GitBox
hudi-bot commented on PR #7339: URL: https://github.com/apache/hudi/pull/7339#issuecomment-1334858772 ## CI report: * d63ed0612b7639a3384965ef6f333b1e30c43064 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=134

[jira] [Created] (HUDI-5316) Add simple schema evolution example to our website

2022-12-01 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5316: - Summary: Add simple schema evolution example to our website Key: HUDI-5316 URL: https://issues.apache.org/jira/browse/HUDI-5316 Project: Apache Hudi

[jira] [Updated] (HUDI-5316) Add simple schema evolution example to our website

2022-12-01 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5316: -- Labels: docs (was: ) > Add simple schema evolution example to our website > --

[jira] [Updated] (HUDI-5316) Add simple schema evolution example to our website

2022-12-01 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5316: -- Priority: Critical (was: Major) > Add simple schema evolution example to our website >

[jira] [Updated] (HUDI-5316) Add simple schema evolution example to our website

2022-12-01 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5316: -- Fix Version/s: 0.13.0 > Add simple schema evolution example to our website > --

[GitHub] [hudi] hudi-bot commented on pull request #7320: [HUDI-5290] remove the lock in #writeTableMetadata

2022-12-01 Thread GitBox
hudi-bot commented on PR #7320: URL: https://github.com/apache/hudi/pull/7320#issuecomment-1334858673 ## CI report: * 56aa2ec4d7955f7db505b908a14cedbc285e692c UNKNOWN * 1f05c6d86c573e7609a073c08dd21d3497566807 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[jira] [Updated] (HUDI-5315) The record size is dynamically estimated when the table is first written

2022-12-01 Thread weiming (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiming updated HUDI-5315: -- Description: Although hudi has the function of dynamically estimating the size of the record, but it can only t

[GitHub] [hudi] hudi-bot commented on pull request #7362: [HUDI-5315] The record size is dynamically estimated when the table i…

2022-12-01 Thread GitBox
hudi-bot commented on PR #7362: URL: https://github.com/apache/hudi/pull/7362#issuecomment-1334854661 ## CI report: * 645ff88bce4491c51ebfbfa8a460918890fa0697 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7361: [HUDI-5314] add call help procedure

2022-12-01 Thread GitBox
hudi-bot commented on PR #7361: URL: https://github.com/apache/hudi/pull/7361#issuecomment-1334854624 ## CI report: * fc6b22cb5fcf726f1fc5153c7e5381e941f7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1340

[GitHub] [hudi] hudi-bot commented on pull request #7345: [HUDI-3378] RFC46 rebase

2022-12-01 Thread GitBox
hudi-bot commented on PR #7345: URL: https://github.com/apache/hudi/pull/7345#issuecomment-1334854577 ## CI report: * e5f1dba84479f08417f25f53a79f6dae4425ba23 UNKNOWN * 75cc2b5607bc7ef7438631aa455749ed8c420bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7339: [HUDI-5301] Spark SQL queries support setting parameters through set

2022-12-01 Thread GitBox
hudi-bot commented on PR #7339: URL: https://github.com/apache/hudi/pull/7339#issuecomment-1334854530 ## CI report: * adef447a18527e5628c8ba53dfb245bafc5cae8e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1338

[GitHub] [hudi] hudi-bot commented on pull request #7320: [HUDI-5290] remove the lock in #writeTableMetadata

2022-12-01 Thread GitBox
hudi-bot commented on PR #7320: URL: https://github.com/apache/hudi/pull/7320#issuecomment-1334854438 ## CI report: * 56aa2ec4d7955f7db505b908a14cedbc285e692c UNKNOWN * 1f05c6d86c573e7609a073c08dd21d3497566807 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7361: [HUDI-5314] add call help procedure

2022-12-01 Thread GitBox
hudi-bot commented on PR #7361: URL: https://github.com/apache/hudi/pull/7361#issuecomment-1334850599 ## CI report: * fc6b22cb5fcf726f1fc5153c7e5381e941f7 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7342: [HUDI-5302] Fix: compute hash key from recordKey failed when recordKeyValue contains ','

2022-12-01 Thread GitBox
hudi-bot commented on PR #7342: URL: https://github.com/apache/hudi/pull/7342#issuecomment-1334850510 ## CI report: * 142b4e2f9fb1f12a4c7d9fa9adf56c3f7b93cd1e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1339

[GitHub] [hudi] hudi-bot commented on pull request #7238: [HUDI-5023] Cleaning up `QueueBasedExecutor` impls

2022-12-01 Thread GitBox
hudi-bot commented on PR #7238: URL: https://github.com/apache/hudi/pull/7238#issuecomment-1334850264 ## CI report: * ba0309167969884856506a0c9b903651ce935e52 UNKNOWN * a76c39fde821f8a7053a3bbc5215bd7bc3aa3af4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7326: [HUDI-5294] Support type change for schema on read + reconcile schema

2022-12-01 Thread GitBox
hudi-bot commented on PR #7326: URL: https://github.com/apache/hudi/pull/7326#issuecomment-1334850428 ## CI report: * 8ace44e022d08adaf99ff8b243417728079483ae Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1340

[GitHub] [hudi] codope commented on a diff in pull request #7238: [HUDI-5023] Cleaning up `QueueBasedExecutor` impls

2022-12-01 Thread GitBox
codope commented on code in PR #7238: URL: https://github.com/apache/hudi/pull/7238#discussion_r1037838713 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/RealtimeUnmergedRecordReader.java: ## @@ -149,6 +153,7 @@ public void close() throws IOException { @Ove

[GitHub] [hudi] scxwhite commented on pull request #7361: [HUDI-5314] add call help procedure

2022-12-01 Thread GitBox
scxwhite commented on PR #7361: URL: https://github.com/apache/hudi/pull/7361#issuecomment-1334848012 At present, all call commands do not rewrite the described methods. The default is the fully qualified name of the class, which needs to be fixed here. -- This is an automated message fro

[jira] [Updated] (HUDI-5315) The record size is dynamically estimated when the table is first written

2022-12-01 Thread weiming (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weiming updated HUDI-5315: -- Attachment: image2022-11-24_15-42-49.png image2022-11-24_15-44-16.png > The record size is dynam

[jira] [Updated] (HUDI-5315) The record size is dynamically estimated when the table is first written

2022-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5315: - Labels: pull-request-available (was: ) > The record size is dynamically estimated when the table

[GitHub] [hudi] weimingdiit opened a new pull request, #7362: [HUDI-5315] The record size is dynamically estimated when the table i…

2022-12-01 Thread GitBox
weimingdiit opened a new pull request, #7362: URL: https://github.com/apache/hudi/pull/7362 …s first written ### Change Logs Although hudi has the function of dynamically estimating the size of the record, but it can only take effect if certain conditions are met, when the use

[GitHub] [hudi] microbearz commented on pull request #7342: [HUDI-5302] Fix: compute hash key from recordKey failed when recordKeyValue contains ','

2022-12-01 Thread GitBox
microbearz commented on PR #7342: URL: https://github.com/apache/hudi/pull/7342#issuecomment-1334838898 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] just-JL commented on pull request #7320: [HUDI-5290] remove the lock in #writeTableMetadata

2022-12-01 Thread GitBox
just-JL commented on PR #7320: URL: https://github.com/apache/hudi/pull/7320#issuecomment-1334819954 > @just-JL Hi, can you apply the patch and rebase with the latest master then force-push again ? @danny0405 Sorry for the late apply.Now it's done. -- This is an automated message f

[jira] [Created] (HUDI-5315) The record size is dynamically estimated when the table is first written

2022-12-01 Thread weiming (Jira)
weiming created HUDI-5315: - Summary: The record size is dynamically estimated when the table is first written Key: HUDI-5315 URL: https://issues.apache.org/jira/browse/HUDI-5315 Project: Apache Hudi

[jira] [Updated] (HUDI-5314) add call help procedure

2022-12-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5314: - Labels: pull-request-available (was: ) > add call help procedure > --- > >

[GitHub] [hudi] scxwhite opened a new pull request, #7361: [HUDI-5314] add call help procedure

2022-12-01 Thread GitBox
scxwhite opened a new pull request, #7361: URL: https://github.com/apache/hudi/pull/7361 ### Change Logs add call help procedure. https://user-images.githubusercontent.com/23207189/205226568-a6a8dc74-a546-4835-b43b-edb5006598ca.png";> https://user-images.githubusercontent.co

[GitHub] [hudi] hudi-bot commented on pull request #7345: [HUDI-3378] RFC46 rebase

2022-12-01 Thread GitBox
hudi-bot commented on PR #7345: URL: https://github.com/apache/hudi/pull/7345#issuecomment-1334805198 ## CI report: * e5f1dba84479f08417f25f53a79f6dae4425ba23 UNKNOWN * 75cc2b5607bc7ef7438631aa455749ed8c420bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] guanziyue commented on a diff in pull request #6878: [HUDI-3397] Guard repeated rdd triggers

2022-12-01 Thread GitBox
guanziyue commented on code in PR #6878: URL: https://github.com/apache/hudi/pull/6878#discussion_r1037815520 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BaseSparkCommitActionExecutor.java: ## @@ -176,7 +177,10 @@ public HoodieWriteMetadata

[jira] [Created] (HUDI-5314) add call help procedure

2022-12-01 Thread scx (Jira)
scx created HUDI-5314: - Summary: add call help procedure Key: HUDI-5314 URL: https://issues.apache.org/jira/browse/HUDI-5314 Project: Apache Hudi Issue Type: New Feature Reporter: scx -

[GitHub] [hudi] hudi-bot commented on pull request #7345: [HUDI-3378] RFC46 rebase

2022-12-01 Thread GitBox
hudi-bot commented on PR #7345: URL: https://github.com/apache/hudi/pull/7345#issuecomment-1334800721 ## CI report: * e5f1dba84479f08417f25f53a79f6dae4425ba23 UNKNOWN * 75cc2b5607bc7ef7438631aa455749ed8c420bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7355: [HUDI-5308] The hive query returns null when the where condition has a partition field

2022-12-01 Thread GitBox
hudi-bot commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1334797041 ## CI report: * d6131bc518f6828af760eb2d434f042bc6a0522e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1337

[GitHub] [hudi] xicm commented on pull request #7355: [HUDI-5308] The hive query returns null when the where condition has a partition field

2022-12-01 Thread GitBox
xicm commented on PR #7355: URL: https://github.com/apache/hudi/pull/7355#issuecomment-1334772548 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [hudi] hudi-bot commented on pull request #7342: [HUDI-5302] Fix: compute hash key from recordKey failed when recordKeyValue contains ','

2022-12-01 Thread GitBox
hudi-bot commented on PR #7342: URL: https://github.com/apache/hudi/pull/7342#issuecomment-1334762507 ## CI report: * 142b4e2f9fb1f12a4c7d9fa9adf56c3f7b93cd1e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1339

[GitHub] [hudi] hudi-bot commented on pull request #7339: [HUDI-5301] Spark SQL queries support setting parameters through set

2022-12-01 Thread GitBox
hudi-bot commented on PR #7339: URL: https://github.com/apache/hudi/pull/7339#issuecomment-1334756804 ## CI report: * adef447a18527e5628c8ba53dfb245bafc5cae8e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1338

[GitHub] [hudi] hudi-bot commented on pull request #7360: [MINOR] Fix Out of Bounds Exception for DayBasedCompactionStrategy

2022-12-01 Thread GitBox
hudi-bot commented on PR #7360: URL: https://github.com/apache/hudi/pull/7360#issuecomment-1334754039 ## CI report: * 067360445f6ba0a94c891052103cc8d383db733a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1339

[GitHub] [hudi] hudi-bot commented on pull request #7238: [HUDI-5023] Cleaning up `QueueBasedExecutor` impls

2022-12-01 Thread GitBox
hudi-bot commented on PR #7238: URL: https://github.com/apache/hudi/pull/7238#issuecomment-1334753883 ## CI report: * ba0309167969884856506a0c9b903651ce935e52 UNKNOWN * a76c39fde821f8a7053a3bbc5215bd7bc3aa3af4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7339: [HUDI-5301] Spark SQL queries support setting parameters through set

2022-12-01 Thread GitBox
hudi-bot commented on PR #7339: URL: https://github.com/apache/hudi/pull/7339#issuecomment-1334753999 ## CI report: * adef447a18527e5628c8ba53dfb245bafc5cae8e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1338

[GitHub] [hudi] voonhous commented on a diff in pull request #6915: [HUDI-5004] Allow nested field as primary key and preCombineField in flink sql

2022-12-01 Thread GitBox
voonhous commented on code in PR #6915: URL: https://github.com/apache/hudi/pull/6915#discussion_r1037771871 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java: ## @@ -120,6 +122,7 @@ public Set> optionalOptions() { */ private

[GitHub] [hudi] voonhous commented on a diff in pull request #6915: [HUDI-5004] Allow nested field as primary key and preCombineField in flink sql

2022-12-01 Thread GitBox
voonhous commented on code in PR #6915: URL: https://github.com/apache/hudi/pull/6915#discussion_r1037771871 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java: ## @@ -120,6 +122,7 @@ public Set> optionalOptions() { */ private

[GitHub] [hudi] CodeCooker17 commented on pull request #6793: [HUDI-4917] Optimized the way to get HoodieBaseFile of loadColumnRangesFromFiles of Bloom Index

2022-12-01 Thread GitBox
CodeCooker17 commented on PR #6793: URL: https://github.com/apache/hudi/pull/6793#issuecomment-1334730506 > @CodeCooker17 can you plz help address the comment? Let me try to fix it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hudi] zhangyue19921010 commented on pull request #7238: [HUDI-5023] Cleaning up `QueueBasedExecutor` impls

2022-12-01 Thread GitBox
zhangyue19921010 commented on PR #7238: URL: https://github.com/apache/hudi/pull/7238#issuecomment-1334724273 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [hudi] danny0405 commented on a diff in pull request #7306: [HUDI-5280] Fix taskmanager concurrent registry

2022-12-01 Thread GitBox
danny0405 commented on code in PR #7306: URL: https://github.com/apache/hudi/pull/7306#discussion_r1037758501 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/metrics/HoodieLockMetrics.java: ## @@ -70,11 +71,13 @@ private String getMetricsN

[GitHub] [hudi] danny0405 commented on a diff in pull request #6915: [HUDI-5004] Allow nested field as primary key and preCombineField in flink sql

2022-12-01 Thread GitBox
danny0405 commented on code in PR #6915: URL: https://github.com/apache/hudi/pull/6915#discussion_r1037755466 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java: ## @@ -120,6 +122,7 @@ public Set> optionalOptions() { */ private

[GitHub] [hudi] hudi-bot commented on pull request #7326: [HUDI-5294] Support type change for schema on read + reconcile schema

2022-12-01 Thread GitBox
hudi-bot commented on PR #7326: URL: https://github.com/apache/hudi/pull/7326#issuecomment-1334715565 ## CI report: * 5deae66bc506bb01f0bfce1cac16509ae38c07c8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1338

[GitHub] [hudi] hudi-bot commented on pull request #7326: [HUDI-5294] Support type change for schema on read + reconcile schema

2022-12-01 Thread GitBox
hudi-bot commented on PR #7326: URL: https://github.com/apache/hudi/pull/7326#issuecomment-1334712544 ## CI report: * 5deae66bc506bb01f0bfce1cac16509ae38c07c8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1338

[GitHub] [hudi] danny0405 commented on issue #7352: [SUPPORT]When writing to a hudi table synchronized with hive via Flink, the amount of data in the hudi table does not match the amount of data in th

2022-12-01 Thread GitBox
danny0405 commented on issue #7352: URL: https://github.com/apache/hudi/issues/7352#issuecomment-1334710488 What kind of input format did you set up with, I see you uses the MERGE_ON_READ as table type. -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [hudi] hudi-bot commented on pull request #7359: [HUDI-3304] WIP - Allow selective partial update

2022-12-01 Thread GitBox
hudi-bot commented on PR #7359: URL: https://github.com/apache/hudi/pull/7359#issuecomment-1334709261 ## CI report: * 6e40669c65328c4247f3cc390f2dd15920fde590 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1339

[GitHub] [hudi] danny0405 commented on pull request #7320: [HUDI-5290] remove the lock in #writeTableMetadata

2022-12-01 Thread GitBox
danny0405 commented on PR #7320: URL: https://github.com/apache/hudi/pull/7320#issuecomment-1334707283 @just-JL Hi, can you apply the patch and rebase with the latest master then force-push again ? -- This is an automated message from the Apache Git Service. To respond to the message, ple

[jira] [Updated] (HUDI-5303) Allow users to control the concurrency to submit jobs in clustering

2022-12-01 Thread Hui An (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui An updated HUDI-5303: - Description: If there are sufficient resources in the clustering job, some clustering groups sometimes could still

[GitHub] [hudi] xushiyan commented on issue #7261: [SUPPORT] Meta sync error when trying to write to s3 bucket

2022-12-01 Thread GitBox
xushiyan commented on issue #7261: URL: https://github.com/apache/hudi/issues/7261#issuecomment-1334697019 @lokeshj1703 can you take a look pls? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [hudi] xushiyan commented on issue #7353: [BUG] Hudi 0.11.x support for Spark with CTAS throws hive sync error

2022-12-01 Thread GitBox
xushiyan commented on issue #7353: URL: https://github.com/apache/hudi/issues/7353#issuecomment-1334691281 @jonvex can you check pls? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] xushiyan commented on issue #7354: [BUG] Hudi 0.11.x Spark SQL CTAS requires `hoodie` properties in addition to table properties

2022-12-01 Thread GitBox
xushiyan commented on issue #7354: URL: https://github.com/apache/hudi/issues/7354#issuecomment-1334690373 @jonvex can you look into this pls? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Assigned] (HUDI-5051) Add a functional regression test for Bloom Index followed on w/ Upserts

2022-12-01 Thread Raymond Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu reassigned HUDI-5051: Assignee: sivabalan narayanan (was: Alexey Kudinkin) > Add a functional regression test for Bloom

[GitHub] [hudi] hudi-bot commented on pull request #7349: [HUDI-5291] Fixing NPE in MOR column stats accounting

2022-12-01 Thread GitBox
hudi-bot commented on PR #7349: URL: https://github.com/apache/hudi/pull/7349#issuecomment-1334668826 ## CI report: * 6a9050f88a70db883ace5df987092c0eb7f0764a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1338

[GitHub] [hudi] hudi-bot commented on pull request #7349: [HUDI-5291] Fixing NPE in MOR column stats accounting

2022-12-01 Thread GitBox
hudi-bot commented on PR #7349: URL: https://github.com/apache/hudi/pull/7349#issuecomment-1334665288 ## CI report: * 6a9050f88a70db883ace5df987092c0eb7f0764a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1338

[GitHub] [hudi] hudi-bot commented on pull request #7333: [HUDI-5296] Allow disable schema on read after enabling

2022-12-01 Thread GitBox
hudi-bot commented on PR #7333: URL: https://github.com/apache/hudi/pull/7333#issuecomment-1334665209 ## CI report: * c7dae19174b0bb8112990887b39ec5e487732591 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1339

[GitHub] [hudi] hudi-bot commented on pull request #7342: [HUDI-5302] Fix: compute hash key from recordKey failed when recordKeyValue contains ','

2022-12-01 Thread GitBox
hudi-bot commented on PR #7342: URL: https://github.com/apache/hudi/pull/7342#issuecomment-1334662259 ## CI report: * 142b4e2f9fb1f12a4c7d9fa9adf56c3f7b93cd1e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1339

[GitHub] [hudi] hudi-bot commented on pull request #5064: [HUDI-3654] Add new module `hudi-metaserver`

2022-12-01 Thread GitBox
hudi-bot commented on PR #5064: URL: https://github.com/apache/hudi/pull/5064#issuecomment-1334660830 ## CI report: * 53aa21bf23d2f8b0404743e6d016cfb2fac444f7 UNKNOWN * 07a3ea3956e5ce02a33a55eae4a0339796275f9d UNKNOWN * 810af96ee856bd94cfc82b01b67765a735f29c44 UNKNOWN * c4

[GitHub] [hudi] XuQianJin-Stars commented on pull request #7342: [HUDI-5302] Fix: compute hash key from recordKey failed when recordKeyValue contains ','

2022-12-01 Thread GitBox
XuQianJin-Stars commented on PR #7342: URL: https://github.com/apache/hudi/pull/7342#issuecomment-1334657091 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[hudi] branch asf-site updated: [DOCS] Fix compaction cli usage (#7358)

2022-12-01 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new 6445fad28af [DOCS] Fix compaction cli usage (#

[GitHub] [hudi] codope merged pull request #7358: [DOCS] Fix compaction cli usage

2022-12-01 Thread GitBox
codope merged PR #7358: URL: https://github.com/apache/hudi/pull/7358 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.or

[GitHub] [hudi] hudi-bot commented on pull request #7360: [MINOR] Fix Out of Bounds Exception for DayBasedCompactionStrategy

2022-12-01 Thread GitBox
hudi-bot commented on PR #7360: URL: https://github.com/apache/hudi/pull/7360#issuecomment-1334618369 ## CI report: * 067360445f6ba0a94c891052103cc8d383db733a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1339

[GitHub] [hudi] hudi-bot commented on pull request #7360: [MINOR] Fix Out of Bounds Exception for DayBasedCompactionStrategy

2022-12-01 Thread GitBox
hudi-bot commented on PR #7360: URL: https://github.com/apache/hudi/pull/7360#issuecomment-1334614326 ## CI report: * 067360445f6ba0a94c891052103cc8d383db733a UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7345: [HUDI-3378] RFC46 rebase

2022-12-01 Thread GitBox
hudi-bot commented on PR #7345: URL: https://github.com/apache/hudi/pull/7345#issuecomment-1334614269 ## CI report: * e5f1dba84479f08417f25f53a79f6dae4425ba23 UNKNOWN * 75cc2b5607bc7ef7438631aa455749ed8c420bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2

[GitHub] [hudi] hudi-bot commented on pull request #7342: [HUDI-5302] Fix: compute hash key from recordKey failed when recordKeyValue contains ','

2022-12-01 Thread GitBox
hudi-bot commented on PR #7342: URL: https://github.com/apache/hudi/pull/7342#issuecomment-1334614223 ## CI report: * 142b4e2f9fb1f12a4c7d9fa9adf56c3f7b93cd1e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1339

[GitHub] [hudi] praneethh commented on issue #7335: [SUPPORT] Update only if precombine field of incoming record is greater than the existing record.

2022-12-01 Thread GitBox
praneethh commented on issue #7335: URL: https://github.com/apache/hudi/issues/7335#issuecomment-1334584083 Hi @kazdy, tried by setting this config ```hoodie.compaction.payload.class``` but still got the same result. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] vamshigv closed pull request #7331: [MINOR] Fix out of bounds with Compaction based strategy

2022-12-01 Thread GitBox
vamshigv closed pull request #7331: [MINOR] Fix out of bounds with Compaction based strategy URL: https://github.com/apache/hudi/pull/7331 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [hudi] vamshigv opened a new pull request, #7360: Fix Out of Bounds Exception for DayBasedCompactionStrategy

2022-12-01 Thread GitBox
vamshigv opened a new pull request, #7360: URL: https://github.com/apache/hudi/pull/7360 ### Change Logs Fix possible out of bounds exception in DayBasedCompactionStrategy during sublist operation. ### Impact Bug fix. ### Risk level (write none, low medium or high

[GitHub] [hudi] hudi-bot commented on pull request #7359: [HUDI-3304] WIP - Allow selective partial update

2022-12-01 Thread GitBox
hudi-bot commented on PR #7359: URL: https://github.com/apache/hudi/pull/7359#issuecomment-1334569523 ## CI report: * 6e40669c65328c4247f3cc390f2dd15920fde590 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1339

[GitHub] [hudi] hudi-bot commented on pull request #7333: [HUDI-5296] Allow disable schema on read after enabling

2022-12-01 Thread GitBox
hudi-bot commented on PR #7333: URL: https://github.com/apache/hudi/pull/7333#issuecomment-1334569443 ## CI report: * 858997f2402215c86b4740ab06fe2056b3ad1fcc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1332

[GitHub] [hudi] hudi-bot commented on pull request #7331: [MINOR] Fix out of bounds with Compaction based strategy

2022-12-01 Thread GitBox
hudi-bot commented on PR #7331: URL: https://github.com/apache/hudi/pull/7331#issuecomment-1334569413 ## CI report: * 1c61c4a01a58f93a190befa3ff8429e4c83fba36 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1332

[GitHub] [hudi] CTTY commented on a diff in pull request #6882: [WIP][HUDI-3625] Object Store Storage Strategy

2022-12-01 Thread GitBox
CTTY commented on code in PR #6882: URL: https://github.com/apache/hudi/pull/6882#discussion_r1037645450 ## hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java: ## @@ -127,6 +132,20 @@ protected HoodieTableMetaClient(Configuration conf, String base

[GitHub] [hudi] hudi-bot commented on pull request #5064: [HUDI-3654] Add new module `hudi-metaserver`

2022-12-01 Thread GitBox
hudi-bot commented on PR #5064: URL: https://github.com/apache/hudi/pull/5064#issuecomment-1334567599 ## CI report: * 53aa21bf23d2f8b0404743e6d016cfb2fac444f7 UNKNOWN * 07a3ea3956e5ce02a33a55eae4a0339796275f9d UNKNOWN * 810af96ee856bd94cfc82b01b67765a735f29c44 UNKNOWN * d8

[GitHub] [hudi] hudi-bot commented on pull request #7359: [HUDI-3304] WIP - Allow selective partial update

2022-12-01 Thread GitBox
hudi-bot commented on PR #7359: URL: https://github.com/apache/hudi/pull/7359#issuecomment-1334564747 ## CI report: * 6e40669c65328c4247f3cc390f2dd15920fde590 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7333: [HUDI-5296] Allow disable schema on read after enabling

2022-12-01 Thread GitBox
hudi-bot commented on PR #7333: URL: https://github.com/apache/hudi/pull/7333#issuecomment-1334564640 ## CI report: * 858997f2402215c86b4740ab06fe2056b3ad1fcc Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1332

[GitHub] [hudi] hudi-bot commented on pull request #5064: [HUDI-3654] Add new module `hudi-metaserver`

2022-12-01 Thread GitBox
hudi-bot commented on PR #5064: URL: https://github.com/apache/hudi/pull/5064#issuecomment-1334562453 ## CI report: * 53aa21bf23d2f8b0404743e6d016cfb2fac444f7 UNKNOWN * bb3c7a9d580970b38f797cee45e78748468aa194 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-

[GitHub] [hudi] nsivabalan commented on pull request #7333: [HUDI-5296] Allow disable schema on read after enabling

2022-12-01 Thread GitBox
nsivabalan commented on PR #7333: URL: https://github.com/apache/hudi/pull/7333#issuecomment-1334546960 @alexeykudinkin @xiarixiaoyao : addressed feedback -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hudi] CTTY commented on a diff in pull request #6882: [WIP][HUDI-3625] Object Store Storage Strategy

2022-12-01 Thread GitBox
CTTY commented on code in PR #6882: URL: https://github.com/apache/hudi/pull/6882#discussion_r1037629840 ## hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java: ## @@ -533,6 +540,18 @@ public static Stream getAllLogFiles(FileSystem fs, Path partition } }

[GitHub] [hudi] bschell opened a new pull request, #7359: [HUDI-3304] WIP - Allow selective partial update

2022-12-01 Thread GitBox
bschell opened a new pull request, #7359: URL: https://github.com/apache/hudi/pull/7359 ### Change Logs This is a WIP Pull Request. This is intended to receive early feedback on the design approach. Expand on the Hudi partial update payload to create one that will allow "null"

[GitHub] [hudi] CTTY commented on a diff in pull request #6882: [WIP][HUDI-3625] Object Store Storage Strategy

2022-12-01 Thread GitBox
CTTY commented on code in PR #6882: URL: https://github.com/apache/hudi/pull/6882#discussion_r1037628682 ## hudi-common/src/main/java/org/apache/hudi/metadata/BaseTableMetadata.java: ## @@ -367,7 +369,9 @@ Map fetchAllFilesInPartitionPaths(List partitionPath Hoodie

[GitHub] [hudi] CTTY commented on a diff in pull request #6882: [WIP][HUDI-3625] Object Store Storage Strategy

2022-12-01 Thread GitBox
CTTY commented on code in PR #6882: URL: https://github.com/apache/hudi/pull/6882#discussion_r1037627383 ## hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java: ## @@ -195,8 +196,16 @@ public static long getFileSize(FileSystem fs, Path path) throws IOException {

[jira] [Closed] (HUDI-5311) Add index, finalize and commit metrics for insertOverwrite and insertOverwriteTable operations

2022-12-01 Thread Hussein Awala (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hussein Awala closed HUDI-5311. --- Fix Version/s: (was: 0.13.0) Resolution: Implemented > Add index, finalize and commit metri

[GitHub] [hudi] CTTY commented on a diff in pull request #6882: [WIP][HUDI-3625] Object Store Storage Strategy

2022-12-01 Thread GitBox
CTTY commented on code in PR #6882: URL: https://github.com/apache/hudi/pull/6882#discussion_r1037614402 ## hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java: ## @@ -127,6 +132,20 @@ protected HoodieTableMetaClient(Configuration conf, String base

[GitHub] [hudi] CTTY commented on a diff in pull request #6882: [WIP][HUDI-3625] Object Store Storage Strategy

2022-12-01 Thread GitBox
CTTY commented on code in PR #6882: URL: https://github.com/apache/hudi/pull/6882#discussion_r1037609735 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCreateHandle.java: ## @@ -56,7 +56,7 @@ private static final Logger LOG = LogManager.getLogger(Ho

[GitHub] [hudi] CTTY commented on a diff in pull request #6882: [WIP][HUDI-3625] Object Store Storage Strategy

2022-12-01 Thread GitBox
CTTY commented on code in PR #6882: URL: https://github.com/apache/hudi/pull/6882#discussion_r1037608955 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java: ## @@ -165,6 +167,16 @@ public class HoodieWriteConfig extends HoodieConfig {

[GitHub] [hudi] hussein-awala closed pull request #7357: [HUDI-5311] Add some metrics for insertOverwrite and insertOverwriteTable operations

2022-12-01 Thread GitBox
hussein-awala closed pull request #7357: [HUDI-5311] Add some metrics for insertOverwrite and insertOverwriteTable operations URL: https://github.com/apache/hudi/pull/7357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [hudi] hudi-bot commented on pull request #7357: [HUDI-5311] Add some metrics for insertOverwrite and insertOverwriteTable operations

2022-12-01 Thread GitBox
hudi-bot commented on PR #7357: URL: https://github.com/apache/hudi/pull/7357#issuecomment-1334500464 ## CI report: * f0882dd7dd3eb171cbb23f822f0b65d70e7110fd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1338

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2022-12-01 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1334481882 ## CI report: * a8dd96042f42ca74fa8789decdea7397072ec890 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1338

[jira] [Updated] (HUDI-5305) Detect concurrent writes during compaction and clustering if they shouldn't happen

2022-12-01 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5305: -- Status: In Progress (was: Open) > Detect concurrent writes during compaction and clustering if

[jira] [Updated] (HUDI-5305) Detect concurrent writes during compaction and clustering if they shouldn't happen

2022-12-01 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5305: -- Sprint: 2022/11/29 > Detect concurrent writes during compaction and clustering if they shouldn't

[jira] [Updated] (HUDI-5295) With multiple meta syncs, one meta sync failure should not impact other meta syncs.

2022-12-01 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5295: -- Status: In Progress (was: Open) > With multiple meta syncs, one meta sync failure should not im

[jira] [Updated] (HUDI-5295) With multiple meta syncs, one meta sync failure should not impact other meta syncs.

2022-12-01 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5295: -- Status: Patch Available (was: In Progress) > With multiple meta syncs, one meta sync failure sh

[jira] [Updated] (HUDI-5295) With multiple meta syncs, one meta sync failure should not impact other meta syncs.

2022-12-01 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-5295: -- Sprint: 2022/11/29 > With multiple meta syncs, one meta sync failure should not impact other met

[GitHub] [hudi] hudi-bot commented on pull request #7349: [HUDI-5291] Fixing NPE in MOR column stats accounting

2022-12-01 Thread GitBox
hudi-bot commented on PR #7349: URL: https://github.com/apache/hudi/pull/7349#issuecomment-1334416876 ## CI report: * 6a9050f88a70db883ace5df987092c0eb7f0764a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1338

[GitHub] [hudi] hudi-bot commented on pull request #5064: [HUDI-3654] Add new module `hudi-metaserver`

2022-12-01 Thread GitBox
hudi-bot commented on PR #5064: URL: https://github.com/apache/hudi/pull/5064#issuecomment-1334408753 ## CI report: * 53aa21bf23d2f8b0404743e6d016cfb2fac444f7 UNKNOWN * bb3c7a9d580970b38f797cee45e78748468aa194 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-

[GitHub] [hudi] hudi-bot commented on pull request #5064: [HUDI-3654] Add new module `hudi-metaserver`

2022-12-01 Thread GitBox
hudi-bot commented on PR #5064: URL: https://github.com/apache/hudi/pull/5064#issuecomment-1334400201 ## CI report: * 53aa21bf23d2f8b0404743e6d016cfb2fac444f7 UNKNOWN * bb3c7a9d580970b38f797cee45e78748468aa194 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-

[GitHub] [hudi] kazdy commented on issue #7335: [SUPPORT] Update only if precombine field of incoming record is greater than the existing record.

2022-12-01 Thread GitBox
kazdy commented on issue #7335: URL: https://github.com/apache/hudi/issues/7335#issuecomment-1334390971 Hi, Have you tried to set this config: `hoodie.compaction.payload.class' (default is OverwriteWithLatestAvroPayload) https://hudi.apache.org/docs/configurations/#hoodiecom

  1   2   3   >