[GitHub] [hudi] ThinkerLei commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-06 Thread GitBox
ThinkerLei commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374407544 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-06 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1374403379 ## CI report: * 36928781f82776d26733679217f244d911db339b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-06 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1374400111 ## CI report: * 36928781f82776d26733679217f244d911db339b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #7619: [MINOR] Optimizing schema validation in Metadata table

2023-01-06 Thread GitBox
hudi-bot commented on PR #7619: URL: https://github.com/apache/hudi/pull/7619#issuecomment-1374397323 ## CI report: * 2e36086458f1f634fab4aa3cbca75c30a2afe592 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-06 Thread GitBox
hudi-bot commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374397317 ## CI report: * 94a8e3bb534c386cc55c3150120c8e56b7596f29 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] xushiyan commented on a diff in pull request #7607: [HUDI-5499] Fixing Spark SQL configs not being properly propagated for CTAS and other commands

2023-01-06 Thread GitBox
xushiyan commented on code in PR #7607: URL: https://github.com/apache/hudi/pull/7607#discussion_r1063958149 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -81,10 +80,8 @@ trait ProvidesHoodieConfig extends L

[GitHub] [hudi] xushiyan commented on a diff in pull request #7607: [HUDI-5499] Fixing Spark SQL configs not being properly propagated for CTAS and other commands

2023-01-06 Thread GitBox
xushiyan commented on code in PR #7607: URL: https://github.com/apache/hudi/pull/7607#discussion_r1063957987 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -81,10 +80,8 @@ trait ProvidesHoodieConfig extends L

[GitHub] [hudi] hudi-bot commented on pull request #7620: [HUDI-5511] Do not clean the CkpMetadata dir when restart the job

2023-01-06 Thread GitBox
hudi-bot commented on PR #7620: URL: https://github.com/apache/hudi/pull/7620#issuecomment-1374386293 ## CI report: * cd670233392323f8602950a5d2595661b668f3e9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1416

[GitHub] [hudi] xushiyan commented on a diff in pull request #7607: [HUDI-5499] Fixing Spark SQL configs not being properly propagated for CTAS and other commands

2023-01-06 Thread GitBox
xushiyan commented on code in PR #7607: URL: https://github.com/apache/hudi/pull/7607#discussion_r1062620289 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala: ## @@ -203,26 +193,20 @@ trait ProvidesHoodieConfig extend

[GitHub] [hudi] hudi-bot commented on pull request #7620: [HUDI-5511] Do not clean the CkpMetadata dir when restart the job

2023-01-06 Thread GitBox
hudi-bot commented on PR #7620: URL: https://github.com/apache/hudi/pull/7620#issuecomment-1374385575 ## CI report: * cd670233392323f8602950a5d2595661b668f3e9 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-5511) Do not clean the CkpMetadata dir when restart the job

2023-01-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5511: - Labels: pull-request-available (was: ) > Do not clean the CkpMetadata dir when restart the job >

[GitHub] [hudi] hudi-bot commented on pull request #7585: [MINOR] Extending support to other special characters

2023-01-06 Thread GitBox
hudi-bot commented on PR #7585: URL: https://github.com/apache/hudi/pull/7585#issuecomment-1374384804 ## CI report: * 85370798131dd4272a55fdea95195c0dd880a2ef Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] danny0405 opened a new pull request, #7620: [HUDI-5511] Do not clean the CkpMetadata dir when restart the job

2023-01-06 Thread GitBox
danny0405 opened a new pull request, #7620: URL: https://github.com/apache/hudi/pull/7620 At the beginning, we bootstrap the ckp metadata by cleaning all the messages. This brings in some corner case like 'the write task can not fetch the pending instant correctly when restartting the job',

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-06 Thread GitBox
alexeykudinkin commented on code in PR #6361: URL: https://github.com/apache/hudi/pull/6361#discussion_r1056726252 ## hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala: ## @@ -57,60 +54,82 @@ object HoodieAnalysis {

[jira] [Updated] (HUDI-5511) Do not clean the CkpMetadata dir when restart the job

2023-01-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-5511: - Sprint: 0.13.0 Final Sprint > Do not clean the CkpMetadata dir when restart the job >

[jira] [Updated] (HUDI-5511) Do not clean the CkpMetadata dir when restart the job

2023-01-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-5511: - Status: In Progress (was: Open) > Do not clean the CkpMetadata dir when restart the job > ---

[jira] [Updated] (HUDI-5511) Do not clean the CkpMetadata dir when restart the job

2023-01-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-5511: - Priority: Blocker (was: Major) > Do not clean the CkpMetadata dir when restart the job >

[jira] [Created] (HUDI-5511) Do not clean the CkpMetadata dir when restart the job

2023-01-06 Thread Danny Chen (Jira)
Danny Chen created HUDI-5511: Summary: Do not clean the CkpMetadata dir when restart the job Key: HUDI-5511 URL: https://issues.apache.org/jira/browse/HUDI-5511 Project: Apache Hudi Issue Type: B

[hudi] branch master updated (8ae1e8c6761 -> 633fe09514e)

2023-01-06 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 8ae1e8c6761 [HUDI-5231] suppress checkstyle warnings (#7473) add 633fe09514e [MINOR] Set engine when creating me

[GitHub] [hudi] nsivabalan commented on a diff in pull request #6782: [HUDI-4911][HUDI-3301] Fixing `HoodieMetadataLogRecordReader` to avoid flushing cache for every lookup

2023-01-06 Thread GitBox
nsivabalan commented on code in PR #6782: URL: https://github.com/apache/hudi/pull/6782#discussion_r1063952944 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java: ## @@ -108,30 +112,85 @@ protected HoodieMergedLogRecordScanner(FileSys

[GitHub] [hudi] danny0405 merged pull request #7575: [MINOR] Set engine when creating meta write config

2023-01-06 Thread GitBox
danny0405 merged PR #7575: URL: https://github.com/apache/hudi/pull/7575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache

[GitHub] [hudi] nsivabalan commented on a diff in pull request #6782: [HUDI-4911][HUDI-3301] Fixing `HoodieMetadataLogRecordReader` to avoid flushing cache for every lookup

2023-01-06 Thread GitBox
nsivabalan commented on code in PR #6782: URL: https://github.com/apache/hudi/pull/6782#discussion_r1063950675 ## hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java: ## @@ -108,30 +112,85 @@ protected HoodieMergedLogRecordScanner(FileSys

[GitHub] [hudi] hudi-bot commented on pull request #7619: [MINOR] Optimizing schema validation in Metadata table

2023-01-06 Thread GitBox
hudi-bot commented on PR #7619: URL: https://github.com/apache/hudi/pull/7619#issuecomment-1374372971 ## CI report: * 2e36086458f1f634fab4aa3cbca75c30a2afe592 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-06 Thread GitBox
hudi-bot commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374372947 ## CI report: * be11f7842cdb9fd4e108c6a7a5662281ec9e2dcc Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1413

[GitHub] [hudi] hudi-bot commented on pull request #7619: [MINOR] Optimizing schema validation in Metadata table

2023-01-06 Thread GitBox
hudi-bot commented on PR #7619: URL: https://github.com/apache/hudi/pull/7619#issuecomment-1374372068 ## CI report: * 2e36086458f1f634fab4aa3cbca75c30a2afe592 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-06 Thread GitBox
hudi-bot commented on PR #7609: URL: https://github.com/apache/hudi/pull/7609#issuecomment-1374372055 ## CI report: * be11f7842cdb9fd4e108c6a7a5662281ec9e2dcc Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1413

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-06 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1374371824 ## CI report: * 36928781f82776d26733679217f244d911db339b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2023-01-06 Thread GitBox
hudi-bot commented on PR #7490: URL: https://github.com/apache/hudi/pull/7490#issuecomment-1374371039 ## CI report: * 4dbf55bb566e21a4ec857e8c321459eef34900af Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] nsivabalan opened a new pull request, #7619: [MINOR] Optimizing schema validation in Metadata table

2023-01-06 Thread GitBox
nsivabalan opened a new pull request, #7619: URL: https://github.com/apache/hudi/pull/7619 ### Change Logs Metadata table schema is fixes and so there is no real necessity to check for schema compatability everytime. we can save off some latencies. ### Impact Avoiding s

[GitHub] [hudi] ThinkerLei commented on a diff in pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-06 Thread GitBox
ThinkerLei commented on code in PR #7609: URL: https://github.com/apache/hudi/pull/7609#discussion_r1063944665 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/compact/CompactOperator.java: ## @@ -98,10 +99,16 @@ public void open() throws Exception { }

[jira] [Closed] (HUDI-5506) StreamWriteOperatorCoordinator may not recommit with partial uncommitted write metadata event

2023-01-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-5506. Resolution: Fixed > StreamWriteOperatorCoordinator may not recommit with partial uncommitted > write metada

[GitHub] [hudi] danny0405 commented on a diff in pull request #7609: [HUDI-5504]Fix concurrency conflict when asyncCompaction is enabled

2023-01-06 Thread GitBox
danny0405 commented on code in PR #7609: URL: https://github.com/apache/hudi/pull/7609#discussion_r1063077476 ## hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/compact/CompactOperator.java: ## @@ -18,6 +18,7 @@ package org.apache.hudi.sink.compact; +imp

[jira] [Updated] (HUDI-5506) StreamWriteOperatorCoordinator may not recommit with partial uncommitted write metadata event

2023-01-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-5506: - Status: Patch Available (was: In Progress) > StreamWriteOperatorCoordinator may not recommit with partial

[jira] [Updated] (HUDI-5506) StreamWriteOperatorCoordinator may not recommit with partial uncommitted write metadata event

2023-01-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-5506: - Attachment: image-2023-01-07-10-58-14-728.png > StreamWriteOperatorCoordinator may not recommit with parti

[jira] [Commented] (HUDI-5506) StreamWriteOperatorCoordinator may not recommit with partial uncommitted write metadata event

2023-01-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655631#comment-17655631 ] Danny Chen commented on HUDI-5506: -- Fixed via master branch: ac296c702417461cedeef6e8974b

[jira] [Updated] (HUDI-5506) StreamWriteOperatorCoordinator may not recommit with partial uncommitted write metadata event

2023-01-06 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-5506: - Status: In Progress (was: Open) > StreamWriteOperatorCoordinator may not recommit with partial uncommitte

[GitHub] [hudi] hudi-bot commented on pull request #7612: [HUDI-5336] Fixing log file pattern match to ignore extraneous files

2023-01-06 Thread GitBox
hudi-bot commented on PR #7612: URL: https://github.com/apache/hudi/pull/7612#issuecomment-1374357360 ## CI report: * 66370e1d4085619050625bf32e08dc9c8cef8f76 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #7585: [MINOR] Extending support to other special characters

2023-01-06 Thread GitBox
hudi-bot commented on PR #7585: URL: https://github.com/apache/hudi/pull/7585#issuecomment-1374334148 ## CI report: * 9feb5f9256e9dae62fa657420b7c4b73af70ef92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2023-01-06 Thread GitBox
hudi-bot commented on PR #7490: URL: https://github.com/apache/hudi/pull/7490#issuecomment-1374334066 ## CI report: * f0b791ecbc8da893188991f34490c165dceaf24d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1412

[GitHub] [hudi] hudi-bot commented on pull request #7605: [HUDI-5349] Clean up partially failed restore

2023-01-06 Thread GitBox
hudi-bot commented on PR #7605: URL: https://github.com/apache/hudi/pull/7605#issuecomment-1374331958 ## CI report: * 943601c9ea133a988fb8b809e29889cb0df07a94 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #7585: [MINOR] Extending support to other special characters

2023-01-06 Thread GitBox
hudi-bot commented on PR #7585: URL: https://github.com/apache/hudi/pull/7585#issuecomment-1374331934 ## CI report: * 9feb5f9256e9dae62fa657420b7c4b73af70ef92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[GitHub] [hudi] hudi-bot commented on pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2023-01-06 Thread GitBox
hudi-bot commented on PR #7490: URL: https://github.com/apache/hudi/pull/7490#issuecomment-1374331841 ## CI report: * f0b791ecbc8da893188991f34490c165dceaf24d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1412

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7605: [HUDI-5349] Clean up partially failed restore

2023-01-06 Thread GitBox
nsivabalan commented on code in PR #7605: URL: https://github.com/apache/hudi/pull/7605#discussion_r1063907200 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java: ## @@ -836,17 +838,19 @@ public boolean rollback(final String commitI

[GitHub] [hudi] nsivabalan commented on pull request #7585: [MINOR] Extending support to other special characters

2023-01-06 Thread GitBox
nsivabalan commented on PR #7585: URL: https://github.com/apache/hudi/pull/7585#issuecomment-1374302238 there were some check style issues, fixed and pushed them -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] hudi-bot commented on pull request #7585: [MINOR] Extending support to other special characters

2023-01-06 Thread GitBox
hudi-bot commented on PR #7585: URL: https://github.com/apache/hudi/pull/7585#issuecomment-1374275466 ## CI report: * 9feb5f9256e9dae62fa657420b7c4b73af70ef92 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1415

[jira] [Commented] (HUDI-5508) Revamp hudi homepage website

2023-01-06 Thread nadine (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655597#comment-17655597 ] nadine commented on HUDI-5508: -- closing issue- pr is merged.  > Revamp hudi homepage website

[jira] [Closed] (HUDI-5508) Revamp hudi homepage website

2023-01-06 Thread nadine (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nadine closed HUDI-5508. Resolution: Fixed > Revamp hudi homepage website > > > Key: HUDI-5508 >

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-06 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1374209822 ## CI report: * 04d18204862b8eccdb6f3d78b5b4bc91ad1b4915 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1413

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-06 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1374205412 ## CI report: * 04d18204862b8eccdb6f3d78b5b4bc91ad1b4915 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1413

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7573: [HUDI-5484] Avoid using `GenericRecord` in `HoodieColumnStatMetadata`

2023-01-06 Thread GitBox
alexeykudinkin commented on code in PR #7573: URL: https://github.com/apache/hudi/pull/7573#discussion_r1063822975 ## hudi-common/src/test/java/org/apache/hudi/common/util/TestSerializationUtils.java: ## @@ -80,6 +86,30 @@ public void testClassFullyQualifiedNameSerialization()

[GitHub] [hudi] alexeykudinkin commented on pull request #7573: [HUDI-5484] Avoid using GenericRecord in ColumnStatMetadata

2023-01-06 Thread GitBox
alexeykudinkin commented on PR #7573: URL: https://github.com/apache/hudi/pull/7573#issuecomment-1374167160 @cxzl25 please update the issue with the description of the root-cause as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Updated] (HUDI-5484) Avoid using GenericRecord in ColumnStatMetadata

2023-01-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5484: -- Sprint: 0.13.0 Final Sprint > Avoid using GenericRecord in ColumnStatMetadata >

[jira] [Updated] (HUDI-5484) Avoid using GenericRecord in ColumnStatMetadata

2023-01-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5484: -- Affects Version/s: 0.12.1 > Avoid using GenericRecord in ColumnStatMetadata > --

[jira] [Updated] (HUDI-5484) Avoid using GenericRecord in ColumnStatMetadata

2023-01-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5484: -- Fix Version/s: 0.13.0 > Avoid using GenericRecord in ColumnStatMetadata > --

[jira] [Updated] (HUDI-5484) Avoid using GenericRecord in ColumnStatMetadata

2023-01-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5484: -- Priority: Blocker (was: Critical) > Avoid using GenericRecord in ColumnStatMetadata > -

[jira] [Updated] (HUDI-5484) Avoid using GenericRecord in ColumnStatMetadata

2023-01-06 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5484: -- Reviewers: Alexey Kudinkin > Avoid using GenericRecord in ColumnStatMetadata > -

[jira] [Updated] (HUDI-3775) Allow for offline compaction of MOR tables via spark streaming

2023-01-06 Thread Jonathan Vexler (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Vexler updated HUDI-3775: -- Status: In Progress (was: Open) > Allow for offline compaction of MOR tables via spark streamin

[GitHub] [hudi] bhasudha merged pull request #7537: [HUDI-5508][DOCS] Feat: landing page design changes

2023-01-06 Thread GitBox
bhasudha merged PR #7537: URL: https://github.com/apache/hudi/pull/7537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.

[GitHub] [hudi] bhasudha commented on pull request #7537: [HUDI-5508][DOCS] Feat: landing page design changes

2023-01-06 Thread GitBox
bhasudha commented on PR #7537: URL: https://github.com/apache/hudi/pull/7537#issuecomment-1374157224 LGTM. Thanks @pintusoliya for doing the UI fixes and @nfarah86 for helping with content and images. -- This is an automated message from the Apache Git Service. To respond to the message

[jira] [Updated] (HUDI-5508) Revamp hudi homepage website

2023-01-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5508: - Labels: pull-request-available website (was: website) > Revamp hudi homepage website > --

[GitHub] [hudi] bhasudha commented on pull request #7537: [HUDI-5508][DOCS] Feat: landing page design changes

2023-01-06 Thread GitBox
bhasudha commented on PR #7537: URL: https://github.com/apache/hudi/pull/7537#issuecomment-1374155471 I tested the PR locally. Checked the site loads in both languages, all links in the homepage work. Here are the screenshots from local testing. ![Screen Shot 2023-01-06 at 1 26 50 PM](ht

[GitHub] [hudi] hudi-bot commented on pull request #7612: [HUDI-5336] Fixing log file pattern match to ignore extraneous files

2023-01-06 Thread GitBox
hudi-bot commented on PR #7612: URL: https://github.com/apache/hudi/pull/7612#issuecomment-1374131357 ## CI report: * 82e6b63740ebbe7239da0e67c798829e12765aeb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1412

[GitHub] [hudi] hudi-bot commented on pull request #7605: [HUDI-5349] Clean up partially failed restore

2023-01-06 Thread GitBox
hudi-bot commented on PR #7605: URL: https://github.com/apache/hudi/pull/7605#issuecomment-1374131290 ## CI report: * 649ba1ec470943ba2c1d02570447fd4a0294fd98 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1412

[GitHub] [hudi] hudi-bot commented on pull request #7612: [HUDI-5336] Fixing log file pattern match to ignore extraneous files

2023-01-06 Thread GitBox
hudi-bot commented on PR #7612: URL: https://github.com/apache/hudi/pull/7612#issuecomment-1374124965 ## CI report: * 82e6b63740ebbe7239da0e67c798829e12765aeb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1412

[GitHub] [hudi] hudi-bot commented on pull request #7605: [HUDI-5349] Clean up partially failed restore

2023-01-06 Thread GitBox
hudi-bot commented on PR #7605: URL: https://github.com/apache/hudi/pull/7605#issuecomment-1374124876 ## CI report: * 649ba1ec470943ba2c1d02570447fd4a0294fd98 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1412

[GitHub] [hudi] hudi-bot commented on pull request #7618: [MINOR] Polish README and template.md in RFC.

2023-01-06 Thread GitBox
hudi-bot commented on PR #7618: URL: https://github.com/apache/hudi/pull/7618#issuecomment-1374119434 ## CI report: * a8e6918f30422bfbd7f9be740cb05b35609cd9dc Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1414

[GitHub] [hudi] hudi-bot commented on pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-06 Thread GitBox
hudi-bot commented on PR #6732: URL: https://github.com/apache/hudi/pull/6732#issuecomment-1374118065 ## CI report: * Unknown: [CANCELED](TBD) Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build

[GitHub] [hudi] jonvex commented on a diff in pull request #7605: [HUDI-5349] Clean up partially failed restore

2023-01-06 Thread GitBox
jonvex commented on code in PR #7605: URL: https://github.com/apache/hudi/pull/7605#discussion_r1063758391 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java: ## @@ -840,11 +841,25 @@ public boolean rollback(final String commitInsta

[GitHub] [hudi] hudi-bot commented on pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-06 Thread GitBox
hudi-bot commented on PR #6732: URL: https://github.com/apache/hudi/pull/6732#issuecomment-1374047811 ## CI report: * af728e3eeb1fd694eba037bc9a48869831ddb053 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1414

[GitHub] [hudi] hudi-bot commented on pull request #7585: [MINOR] Extending support to other special characters

2023-01-06 Thread GitBox
hudi-bot commented on PR #7585: URL: https://github.com/apache/hudi/pull/7585#issuecomment-1374043600 ## CI report: * 804591fe22d3afa9b3b365222c67bbf4a1b6e156 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1405

[GitHub] [hudi] hudi-bot commented on pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-06 Thread GitBox
hudi-bot commented on PR #6732: URL: https://github.com/apache/hudi/pull/6732#issuecomment-1374042226 ## CI report: * af728e3eeb1fd694eba037bc9a48869831ddb053 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] Shagish commented on issue #7577: [SUPPORT]

2023-01-06 Thread GitBox
Shagish commented on issue #7577: URL: https://github.com/apache/hudi/issues/7577#issuecomment-1374041500 hi @yihua , We have included hoodie.metadata.enable=false and didn't find any further issue. Question - what is the implication if we keep hoodie.metadata.enable=false thr

[GitHub] [hudi] hudi-bot commented on pull request #7585: [MINOR] Extending support to other special characters

2023-01-06 Thread GitBox
hudi-bot commented on PR #7585: URL: https://github.com/apache/hudi/pull/7585#issuecomment-1374038082 ## CI report: * 804591fe22d3afa9b3b365222c67bbf4a1b6e156 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1405

[GitHub] [hudi] hudi-bot commented on pull request #7615: [HUDI-5510] Reload active timeline when getInstantsToArchive

2023-01-06 Thread GitBox
hudi-bot commented on PR #7615: URL: https://github.com/apache/hudi/pull/7615#issuecomment-1374032879 ## CI report: * 6d63669cac8191009b6fc7df2e9ff768463f00d1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1414

[GitHub] [hudi] hudi-bot commented on pull request #6732: [HUDI-4148] Add client for hudi table service manager

2023-01-06 Thread GitBox
hudi-bot commented on PR #6732: URL: https://github.com/apache/hudi/pull/6732#issuecomment-1373960675 ## CI report: * af728e3eeb1fd694eba037bc9a48869831ddb053 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=141

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7605: [HUDI-5349] Clean up partially failed restore

2023-01-06 Thread GitBox
nsivabalan commented on code in PR #7605: URL: https://github.com/apache/hudi/pull/7605#discussion_r1063640166 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/restore/RestoreUtils.java: ## @@ -43,4 +46,24 @@ public static HoodieRestorePlan getRestor

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7605: [HUDI-5349] Clean up partially failed restore

2023-01-06 Thread GitBox
nsivabalan commented on code in PR #7605: URL: https://github.com/apache/hudi/pull/7605#discussion_r1063633199 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/TestClientRollback.java: ## @@ -740,4 +741,117 @@ public void testFallbackToListingBasedRollbackF

[hudi] branch master updated: [HUDI-5231] suppress checkstyle warnings (#7473)

2023-01-06 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8ae1e8c6761 [HUDI-5231] suppress checkstyle war

[GitHub] [hudi] nsivabalan merged pull request #7473: [HUDI-5231] Suppress Checkstyle Warnings

2023-01-06 Thread GitBox
nsivabalan merged PR #7473: URL: https://github.com/apache/hudi/pull/7473 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[hudi] branch master updated: [MINOR] fixed spelling and syntax (#7535)

2023-01-06 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 4e2a5bb488b [MINOR] fixed spelling and syntax (

[GitHub] [hudi] nsivabalan merged pull request #7535: [MINOR] Fixed some spelling and syntax issues in InternalSchemaUtils

2023-01-06 Thread GitBox
nsivabalan merged PR #7535: URL: https://github.com/apache/hudi/pull/7535 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[hudi] branch master updated: [HUDI-5192] add non-code file extensions to ignore list (#7597)

2023-01-06 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new f75671454c9 [HUDI-5192] add non-code file exten

[GitHub] [hudi] nsivabalan merged pull request #7597: [HUDI-5192] Prevent GH actions from running on trivial file changes

2023-01-06 Thread GitBox
nsivabalan merged PR #7597: URL: https://github.com/apache/hudi/pull/7597 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apach

[GitHub] [hudi] jonvex commented on a diff in pull request #7605: [HUDI-5349] Clean up partially failed restore

2023-01-06 Thread GitBox
jonvex commented on code in PR #7605: URL: https://github.com/apache/hudi/pull/7605#discussion_r1063615487 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java: ## @@ -840,11 +841,25 @@ public boolean rollback(final String commitInsta

[GitHub] [hudi] hudi-bot commented on pull request #7618: [MINOR] Polish README and template.md in RFC.

2023-01-06 Thread GitBox
hudi-bot commented on PR #7618: URL: https://github.com/apache/hudi/pull/7618#issuecomment-1373869504 ## CI report: * a8e6918f30422bfbd7f9be740cb05b35609cd9dc Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1414

[GitHub] [hudi] hudi-bot commented on pull request #7614: [HUDI-5509] check if dfs support atomic creation when using filesyste…

2023-01-06 Thread GitBox
hudi-bot commented on PR #7614: URL: https://github.com/apache/hudi/pull/7614#issuecomment-1373869430 ## CI report: * f24cdfcff263e0ccad4f241477997c45b70abe6b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1414

[hudi] branch master updated: [MINOR] fix typo (#7616)

2023-01-06 Thread zhangyue19921010
This is an automated email from the ASF dual-hosted git repository. zhangyue19921010 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new cf20c95f48d [MINOR] fix typo (#7616) cf2

[GitHub] [hudi] zhangyue19921010 merged pull request #7616: [MINOR] fix typo

2023-01-06 Thread GitBox
zhangyue19921010 merged PR #7616: URL: https://github.com/apache/hudi/pull/7616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi

[GitHub] [hudi] hudi-bot commented on pull request #7618: [MINOR] Polish README and template.md in RFC.

2023-01-06 Thread GitBox
hudi-bot commented on PR #7618: URL: https://github.com/apache/hudi/pull/7618#issuecomment-1373768283 ## CI report: * a8e6918f30422bfbd7f9be740cb05b35609cd9dc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] jackwener opened a new pull request, #7618: [MINOR] Polish README and template.md in RFC.

2023-01-06 Thread GitBox
jackwener opened a new pull request, #7618: URL: https://github.com/apache/hudi/pull/7618 ### Change Logs Polish the format of `README.md` and `template.md` in RFC. ### Impact N/A ### Risk level (write none, low medium or high below) none ### Document

[GitHub] [hudi] hudi-bot commented on pull request #7616: [MINOR] fix typo

2023-01-06 Thread GitBox
hudi-bot commented on PR #7616: URL: https://github.com/apache/hudi/pull/7616#issuecomment-1373758095 ## CI report: * 8088f0b69cc12be9ae022be1442ccc3d2500be42 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1414

[GitHub] [hudi] hudi-bot commented on pull request #7616: [MINOR] fix typo

2023-01-06 Thread GitBox
hudi-bot commented on PR #7616: URL: https://github.com/apache/hudi/pull/7616#issuecomment-1373751141 ## CI report: * 8088f0b69cc12be9ae022be1442ccc3d2500be42 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7468: [HUDI-5394] Fix RowCustomColumnsSortPartitioner

2023-01-06 Thread GitBox
hudi-bot commented on PR #7468: URL: https://github.com/apache/hudi/pull/7468#issuecomment-1373750636 ## CI report: * cbd6e7916ba0093e67b28e2bb2d6877e38f33623 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1414

[GitHub] [hudi] hudi-bot commented on pull request #6361: [HUDI-4690][HUDI-4503] Cleaning up Hudi custom Spark `Rule`s

2023-01-06 Thread GitBox
hudi-bot commented on PR #6361: URL: https://github.com/apache/hudi/pull/6361#issuecomment-1373748919 ## CI report: * 04d18204862b8eccdb6f3d78b5b4bc91ad1b4915 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=1413

[GitHub] [hudi] MichaelUryukin opened a new issue, #7617: [SUPPORT] Hudi "write" command doesn't fail when on incompatible partition type, but "read" command fails.

2023-01-06 Thread GitBox
MichaelUryukin opened a new issue, #7617: URL: https://github.com/apache/hudi/issues/7617 **Describe the problem you faced** When we write a DF to a Hudi table, which is partitioned by column of a type "date", and the value of one of the rows for this column is NULL, Hudi will try

[GitHub] [hudi] wuyouwuyoulian opened a new pull request, #7616: [MINOR] fix typo

2023-01-06 Thread GitBox
wuyouwuyoulian opened a new pull request, #7616: URL: https://github.com/apache/hudi/pull/7616 ### Change Logs fix typo ### Impact fix typo ### Risk level (write none, low medium or high below) none ### Documentation Update none ### Cont

[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer

2023-01-06 Thread GitBox
hudi-bot commented on PR #6133: URL: https://github.com/apache/hudi/pull/6133#issuecomment-1373633366 ## CI report: * dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN * 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN * 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN * 0b

[GitHub] [hudi] soumilshah1995 commented on issue #7430: [BUG] MOR Table Hard Deletes Create issue with Athena Querying RT Tables

2023-01-06 Thread GitBox
soumilshah1995 commented on issue #7430: URL: https://github.com/apache/hudi/issues/7430#issuecomment-1373626282 Hi do we have any updates for this Task ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [hudi] soumilshah1995 commented on issue #7591: [SUPPORT] Kinesis Data Analytics Flink1.13 to HUDI

2023-01-06 Thread GitBox
soumilshah1995 commented on issue #7591: URL: https://github.com/apache/hudi/issues/7591#issuecomment-1373625630 Hi danny i do not have logs i am using AWS kinesis managed service for Flink i have provided all necessary steps if needed team can replicate the error. please let me know i

[hudi] branch master updated (f745e645735 -> 3e49e4c26da)

2023-01-06 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from f745e645735 [HUDI-5341] CleanPlanner retains earliest commits must not be later than earliest pending commit (#7568)

  1   2   >