[GitHub] [hudi] codope closed pull request #7492: Fix input format

2022-12-16 Thread GitBox
codope closed pull request #7492: Fix input format URL: https://github.com/apache/hudi/pull/7492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] codope opened a new pull request, #7493: Avoid file index and use fs view cache in COW input format

2022-12-16 Thread GitBox
codope opened a new pull request, #7493: URL: https://github.com/apache/hudi/pull/7493 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[GitHub] [hudi] codope opened a new pull request, #7492: Fix input format

2022-12-16 Thread GitBox
codope opened a new pull request, #7492: URL: https://github.com/apache/hudi/pull/7492 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any performance

[GitHub] [hudi] yihua commented on a diff in pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
yihua commented on code in PR #7488: URL: https://github.com/apache/hudi/pull/7488#discussion_r1051349242 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -291,21 +292,18 @@ object HoodieFileIndex extends Logging {

[hudi] branch master updated: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex (#7488)

2022-12-16 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 7b17e6fa1f [HUDI-5403] Turn off

[GitHub] [hudi] nsivabalan merged pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
nsivabalan merged PR #7488: URL: https://github.com/apache/hudi/pull/7488 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] nsivabalan commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
nsivabalan commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1356103489 CI is green ![Screen Shot 2022-12-16 at 11 48 12 PM](https://user-images.githubusercontent.com/513218/208231697-308c53b3-e872-4f53-b7bf-0fd47eddff03.png) -- This is an automated

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7489: [DNM][Stacked on 7488] Adding global `FileStatusCache` to properly support caching of the file-listing for Hive's `InputFormat` im

2022-12-16 Thread GitBox
nsivabalan commented on code in PR #7489: URL: https://github.com/apache/hudi/pull/7489#discussion_r1051329950 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -291,21 +292,18 @@ object HoodieFileIndex extends Logging {

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
nsivabalan commented on code in PR #7488: URL: https://github.com/apache/hudi/pull/7488#discussion_r1051326658 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -111,6 +111,18 @@ public class HoodieTableMetadataUtil { public static

[GitHub] [hudi] hudi-bot commented on pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2022-12-16 Thread GitBox
hudi-bot commented on PR #7490: URL: https://github.com/apache/hudi/pull/7490#issuecomment-1356070894 ## CI report: * a51141f857e0ddbd6bf7007597299156c6c4d3a5 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7482: [MINOR] Fix the inconsistent behavior to calculate the value count between COW and MOR

2022-12-16 Thread GitBox
hudi-bot commented on PR #7482: URL: https://github.com/apache/hudi/pull/7482#issuecomment-1356070883 ## CI report: * dd5d168ff9a755b24c53762d69d884373f47d017 Azure:

[GitHub] [hudi] boneanxs commented on pull request #7482: [MINOR] Fix the inconsistent behavior to calculate the value count between COW and MOR

2022-12-16 Thread GitBox
boneanxs commented on PR #7482: URL: https://github.com/apache/hudi/pull/7482#issuecomment-1356058703 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hudi] hudi-bot commented on pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2022-12-16 Thread GitBox
hudi-bot commented on PR #7490: URL: https://github.com/apache/hudi/pull/7490#issuecomment-1356015055 ## CI report: * 7f35b24cac747a010918e7bda5ba4b39a5fecdcc Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1356015044 ## CI report: * 46d31413f94276baf450022d7bbcafce9ec6a911 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2022-12-16 Thread GitBox
hudi-bot commented on PR #7490: URL: https://github.com/apache/hudi/pull/7490#issuecomment-1356014183 ## CI report: * 7f35b24cac747a010918e7bda5ba4b39a5fecdcc Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1356014175 ## CI report: * 46d31413f94276baf450022d7bbcafce9ec6a911 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7489: [DNM][Stacked on 7488] Adding global `FileStatusCache` to properly support caching of the file-listing for Hive's `InputFormat` implementation

2022-12-16 Thread GitBox
hudi-bot commented on PR #7489: URL: https://github.com/apache/hudi/pull/7489#issuecomment-1356013385 ## CI report: * 9db66d4ad23a9d6bd9881b3f82f2f02698012a31 UNKNOWN * 2df0100d28efb2ff634d28a980915e81d7297170 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1356013379 ## CI report: * 46d31413f94276baf450022d7bbcafce9ec6a911 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #6782: [HUDI-4911][HUDI-3301] Fixing `HoodieMetadataLogRecordReader` to avoid flushing cache for every lookup

2022-12-16 Thread GitBox
hudi-bot commented on PR #6782: URL: https://github.com/apache/hudi/pull/6782#issuecomment-1356012680 ## CI report: * d4821274f89bb426539c77b2f56afb002a46bf05 Azure:

[GitHub] [hudi] weimingdiit commented on a diff in pull request #7362: [HUDI-5315] The record size is dynamically estimated when the table i…

2022-12-16 Thread GitBox
weimingdiit commented on code in PR #7362: URL: https://github.com/apache/hudi/pull/7362#discussion_r1051303059 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BaseSparkCommitActionExecutor.java: ## @@ -419,4 +427,23 @@ public Partitioner

[GitHub] [hudi] nsivabalan commented on pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2022-12-16 Thread GitBox
nsivabalan commented on PR #7490: URL: https://github.com/apache/hudi/pull/7490#issuecomment-1355976373 @alexeykudinkin : fixed the test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] leesf commented on a diff in pull request #7440: [HUDI-5377] Write call stack information to lock file

2022-12-16 Thread GitBox
leesf commented on code in PR #7440: URL: https://github.com/apache/hudi/pull/7440#discussion_r1051295344 ## hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java: ## @@ -76,6 +76,7 @@ public void lock() { break;

[GitHub] [hudi] hudi-bot commented on pull request #6782: [HUDI-4911][HUDI-3301] Fixing `HoodieMetadataLogRecordReader` to avoid flushing cache for every lookup

2022-12-16 Thread GitBox
hudi-bot commented on PR #6782: URL: https://github.com/apache/hudi/pull/6782#issuecomment-1355973773 ## CI report: * 082f4d91dc7cc794a7f3e38939bf2485657493de Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2022-12-16 Thread GitBox
hudi-bot commented on PR #7490: URL: https://github.com/apache/hudi/pull/7490#issuecomment-1355970380 ## CI report: * 7f35b24cac747a010918e7bda5ba4b39a5fecdcc Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2022-12-16 Thread GitBox
hudi-bot commented on PR #7490: URL: https://github.com/apache/hudi/pull/7490#issuecomment-1355964741 ## CI report: * 7f35b24cac747a010918e7bda5ba4b39a5fecdcc Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2022-12-16 Thread GitBox
hudi-bot commented on PR #7490: URL: https://github.com/apache/hudi/pull/7490#issuecomment-1355958986 ## CI report: * 7f35b24cac747a010918e7bda5ba4b39a5fecdcc UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[jira] [Updated] (HUDI-5408) Partially failed commits in MDT is not rolledback in all cases

2022-12-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5408: - Labels: pull-request-available (was: ) > Partially failed commits in MDT is not rolledback in

[GitHub] [hudi] nsivabalan opened a new pull request, #7491: [HUDI-5408][WIP] Fixing the way we rollback already completed commit in MDT, but failed in DT and re-attempted

2022-12-16 Thread GitBox
nsivabalan opened a new pull request, #7491: URL: https://github.com/apache/hudi/pull/7491 ### Change Logs when compaction failed after completing in MDT but before completing in DT. and later when we re-attempt to apply the same compaction instant to MDT, we might miss to rollback

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2022-12-16 Thread GitBox
nsivabalan commented on code in PR #7490: URL: https://github.com/apache/hudi/pull/7490#discussion_r1051282649 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/functional/TestHoodieBackedMetadata.java: ## @@ -1556,6 +1556,47 @@ public void

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2022-12-16 Thread GitBox
alexeykudinkin commented on code in PR #7490: URL: https://github.com/apache/hudi/pull/7490#discussion_r1051282103 ## hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/functional/TestHoodieBackedMetadata.java: ## @@ -1556,6 +1556,47 @@ public void

[jira] [Created] (HUDI-5408) Partially failed commits in MDT is not rolledback in all cases

2022-12-16 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5408: - Summary: Partially failed commits in MDT is not rolledback in all cases Key: HUDI-5408 URL: https://issues.apache.org/jira/browse/HUDI-5408 Project: Apache

[jira] [Updated] (HUDI-5407) Rollbacks in MDT is not effective

2022-12-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5407: - Labels: pull-request-available (was: ) > Rollbacks in MDT is not effective >

[GitHub] [hudi] nsivabalan opened a new pull request, #7490: [HUDI-5407] Fixing rollback in MDT to be eager

2022-12-16 Thread GitBox
nsivabalan opened a new pull request, #7490: URL: https://github.com/apache/hudi/pull/7490 ### Change Logs Metadata table could deem some invalid data as valid in some rare conditions. Especially when there are partially failed commits in MDT and the commit that failed refers to

[jira] [Updated] (HUDI-5407) Rollbacks in MDT is not effective

2022-12-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5407: -- Priority: Critical (was: Major) > Rollbacks in MDT is not effective >

[jira] [Created] (HUDI-5407) Rollbacks in MDT is not effective

2022-12-16 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5407: - Summary: Rollbacks in MDT is not effective Key: HUDI-5407 URL: https://issues.apache.org/jira/browse/HUDI-5407 Project: Apache Hudi Issue Type:

[jira] [Updated] (HUDI-5407) Rollbacks in MDT is not effective

2022-12-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5407: -- Fix Version/s: 0.12.2 > Rollbacks in MDT is not effective >

[jira] [Assigned] (HUDI-5407) Rollbacks in MDT is not effective

2022-12-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-5407: - Assignee: sivabalan narayanan > Rollbacks in MDT is not effective >

[GitHub] [hudi] hudi-bot commented on pull request #7489: [DNM][Stacked on 7488] Adding global `FileStatusCache` to properly support caching of the file-listing for Hive's `InputFormat` implementation

2022-12-16 Thread GitBox
hudi-bot commented on PR #7489: URL: https://github.com/apache/hudi/pull/7489#issuecomment-1355895040 ## CI report: * 9db66d4ad23a9d6bd9881b3f82f2f02698012a31 UNKNOWN * 3bae08d3d86c63daa262a4b95eec94cdf24fc62b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1355894967 ## CI report: * cc5f2c8f8b58d2c7b3cd6f2ba70f03536547466f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7489: [DNM][Stacked on 7488] Adding global `FileStatusCache` to properly support caching of the file-listing for Hive's `InputFormat` implementation

2022-12-16 Thread GitBox
hudi-bot commented on PR #7489: URL: https://github.com/apache/hudi/pull/7489#issuecomment-1355887004 ## CI report: * 9db66d4ad23a9d6bd9881b3f82f2f02698012a31 UNKNOWN * 3bae08d3d86c63daa262a4b95eec94cdf24fc62b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1355886997 ## CI report: * cc5f2c8f8b58d2c7b3cd6f2ba70f03536547466f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1355885420 ## CI report: * cc5f2c8f8b58d2c7b3cd6f2ba70f03536547466f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7455: [DO_NOT_MERGE] Release 0.12.2 blockers candidate

2022-12-16 Thread GitBox
hudi-bot commented on PR #7455: URL: https://github.com/apache/hudi/pull/7455#issuecomment-1355885365 ## CI report: * 9ad6939e4e9bacfb1a324bab216198a56f410c9d UNKNOWN * d971f3e16869bfed4f1911c4eb7c5c025f3ff8e9 Azure:

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
alexeykudinkin commented on code in PR #7488: URL: https://github.com/apache/hudi/pull/7488#discussion_r1051253486 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -291,21 +292,18 @@ object HoodieFileIndex extends Logging {

[GitHub] [hudi] yihua commented on a diff in pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
yihua commented on code in PR #7488: URL: https://github.com/apache/hudi/pull/7488#discussion_r1051252788 ## hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileIndex.scala: ## @@ -291,21 +292,18 @@ object HoodieFileIndex extends Logging {

[GitHub] [hudi] yihua commented on a diff in pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
yihua commented on code in PR #7488: URL: https://github.com/apache/hudi/pull/7488#discussion_r1051216097 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -171,23 +183,23 @@ class ColumnStats { Collectors.toMap(colRangeMetadata

[GitHub] [hudi] hudi-bot commented on pull request #7489: [DNM][Stacked on 7488] Adding global `FileStatusCache` to properly support caching of the file-listing for Hive's `InputFormat` implementation

2022-12-16 Thread GitBox
hudi-bot commented on PR #7489: URL: https://github.com/apache/hudi/pull/7489#issuecomment-1355818994 ## CI report: * 9db66d4ad23a9d6bd9881b3f82f2f02698012a31 UNKNOWN * 3bae08d3d86c63daa262a4b95eec94cdf24fc62b Azure:

[jira] [Created] (HUDI-5406) Troubleshoot NoClassDefFoundError in Integration Tests logs

2022-12-16 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-5406: - Summary: Troubleshoot NoClassDefFoundError in Integration Tests logs Key: HUDI-5406 URL: https://issues.apache.org/jira/browse/HUDI-5406 Project: Apache Hudi

[hudi] branch master updated (956d20ba2c -> 834a0becd8)

2022-12-16 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from 956d20ba2c [HUDI-5104] Add feature flag to disable HoodieFileIndex and fall back to HoodieROTablePathFilter (#7088)

[GitHub] [hudi] nsivabalan merged pull request #7419: [HUDI-5357] Optimize deployment of release artifacts

2022-12-16 Thread GitBox
nsivabalan merged PR #7419: URL: https://github.com/apache/hudi/pull/7419 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] alexeykudinkin commented on pull request #7413: [HUDI-5321] Fix inconsistencies in arePartitionRecordsSorted and try to limit lots of small files during bulk insert

2022-12-16 Thread GitBox
alexeykudinkin commented on PR #7413: URL: https://github.com/apache/hudi/pull/7413#issuecomment-1355715534 Overall, LGTM! Let's troubleshoot CI failures, and we should be good to go after addressing last round of comments -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7413: [HUDI-5321] Fix inconsistencies in arePartitionRecordsSorted and try to limit lots of small files during bulk insert

2022-12-16 Thread GitBox
alexeykudinkin commented on code in PR #7413: URL: https://github.com/apache/hudi/pull/7413#discussion_r1051199250 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/SpatialCurveSortPartitionerBase.java: ## @@ -78,6 +78,8 @@ protected Dataset

[jira] [Created] (HUDI-5405) Avoid using Projections in generic Merge Into DMLs

2022-12-16 Thread Alexey Kudinkin (Jira)
Alexey Kudinkin created HUDI-5405: - Summary: Avoid using Projections in generic Merge Into DMLs Key: HUDI-5405 URL: https://issues.apache.org/jira/browse/HUDI-5405 Project: Apache Hudi Issue

[GitHub] [hudi] hudi-bot commented on pull request #7455: [DO_NOT_MERGE] Release 0.12.2 blockers candidate

2022-12-16 Thread GitBox
hudi-bot commented on PR #7455: URL: https://github.com/apache/hudi/pull/7455#issuecomment-1355591449 ## CI report: * c90b7fa833230abcc0c9e49923b7ae34f34d1b2b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7455: [DO_NOT_MERGE] Release 0.12.2 blockers candidate

2022-12-16 Thread GitBox
hudi-bot commented on PR #7455: URL: https://github.com/apache/hudi/pull/7455#issuecomment-1355585308 ## CI report: * c90b7fa833230abcc0c9e49923b7ae34f34d1b2b Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7455: [DO_NOT_MERGE] Release 0.12.2 blockers candidate

2022-12-16 Thread GitBox
hudi-bot commented on PR #7455: URL: https://github.com/apache/hudi/pull/7455#issuecomment-1355573728 ## CI report: * 1061de2fce4d65b45af4559b2e84cb8ca84f0b12 Azure:

[hudi] branch release-0.12.2-blockers-candidate updated (9ad6939e4e -> d971f3e168)

2022-12-16 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch release-0.12.2-blockers-candidate in repository https://gitbox.apache.org/repos/asf/hudi.git omit 9ad6939e4e [HUDI-5104] Add feature flag to disable HoodieFileIndex and fall back to

[hudi] branch release-0.12.2-blockers-candidate updated (c90b7fa833 -> 9ad6939e4e)

2022-12-16 Thread yihua
This is an automated email from the ASF dual-hosted git repository. yihua pushed a change to branch release-0.12.2-blockers-candidate in repository https://gitbox.apache.org/repos/asf/hudi.git omit c90b7fa833 [HUDI-5104] Add feature flag to disable HoodieFileIndex and fall back to

[GitHub] [hudi] hudi-bot commented on pull request #7489: [DNM][Stacked on 7488] Adding global `FileStatusCache` to properly support caching of the file-listing for Hive's `InputFormat` implementation

2022-12-16 Thread GitBox
hudi-bot commented on PR #7489: URL: https://github.com/apache/hudi/pull/7489#issuecomment-1355489102 ## CI report: * 9db66d4ad23a9d6bd9881b3f82f2f02698012a31 UNKNOWN * 3bae08d3d86c63daa262a4b95eec94cdf24fc62b Azure:

[GitHub] [hudi] hemanth-gowda-12 commented on a diff in pull request #7474: [HUDI-5246] Added validation for Partition Path to not begin with "/"

2022-12-16 Thread GitBox
hemanth-gowda-12 commented on code in PR #7474: URL: https://github.com/apache/hudi/pull/7474#discussion_r1051085879 ## hudi-common/src/main/java/org/apache/hudi/common/model/HoodieKey.java: ## @@ -53,6 +53,9 @@ public void setPartitionPath(String partitionPath) { }

[GitHub] [hudi] hudi-bot commented on pull request #7489: [DNM][Stacked on 7488] Adding global `FileStatusCache` to properly support caching of the file-listing for Hive's `InputFormat` implementation

2022-12-16 Thread GitBox
hudi-bot commented on PR #7489: URL: https://github.com/apache/hudi/pull/7489#issuecomment-1355477255 ## CI report: * 9db66d4ad23a9d6bd9881b3f82f2f02698012a31 UNKNOWN * 3bae08d3d86c63daa262a4b95eec94cdf24fc62b UNKNOWN Bot commands @hudi-bot supports the

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1355477115 ## CI report: * fc2238864633cf6c48ffb25cd8bcdbf8908202fb Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7455: [DO_NOT_MERGE] Release 0.12.2 blockers candidate

2022-12-16 Thread GitBox
hudi-bot commented on PR #7455: URL: https://github.com/apache/hudi/pull/7455#issuecomment-1355476571 ## CI report: * 1061de2fce4d65b45af4559b2e84cb8ca84f0b12 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7489: [DNM][Stacked on 7488] Adding global `FileStatusCache` to properly support caching of the file-listing for Hive's `InputFormat` implementation

2022-12-16 Thread GitBox
hudi-bot commented on PR #7489: URL: https://github.com/apache/hudi/pull/7489#issuecomment-1355465677 ## CI report: * 9db66d4ad23a9d6bd9881b3f82f2f02698012a31 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7455: [DO_NOT_MERGE] Release 0.12.2 blockers candidate

2022-12-16 Thread GitBox
hudi-bot commented on PR #7455: URL: https://github.com/apache/hudi/pull/7455#issuecomment-1355465205 ## CI report: * b50dc32bcf4fe961c3d0a028e00547864eaa2253 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1355465520 ## CI report: * 874fe46d0e260961ff7d2b707cb7c6c1b9b97cad Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7486: [HUDI-4827] Rebase Azure Image on Ubuntu 22.04 - xbean fix

2022-12-16 Thread GitBox
hudi-bot commented on PR #7486: URL: https://github.com/apache/hudi/pull/7486#issuecomment-1355454300 ## CI report: * ac0512db2dfa414960edf35f63933b365e6b27e4 Azure:

[hudi] branch release-0.12.2-blockers-candidate updated (1061de2fce -> c90b7fa833)

2022-12-16 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a change to branch release-0.12.2-blockers-candidate in repository https://gitbox.apache.org/repos/asf/hudi.git from 1061de2fce [HUDI-3661] Flink async compaction is not thread safe when use watermark (#7399)

[GitHub] [hudi] alexeykudinkin opened a new pull request, #7489: [DNM] Adding global `FileStatusCache` to properly support caching of the file-listing for Hive

2022-12-16 Thread GitBox
alexeykudinkin opened a new pull request, #7489: URL: https://github.com/apache/hudi/pull/7489 ### Change Logs _Describe context and summary for this change. Highlight if any code was copied._ ### Impact _Describe any public API or user-facing feature change or any

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1355325754 ## CI report: * 874fe46d0e260961ff7d2b707cb7c6c1b9b97cad Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7455: [DO_NOT_MERGE] Release 0.12.2 blockers candidate

2022-12-16 Thread GitBox
hudi-bot commented on PR #7455: URL: https://github.com/apache/hudi/pull/7455#issuecomment-1355325504 ## CI report: * b50dc32bcf4fe961c3d0a028e00547864eaa2253 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1355314516 ## CI report: * 874fe46d0e260961ff7d2b707cb7c6c1b9b97cad Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7485: [MINOR] Fix flaky testStructuredStreamingWithCompaction

2022-12-16 Thread GitBox
hudi-bot commented on PR #7485: URL: https://github.com/apache/hudi/pull/7485#issuecomment-1355314436 ## CI report: * 6a21a3a856b435b8eb1dfe2818593da2c679d59f Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7455: [DO_NOT_MERGE] Release 0.12.2 blockers candidate

2022-12-16 Thread GitBox
hudi-bot commented on PR #7455: URL: https://github.com/apache/hudi/pull/7455#issuecomment-1355314281 ## CI report: * b50dc32bcf4fe961c3d0a028e00547864eaa2253 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1355303252 ## CI report: * 874fe46d0e260961ff7d2b707cb7c6c1b9b97cad Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7455: [DO_NOT_MERGE] Release 0.12.2 blockers candidate

2022-12-16 Thread GitBox
hudi-bot commented on PR #7455: URL: https://github.com/apache/hudi/pull/7455#issuecomment-1355302840 ## CI report: * b50dc32bcf4fe961c3d0a028e00547864eaa2253 Azure:

[GitHub] [hudi] codope commented on a diff in pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
codope commented on code in PR #7488: URL: https://github.com/apache/hudi/pull/7488#discussion_r1051006383 ## hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java: ## @@ -171,23 +183,23 @@ class ColumnStats { Collectors.toMap(colRangeMetadata

[hudi] branch master updated: [HUDI-5104] Add feature flag to disable HoodieFileIndex and fall back to HoodieROTablePathFilter (#7088)

2022-12-16 Thread codope
This is an automated email from the ASF dual-hosted git repository. codope pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 956d20ba2c [HUDI-5104] Add feature flag to

[GitHub] [hudi] codope merged pull request #7088: [HUDI-5104] Add feature flag to disable HoodieFileIndex and fall back to HoodieROTablePathFilter

2022-12-16 Thread GitBox
codope merged PR #7088: URL: https://github.com/apache/hudi/pull/7088 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] xushiyan commented on a diff in pull request #7419: [HUDI-5357] Optimize deployment of release artifacts

2022-12-16 Thread GitBox
xushiyan commented on code in PR #7419: URL: https://github.com/apache/hudi/pull/7419#discussion_r1050980151 ## scripts/release/deploy_staging_jars.sh: ## @@ -36,16 +36,37 @@ if [ "$#" -gt "1" ]; then exit 1 fi +BUNDLE_MODULES=$(find -s packaging -name 'hudi-*-bundle'

[GitHub] [hudi] xushiyan commented on a diff in pull request #7419: [HUDI-5357] Optimize deployment of release artifacts

2022-12-16 Thread GitBox
xushiyan commented on code in PR #7419: URL: https://github.com/apache/hudi/pull/7419#discussion_r1050979466 ## scripts/release/deploy_staging_jars.sh: ## @@ -36,16 +36,37 @@ if [ "$#" -gt "1" ]; then exit 1 fi +BUNDLE_MODULES=$(find -s packaging -name 'hudi-*-bundle'

[jira] [Created] (HUDI-5404) add flink bundle validation

2022-12-16 Thread Raymond Xu (Jira)
Raymond Xu created HUDI-5404: Summary: add flink bundle validation Key: HUDI-5404 URL: https://issues.apache.org/jira/browse/HUDI-5404 Project: Apache Hudi Issue Type: Test Components:

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
nsivabalan commented on code in PR #7488: URL: https://github.com/apache/hudi/pull/7488#discussion_r1050971882 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieCopyOnWriteTableInputFormat.java: ## @@ -246,18 +258,38 @@ private List listStatusForSnapshotMode(JobConf

[GitHub] [hudi] nsivabalan commented on a diff in pull request #7419: [HUDI-5357] Optimize deployment of release artifacts

2022-12-16 Thread GitBox
nsivabalan commented on code in PR #7419: URL: https://github.com/apache/hudi/pull/7419#discussion_r1050960528 ## scripts/release/deploy_staging_jars.sh: ## @@ -36,16 +36,37 @@ if [ "$#" -gt "1" ]; then exit 1 fi +BUNDLE_MODULES=$(find -s packaging -name 'hudi-*-bundle'

[hudi] branch release-0.12.2-blockers-candidate updated (b50dc32bcf -> 1061de2fce)

2022-12-16 Thread sivabalan
This is an automated email from the ASF dual-hosted git repository. sivabalan pushed a change to branch release-0.12.2-blockers-candidate in repository https://gitbox.apache.org/repos/asf/hudi.git from b50dc32bcf [HUDI-5366] Closing metadata writer from within writeClient (0.12.2) (#7464)

[GitHub] [hudi] nsivabalan merged pull request #7465: [HUDI-3661] Flink async compaction is not thread safe when use waterm…

2022-12-16 Thread GitBox
nsivabalan merged PR #7465: URL: https://github.com/apache/hudi/pull/7465 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] nsivabalan commented on pull request #7465: [HUDI-3661] Flink async compaction is not thread safe when use waterm…

2022-12-16 Thread GitBox
nsivabalan commented on PR #7465: URL: https://github.com/apache/hudi/pull/7465#issuecomment-1355208005 CI is green ![Screen Shot 2022-12-16 at 8 48 44 AM](https://user-images.githubusercontent.com/513218/208148089-07758f5e-dbce-43a8-8d8e-87d687bd88aa.png) -- This is an automated

[GitHub] [hudi] hudi-bot commented on pull request #7484: [HUDI-4827] Rebase Azure Image on Ubuntu 22.04 - dependency 1.8

2022-12-16 Thread GitBox
hudi-bot commented on PR #7484: URL: https://github.com/apache/hudi/pull/7484#issuecomment-1355173170 ## CI report: * e75b3464f4668d5e22a9c5c0f8c853dd20e99810 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7483: [HUDI-5402] Fix flink job wait ckpMeta timeout when partial failed

2022-12-16 Thread GitBox
hudi-bot commented on PR #7483: URL: https://github.com/apache/hudi/pull/7483#issuecomment-1355173062 ## CI report: * a42f62618de1379b5165e3f3fc99d3774585bba9 Azure:

[GitHub] [hudi] xushiyan commented on a diff in pull request #7419: [WIP][HUDI-5357] Optimize deployment of release artifacts

2022-12-16 Thread GitBox
xushiyan commented on code in PR #7419: URL: https://github.com/apache/hudi/pull/7419#discussion_r1050932359 ## scripts/release/deploy_staging_jars.sh: ## @@ -37,15 +37,26 @@ if [ "$#" -gt "1" ]; then fi declare -a ALL_VERSION_OPTS=( -"-Dscala-2.11 -Dspark2 -Dflink1.13" #

[GitHub] [hudi] hudi-bot commented on pull request #7480: [HUDI-5400] Fix read issues when Hudi-FULL schema evolution is not enabled

2022-12-16 Thread GitBox
hudi-bot commented on PR #7480: URL: https://github.com/apache/hudi/pull/7480#issuecomment-1355053313 ## CI report: * 90722aa7ae44570b58ef6d6e673a1f39429acf39 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1355039884 ## CI report: * 695d492a7dec65bb8ba9b954d3b5d370f20bfbe0 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7488: [HUDI-5403] Turn off metadata-table-based file listing in BaseHoodieTableFileIndex

2022-12-16 Thread GitBox
hudi-bot commented on PR #7488: URL: https://github.com/apache/hudi/pull/7488#issuecomment-1355029902 ## CI report: * 695d492a7dec65bb8ba9b954d3b5d370f20bfbe0 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7456: [HUDI-4917][FOLLOW_UP]Optimize codes logic to not break the old class meaning

2022-12-16 Thread GitBox
hudi-bot commented on PR #7456: URL: https://github.com/apache/hudi/pull/7456#issuecomment-1355017741 ## CI report: * c4041c7446cd25b9809f616b640c069ef2959107 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7482: [MINOR] Fix the inconsistent behavior to calculate the value count between COW and MOR

2022-12-16 Thread GitBox
hudi-bot commented on PR #7482: URL: https://github.com/apache/hudi/pull/7482#issuecomment-1354830995 ## CI report: * dd5d168ff9a755b24c53762d69d884373f47d017 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #5913: [HUDI-4287] Optimize Flink checkpoint meta mechanism to fix mistaken pending instants

2022-12-16 Thread GitBox
hudi-bot commented on PR #5913: URL: https://github.com/apache/hudi/pull/5913#issuecomment-1354773505 ## CI report: * 66cadb8c96370af7b18c8ebb641f304c48dfd5bb Azure:

[GitHub] [hudi] soumilshah1995 commented on issue #7430: [BUG] MOR Table Hard Deletes Create issue with Athena Querying RT Tables

2022-12-16 Thread GitBox
soumilshah1995 commented on issue #7430: URL: https://github.com/apache/hudi/issues/7430#issuecomment-1354745922 Any Updates on this issue ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] trushev commented on pull request #5830: [HUDI-3981] Flink engine support for comprehensive schema evolution

2022-12-16 Thread GitBox
trushev commented on PR #5830: URL: https://github.com/apache/hudi/pull/5830#issuecomment-1354714159 @voonhous As I understand you are talking about this case ```sql Flink SQL> -- write with schema1 create table tbl(`id` int primary key, `value` int) partitioned by

[GitHub] [hudi] Zouxxyy commented on a diff in pull request #7468: [HUDI-5394] Fix RowCustomColumnsSortPartitioner

2022-12-16 Thread GitBox
Zouxxyy commented on code in PR #7468: URL: https://github.com/apache/hudi/pull/7468#discussion_r1050721238 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/RowCustomColumnsSortPartitioner.java: ## @@ -44,8 +44,8 @@ public

[jira] [Commented] (HUDI-1973) Support SCD2 types in Hudi tables

2022-12-16 Thread Purushotham Pushpavanthar (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648600#comment-17648600 ] Purushotham Pushpavanthar commented on HUDI-1973: - Hi, Is there any plan to support this

  1   2   >