[GitHub] [hudi] lokeshj1703 commented on pull request #7521: [HUDI-4827] Rebase Azure Image on Ubuntu 22.04 - scalatest-maven-plugin version update

2022-12-20 Thread GitBox
lokeshj1703 commented on PR #7521: URL: https://github.com/apache/hudi/pull/7521#issuecomment-1360945659 Related issue in scalatest-maven-plugin - https://github.com/scalatest/scalatest-maven-plugin/pull/43 -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] hudi-bot commented on pull request #7528: [HUDI-5443] Fixing exception trying to read MOR table after NestedSchemaPruning rule has been applied

2022-12-20 Thread GitBox
hudi-bot commented on PR #7528: URL: https://github.com/apache/hudi/pull/7528#issuecomment-1360933260 ## CI report: * f3a439884f90500e29da0075f4d0ad7d73a484b3 UNKNOWN * 91a60af68934fce696d23ace1db23d652a5bb109 UNKNOWN Bot commands @hudi-bot supports the

[GitHub] [hudi] hudi-bot commented on pull request #7423: [HUDI-5384] Adding optimization rule to appropriately push down filters into the `HoodieFileIndex`

2022-12-20 Thread GitBox
hudi-bot commented on PR #7423: URL: https://github.com/apache/hudi/pull/7423#issuecomment-1360932876 ## CI report: * 09b901a56869b8282c92d6c05ad746f98f2d6a01 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7528: [HUDI-5443] Fixing exception trying to read MOR table after NestedSchemaPruning rule has been applied

2022-12-20 Thread GitBox
hudi-bot commented on PR #7528: URL: https://github.com/apache/hudi/pull/7528#issuecomment-1360927504 ## CI report: * f3a439884f90500e29da0075f4d0ad7d73a484b3 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7527: [HUDI-5411] Avoid virtual key info for COW table in the input format

2022-12-20 Thread GitBox
hudi-bot commented on PR #7527: URL: https://github.com/apache/hudi/pull/7527#issuecomment-1360927478 ## CI report: * ed2f76f0edfad0ac2175da67a56825ee31a4dd4c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] hudi-bot commented on pull request #7423: [HUDI-5384] Adding optimization rule to appropriately push down filters into the `HoodieFileIndex`

2022-12-20 Thread GitBox
hudi-bot commented on PR #7423: URL: https://github.com/apache/hudi/pull/7423#issuecomment-1360927203 ## CI report: * 09b901a56869b8282c92d6c05ad746f98f2d6a01 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #7526: Revert "[HUDI-5409] Avoid file index and use fs view cache in COW input format (#7493)"

2022-12-20 Thread GitBox
hudi-bot commented on PR #7526: URL: https://github.com/apache/hudi/pull/7526#issuecomment-1360923199 ## CI report: * be61fe2207203761b46a40fa32be8ccd2ad6f12c UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the

[GitHub] [hudi] xccui commented on issue #7375: [SUPPORT] Hudi 0.12.1 support for Spark Structured Streaming. read clustering metadata replace avro file error. Unrecognized token 'Obj^A^B^Vavro'

2022-12-20 Thread GitBox
xccui commented on issue #7375: URL: https://github.com/apache/hudi/issues/7375#issuecomment-1360916357 Built with the latest version but still encountered the same issue (with Flink r/w). -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7528: [HUDI-5443] Fixing exception trying to read MOR table after NestedSchemaPruning rule has been applied

2022-12-20 Thread GitBox
alexeykudinkin commented on code in PR #7528: URL: https://github.com/apache/hudi/pull/7528#discussion_r1054026446 ## hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/common/HoodieSparkEngineContext.java: ## @@ -57,15 +57,15 @@ public class

[jira] [Updated] (HUDI-5443) Fix exception when querying MOR table after applying NestedSchemaPruning optimization

2022-12-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5443: - Labels: pull-request-available (was: ) > Fix exception when querying MOR table after applying

[GitHub] [hudi] alexeykudinkin opened a new pull request, #7528: [HUDI-5443] Fixing exception trying to read MOR table after NestedSchemaPruning rule has been applied

2022-12-20 Thread GitBox
alexeykudinkin opened a new pull request, #7528: URL: https://github.com/apache/hudi/pull/7528 ### Change Logs Currently MOR tables w/ `NestedSchemaPruning` rule successfully applied (ie being able to prune nested schema) would fail to read in case any log-file merging would occur.

[GitHub] [hudi] zhangyue19921010 commented on pull request #7519: [HUDI-5422] Control KEEP_LATEST_VERSIONS clean replaced files immediately or delete after a while

2022-12-20 Thread GitBox
zhangyue19921010 commented on PR #7519: URL: https://github.com/apache/hudi/pull/7519#issuecomment-1360909416 > I guess this PR is related with https://github.com/apache/hudi/pull/7405/files, if the clsutering metadata files are archived but the replaced files are not cleaned, the query

[jira] [Updated] (HUDI-5443) Fix exception when querying MOR table after applying NestedSchemaPruning optimization

2022-12-20 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5443: -- Status: In Progress (was: Open) > Fix exception when querying MOR table after applying

[jira] [Created] (HUDI-5444) FileNotFound issue w/ metadata enabled

2022-12-20 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5444: - Summary: FileNotFound issue w/ metadata enabled Key: HUDI-5444 URL: https://issues.apache.org/jira/browse/HUDI-5444 Project: Apache Hudi Issue

[jira] [Updated] (HUDI-5444) FileNotFound issue w/ metadata enabled

2022-12-20 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5444: -- Priority: Blocker (was: Major) > FileNotFound issue w/ metadata enabled >

[jira] [Assigned] (HUDI-5444) FileNotFound issue w/ metadata enabled

2022-12-20 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-5444: - Assignee: sivabalan narayanan > FileNotFound issue w/ metadata enabled >

[jira] [Updated] (HUDI-5444) FileNotFound issue w/ metadata enabled

2022-12-20 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5444: -- Sprint: 0.13.0 Final Sprint > FileNotFound issue w/ metadata enabled >

[jira] [Updated] (HUDI-5444) FileNotFound issue w/ metadata enabled

2022-12-20 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5444: -- Fix Version/s: 0.13.0 > FileNotFound issue w/ metadata enabled >

[GitHub] [hudi] codope commented on a diff in pull request #7527: [HUDI-5411] Avoid virtual key info for COW table in the input format

2022-12-20 Thread GitBox
codope commented on code in PR #7527: URL: https://github.com/apache/hudi/pull/7527#discussion_r1054022432 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieCopyOnWriteTableInputFormat.java: ## @@ -247,81 +239,33 @@ private List listStatusForSnapshotMode(JobConf

[GitHub] [hudi] shengchiqu commented on issue #7507: [SUPPORT] how to use flink offline with occ

2022-12-20 Thread GitBox
shengchiqu commented on issue #7507: URL: https://github.com/apache/hudi/issues/7507#issuecomment-1360897712 > There is no need OCC here for offline compaction, all you need to do is the start the `HoodieFlinkCompactor` app. @danny0405 thanks, the flink sql set 'metadata.enabled' =

[GitHub] [hudi] danny0405 commented on pull request #7519: [HUDI-5422] Control KEEP_LATEST_VERSIONS clean replaced files immediately or delete after a while

2022-12-20 Thread GitBox
danny0405 commented on PR #7519: URL: https://github.com/apache/hudi/pull/7519#issuecomment-1360897525 I guess this PR is related with https://github.com/apache/hudi/pull/7405/files, if the clsutering metadata files are archived but the replaced files are not cleaned, the query would see

[jira] [Updated] (HUDI-5411) Make sure Trino does not re-instantiates Hive's InputFormat for every partition during file listing

2022-12-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-5411: - Labels: pull-request-available (was: ) > Make sure Trino does not re-instantiates Hive's

[GitHub] [hudi] codope opened a new pull request, #7527: [HUDI-5411] Avoid virtual key info for COW table in the input format

2022-12-20 Thread GitBox
codope opened a new pull request, #7527: URL: https://github.com/apache/hudi/pull/7527 ### Change Logs Fetching virtual key involved reading from commit metadata or data file (`TableSchemaResolver`) which is a costly operation. This is only needed for schema projection in the case

[jira] [Updated] (HUDI-5022) Add better error messages to pr compliance

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5022: - Fix Version/s: 0.13.0 (was: 0.12.2) > Add better error messages to pr compliance >

[jira] [Updated] (HUDI-4970) hudi-kafka-connect-bundle: Could not initialize class org.apache.hadoop.security.UserGroupInformation

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4970: - Fix Version/s: 0.13.0 (was: 0.12.2) > hudi-kafka-connect-bundle: Could not initialize

[jira] [Updated] (HUDI-5285) Exclude hive-site.xml from packaging in hudi-utilities

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5285: - Fix Version/s: 0.13.0 (was: 0.12.2) > Exclude hive-site.xml from packaging in

[GitHub] [hudi] danny0405 commented on issue #7507: [SUPPORT] how to use flink offline with occ

2022-12-20 Thread GitBox
danny0405 commented on issue #7507: URL: https://github.com/apache/hudi/issues/7507#issuecomment-1360882363 There is no need OCC here for offline compaction, all you need to do is the start the `HoodieFlinkCompactor` app. -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] codope opened a new pull request, #7526: Revert "[HUDI-5409] Avoid file index and use fs view cache in COW input format (#7493)"

2022-12-20 Thread GitBox
codope opened a new pull request, #7526: URL: https://github.com/apache/hudi/pull/7526 ### Change Logs This reverts commit cc1c1e7b33d9c95e5a2ba0e9a1db428d1e1b2a00. ### Impact Impacts performance of query engines, such as Trino-Hive connector, that depend on input

[jira] [Updated] (HUDI-4963) Extend InProcessLockProvider to support multiple table ingestion

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4963: - Fix Version/s: 0.13.0 (was: 0.12.2) > Extend InProcessLockProvider to support multiple

[jira] [Updated] (HUDI-5404) add flink bundle validation

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5404: - Fix Version/s: 0.13.0 (was: 0.12.2) > add flink bundle validation >

[jira] [Updated] (HUDI-4605) Upgrade hudi-presto-bundle version to 0.12.0

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4605: - Fix Version/s: 0.13.0 (was: 0.12.2) > Upgrade hudi-presto-bundle version to 0.12.0 >

[jira] [Updated] (HUDI-5145) Remove HDFS from DeltaStreamer UT/FT

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5145: - Fix Version/s: 0.13.0 (was: 0.12.2) > Remove HDFS from DeltaStreamer UT/FT >

[jira] [Updated] (HUDI-5131) Bundle validation: upgrade/downgrade

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5131: - Fix Version/s: 0.13.0 (was: 0.12.2) > Bundle validation: upgrade/downgrade >

[jira] [Updated] (HUDI-5132) Bundle validation: Hive QL 3

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5132: - Fix Version/s: 0.13.0 (was: 0.12.2) > Bundle validation: Hive QL 3 >

[jira] [Updated] (HUDI-5371) Fix flaky testMetadataColumnStatsIndex

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5371: - Fix Version/s: 0.13.0 (was: 0.12.2) > Fix flaky testMetadataColumnStatsIndex >

[jira] [Updated] (HUDI-5099) Update stock data so that new records are added in batch_2

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5099: - Fix Version/s: 0.13.0 (was: 0.12.2) > Update stock data so that new records are added in

[jira] [Updated] (HUDI-5200) Resources are not cleaned up in UT

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5200: - Fix Version/s: 0.13.0 (was: 0.12.2) > Resources are not cleaned up in UT >

[jira] [Updated] (HUDI-4209) Avoid using HDFS in HoodieClientTestHarness

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4209: - Fix Version/s: 0.13.0 (was: 0.12.2) > Avoid using HDFS in HoodieClientTestHarness >

[jira] [Updated] (HUDI-4982) Make bundle combination testing covered in CI

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4982: - Fix Version/s: 0.13.0 (was: 0.12.2) > Make bundle combination testing covered in CI >

[jira] [Updated] (HUDI-5098) Enable Spark2.4 bundle testing in GH Actions

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5098: - Fix Version/s: 0.13.0 (was: 0.12.2) > Enable Spark2.4 bundle testing in GH Actions >

[jira] [Updated] (HUDI-2673) Add integration/e2e test for kafka-connect functionality

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-2673: - Fix Version/s: 0.13.0 (was: 0.12.2) > Add integration/e2e test for kafka-connect

[jira] [Updated] (HUDI-5358) Fix flaky tests in TestCleanerInsertAndCleanByCommits

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5358: - Fix Version/s: 0.13.0 (was: 0.12.2) > Fix flaky tests in

[jira] [Updated] (HUDI-5330) Add docs for virtual keys

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5330: - Fix Version/s: 0.13.0 (was: 0.12.2) > Add docs for virtual keys >

[jira] [Updated] (HUDI-5339) Update docs regarding the behavior change in NONE sort mode for bulk insert

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5339: - Fix Version/s: 0.13.0 (was: 0.12.2) > Update docs regarding the behavior change in NONE

[jira] [Updated] (HUDI-5295) With multiple meta syncs, one meta sync failure should not impact other meta syncs.

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5295: - Fix Version/s: 0.13.0 (was: 0.12.2) > With multiple meta syncs, one meta sync failure

[jira] [Updated] (HUDI-5343) HoodieFlinkStreamer supports async clustering for append mode

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5343: - Fix Version/s: 0.13.0 (was: 0.12.2) > HoodieFlinkStreamer supports async clustering for

[jira] [Updated] (HUDI-5292) Exclude the test resources from every module packaging

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5292: - Fix Version/s: 0.13.0 (was: 0.12.2) > Exclude the test resources from every module

[jira] [Updated] (HUDI-5294) Support type change for schema on read enable + reconcile schema

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5294: - Fix Version/s: 0.13.0 (was: 0.12.2) > Support type change for schema on read enable +

[jira] [Updated] (HUDI-5283) Replace deprecated method Schema.parse with Schema.Parser

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5283: - Fix Version/s: (was: 0.12.2) > Replace deprecated method Schema.parse with Schema.Parser >

[jira] [Updated] (HUDI-5293) Schema on read + reconcile schema fails w/ 0.12.1

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5293: - Fix Version/s: 0.13.0 (was: 0.12.2) > Schema on read + reconcile schema fails w/ 0.12.1 >

[jira] [Updated] (HUDI-5258) Address checkstyle warnings in hudi-common module

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5258: - Fix Version/s: 0.13.0 (was: 0.12.2) > Address checkstyle warnings in hudi-common module >

[jira] [Updated] (HUDI-5261) Use proper parallelism for engine context APIs

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5261: - Fix Version/s: 0.13.0 (was: 0.12.2) > Use proper parallelism for engine context APIs >

[jira] [Updated] (HUDI-5269) Enhancing core user flow tests for spark-sql writes

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5269: - Fix Version/s: 0.13.0 (was: 0.12.2) > Enhancing core user flow tests for spark-sql writes

[jira] [Updated] (HUDI-5252) ClusteringCommitSink supports to rollback clustering

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5252: - Fix Version/s: (was: 0.12.2) > ClusteringCommitSink supports to rollback clustering >

[jira] [Updated] (HUDI-5246) Improve validation for partition path

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5246: - Fix Version/s: (was: 0.12.2) > Improve validation for partition path > -

[jira] [Updated] (HUDI-5241) Optimize HoodieDefaultTimeline API

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5241: - Fix Version/s: 0.13.0 (was: 0.12.2) > Optimize HoodieDefaultTimeline API >

[jira] [Updated] (HUDI-5246) Improve validation for partition path

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5246: - Fix Version/s: 0.13.0 > Improve validation for partition path > - > >

[jira] [Updated] (HUDI-5198) add in minor perf wins in hudi-utilities and locking related tests

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5198: - Fix Version/s: 0.13.0 (was: 0.12.2) > add in minor perf wins in hudi-utilities and locking

[jira] [Updated] (HUDI-5234) Streaming read skip clustering instants Configurable

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5234: - Fix Version/s: (was: 0.12.2) > Streaming read skip clustering instants Configurable >

[jira] [Updated] (HUDI-5167) Reduce test run time for virtual key tests

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5167: - Fix Version/s: 0.13.0 (was: 0.12.2) > Reduce test run time for virtual key tests >

[jira] [Updated] (HUDI-5181) Enhance keygen class validation

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5181: - Fix Version/s: 0.13.0 (was: 0.12.2) > Enhance keygen class validation >

[jira] [Updated] (HUDI-5166) Reduce test run time for top time consuming tests

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5166: - Fix Version/s: 0.13.0 (was: 0.12.2) > Reduce test run time for top time consuming tests >

[jira] [Updated] (HUDI-5178) Add Call show_table_properties for spark sql

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5178: - Fix Version/s: 0.13.0 (was: 0.12.2) > Add Call show_table_properties for spark sql >

[jira] [Updated] (HUDI-5162) Allow user specified start offset for streaming query

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5162: - Fix Version/s: 0.13.0 (was: 0.12.2) > Allow user specified start offset for streaming

[jira] [Updated] (HUDI-5112) Add presto query validation support for all tests in integ tests

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5112: - Fix Version/s: 0.13.0 (was: 0.12.2) > Add presto query validation support for all tests in

[jira] [Updated] (HUDI-5113) Add support to test different indexes with integ test

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5113: - Fix Version/s: 0.13.0 (was: 0.12.2) > Add support to test different indexes with integ

[jira] [Updated] (HUDI-5060) Make all clean policies support incremental mode to find partition paths

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5060: - Fix Version/s: (was: 0.12.2) > Make all clean policies support incremental mode to find partition paths >

[jira] [Updated] (HUDI-5072) Extract transform duplicate code

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5072: - Fix Version/s: 0.13.0 (was: 0.12.2) > Extract transform duplicate code >

[jira] [Updated] (HUDI-5052) Update 0.12.0 docs for regression

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5052: - Fix Version/s: 0.13.0 (was: 0.12.2) > Update 0.12.0 docs for regression >

[jira] [Updated] (HUDI-5051) Add a functional regression test for Bloom Index followed on w/ Upserts

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5051: - Fix Version/s: 0.13.0 (was: 0.12.2) > Add a functional regression test for Bloom Index

[jira] [Updated] (HUDI-5035) Remove deprecated API usage in SparkPreCommitValidator#validate

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5035: - Fix Version/s: 0.13.0 (was: 0.12.2) > Remove deprecated API usage in

[jira] [Updated] (HUDI-5032) Add Archiving to the CLI

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5032: - Fix Version/s: 0.13.0 (was: 0.12.2) > Add Archiving to the CLI >

[jira] [Updated] (HUDI-4990) Parallelize deduplication in CLI tool

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4990: - Fix Version/s: 0.13.0 (was: 0.12.2) > Parallelize deduplication in CLI tool >

[jira] [Updated] (HUDI-5018) Make user-provided copyOnWriteRecordSizeEstimate first precedence

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5018: - Fix Version/s: 0.13.0 (was: 0.12.2) > Make user-provided copyOnWriteRecordSizeEstimate

[jira] [Updated] (HUDI-4967) Improve docs for meta sync with TimestampBasedKeyGenerator

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4967: - Fix Version/s: 0.13.0 (was: 0.12.2) > Improve docs for meta sync with

[jira] [Updated] (HUDI-4888) Add validation to block COW table to use consistent hashing bucket index

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4888: - Fix Version/s: 0.13.0 (was: 0.12.2) > Add validation to block COW table to use consistent

[jira] [Updated] (HUDI-4881) Push down filters if possible when syncing partitions to Hive

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4881: - Fix Version/s: 0.13.0 (was: 0.12.2) > Push down filters if possible when syncing

[jira] [Updated] (HUDI-4839) rocksdbjni is not compatible with apple silicon

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4839: - Fix Version/s: 0.13.0 (was: 0.12.2) > rocksdbjni is not compatible with apple silicon >

[jira] [Updated] (HUDI-4823) Add read_optimize spark_session config to use in spark-sql

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4823: - Fix Version/s: 0.13.0 (was: 0.12.2) > Add read_optimize spark_session config to use in

[jira] [Updated] (HUDI-2913) Disable auto clean in writer task

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-2913: - Fix Version/s: 0.13.0 (was: 0.12.2) > Disable auto clean in writer task >

[jira] [Updated] (HUDI-3954) Don't keep the last commit before the earliest commit to retain

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-3954: - Fix Version/s: 0.13.0 (was: 0.12.2) > Don't keep the last commit before the earliest

[jira] [Updated] (HUDI-712) Improve exporter performance and memory usage

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-712: Fix Version/s: 0.13.0 (was: 0.12.2) > Improve exporter performance and memory usage >

[jira] [Updated] (HUDI-1570) Add Avg record size in commit metadata

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-1570: - Fix Version/s: (was: 0.12.2) > Add Avg record size in commit metadata >

[jira] [Updated] (HUDI-5105) Add Call show_commit_extra_metadata for spark sql

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5105: - Fix Version/s: 0.13.0 > Add Call show_commit_extra_metadata for spark sql >

[jira] [Updated] (HUDI-5201) add totalRecordsDeleted metric

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5201: - Fix Version/s: (was: 0.12.2) > add totalRecordsDeleted metric > -- > >

[jira] [Updated] (HUDI-5105) Add Call show_commit_extra_metadata for spark sql

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5105: - Fix Version/s: (was: 0.12.2) > Add Call show_commit_extra_metadata for spark sql >

[jira] [Updated] (HUDI-5059) Support automatic setting of certain attributes when creating a table in the flash catalog

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5059: - Fix Version/s: 0.13.0 (was: 0.12.2) > Support automatic setting of certain attributes when

[jira] [Updated] (HUDI-5048) add CopyToTempView support

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5048: - Fix Version/s: 0.13.0 (was: 0.12.2) > add CopyToTempView support >

[jira] [Updated] (HUDI-4809) Hudi Support AWS Glue DropPartitions

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4809: - Fix Version/s: 0.13.0 > Hudi Support AWS Glue DropPartitions > - > >

[jira] [Updated] (HUDI-4809) Hudi Support AWS Glue DropPartitions

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-4809: - Fix Version/s: (was: 0.12.2) > Hudi Support AWS Glue DropPartitions > -

[jira] [Updated] (HUDI-5168) Flink metrics integration

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5168: - Fix Version/s: 0.13.0 (was: 0.12.2) > Flink metrics integration >

[jira] [Updated] (HUDI-5334) Get checkpoint from non-completed instant

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5334: - Fix Version/s: 0.13.0 (was: 0.12.2) > Get checkpoint from non-completed instant >

[jira] [Updated] (HUDI-5318) Clustering schduling now will list all partition in table when PARTITION_SELECTED is set

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5318: - Fix Version/s: 0.13.0 (was: 0.12.2) > Clustering schduling now will list all partition in

[jira] [Updated] (HUDI-5229) Add flink avro version entry in root pom

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5229: - Fix Version/s: (was: 0.12.2) > Add flink avro version entry in root pom >

[jira] [Updated] (HUDI-5220) failed to snapshot query in hive when query a empty partition

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5220: - Fix Version/s: (was: 0.12.2) > failed to snapshot query in hive when query a empty partition >

[jira] [Updated] (HUDI-5270) Duplicate key error when insert_overwrite same partition in multi writer

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5270: - Fix Version/s: 0.13.0 (was: 0.12.2) > Duplicate key error when insert_overwrite same

[jira] [Updated] (HUDI-5174) Clustering w/ two multi-writers could lead to issues

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5174: - Fix Version/s: (was: 0.12.2) > Clustering w/ two multi-writers could lead to issues >

[jira] [Updated] (HUDI-5177) Revisit HiveIncrPullSource and JdbcSource for interleaved inflight commits

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5177: - Fix Version/s: (was: 0.12.2) > Revisit HiveIncrPullSource and JdbcSource for interleaved inflight commits >

[jira] [Updated] (HUDI-5171) Ensure validateTableConfig also checks for partition path field value switch

2022-12-20 Thread satish (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] satish updated HUDI-5171: - Fix Version/s: 0.13.0 (was: 0.12.2) > Ensure validateTableConfig also checks for partition

[jira] [Updated] (HUDI-5443) Fix exception when querying MOR table after applying NestedSchemaPruning optimization

2022-12-20 Thread Alexey Kudinkin (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated HUDI-5443: -- Sprint: 0.13.0 Final Sprint > Fix exception when querying MOR table after applying

  1   2   3   4   5   6   >