[jira] [Commented] (HUDI-4430) Incorrect type casting while reading HUDI table created with CustomKeyGenerator and unixtimestamp paritioning field

2024-10-25 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17892760#comment-17892760 ] Lokesh Jain commented on HUDI-4430: --- I have verified that this issue is not seen anymore

[jira] [Updated] (HUDI-8260) Fix col stats metadata validation so that log files are also validated

2024-10-24 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8260: -- Status: Patch Available (was: In Progress) > Fix col stats metadata validation so that log files are al

[jira] [Updated] (HUDI-8260) Fix col stats metadata validation so that log files are also validated

2024-10-24 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8260: -- Status: In Progress (was: Open) > Fix col stats metadata validation so that log files are also validate

[jira] [Updated] (HUDI-8263) Add validation for functional index in HoodieMetadataTableValidator

2024-10-23 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8263: -- Status: Patch Available (was: In Progress) > Add validation for functional index in HoodieMetadataTable

[jira] [Updated] (HUDI-8263) Add validation for functional index in HoodieMetadataTableValidator

2024-10-23 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8263: -- Status: In Progress (was: Open) > Add validation for functional index in HoodieMetadataTableValidator >

[jira] [Closed] (HUDI-8340) Parallelize building functional index over file slice

2024-10-21 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain closed HUDI-8340. - Resolution: Fixed > Parallelize building functional index over file slice > --

[jira] [Updated] (HUDI-8348) Whenever any new file slice is added, do the full computation of stats for the partitions touched

2024-10-17 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8348: -- Status: In Progress (was: Open) > Whenever any new file slice is added, do the full computation of stat

[jira] [Updated] (HUDI-8348) Whenever any new file slice is added, do the full computation of stats for the partitions touched

2024-10-17 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8348: -- Status: Patch Available (was: In Progress) > Whenever any new file slice is added, do the full computat

[jira] [Updated] (HUDI-8386) Fix update for secondary index and corresponding validation in metadata table validator

2024-10-17 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8386: -- Sprint: Hudi 1.0 Blockers+Bugs Sprint > Fix update for secondary index and corresponding validation in m

[jira] [Updated] (HUDI-8386) Fix update for secondary index and corresponding validation in metadata table validator

2024-10-17 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8386: -- Fix Version/s: 1.0.0 > Fix update for secondary index and corresponding validation in metadata table >

[jira] [Created] (HUDI-8386) Fix update for secondary index and corresponding validation in metadata table validator

2024-10-17 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8386: - Summary: Fix update for secondary index and corresponding validation in metadata table validator Key: HUDI-8386 URL: https://issues.apache.org/jira/browse/HUDI-8386 Project

[jira] [Comment Edited] (HUDI-8203) Make record merge mode the primary merging config

2024-10-16 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17889808#comment-17889808 ] Lokesh Jain edited comment on HUDI-8203 at 10/16/24 5:56 PM: -

[jira] [Comment Edited] (HUDI-8203) Make record merge mode the primary merging config

2024-10-16 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17889808#comment-17889808 ] Lokesh Jain edited comment on HUDI-8203 at 10/16/24 5:56 PM: -

[jira] [Comment Edited] (HUDI-8203) Make record merge mode the primary merging config

2024-10-16 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17889808#comment-17889808 ] Lokesh Jain edited comment on HUDI-8203 at 10/16/24 5:54 PM: -

[jira] [Comment Edited] (HUDI-8203) Make record merge mode the primary merging config

2024-10-16 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17889808#comment-17889808 ] Lokesh Jain edited comment on HUDI-8203 at 10/16/24 11:48 AM: --

[jira] [Comment Edited] (HUDI-8203) Make record merge mode the primary merging config

2024-10-16 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17889808#comment-17889808 ] Lokesh Jain edited comment on HUDI-8203 at 10/16/24 8:13 AM: -

[jira] [Updated] (HUDI-8381) Group merger related config fields and use the group in merger

2024-10-16 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8381: -- Fix Version/s: 1.1.0 > Group merger related config fields and use the group in merger >

[jira] [Created] (HUDI-8381) Group merger related config fields and use the group in merger

2024-10-16 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8381: - Summary: Group merger related config fields and use the group in merger Key: HUDI-8381 URL: https://issues.apache.org/jira/browse/HUDI-8381 Project: Apache Hudi I

[jira] [Commented] (HUDI-8203) Make record merge mode the primary merging config

2024-10-15 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17889808#comment-17889808 ] Lokesh Jain commented on HUDI-8203: --- Open items for discussion: # Backwards compatibili

[jira] [Created] (HUDI-8369) Avoid configuring DefaultHoodieRecordPayload delete marker properties every time

2024-10-15 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8369: - Summary: Avoid configuring DefaultHoodieRecordPayload delete marker properties every time Key: HUDI-8369 URL: https://issues.apache.org/jira/browse/HUDI-8369 Project: Apach

[jira] [Commented] (HUDI-7607) Test with timestamp based key generator for Hive integration of new file group reader

2024-10-13 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17889093#comment-17889093 ] Lokesh Jain commented on HUDI-7607: --- I am assuming the new file group reader is enabled

[jira] [Updated] (HUDI-8329) Add functional/integration tests for validating hive sync

2024-10-09 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8329: -- Fix Version/s: 1.1.0 > Add functional/integration tests for validating hive sync > -

[jira] [Commented] (HUDI-7613) Check write/query with CustomKeyGenerator on Flink and Hive

2024-10-09 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17888003#comment-17888003 ] Lokesh Jain commented on HUDI-7613: --- Yeah.. I have created https://issues.apache.org/jir

[jira] [Created] (HUDI-8329) Add functional/integration tests for validating hive sync

2024-10-09 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8329: - Summary: Add functional/integration tests for validating hive sync Key: HUDI-8329 URL: https://issues.apache.org/jira/browse/HUDI-8329 Project: Apache Hudi Issue T

[jira] [Updated] (HUDI-8329) Add functional/integration tests for validating hive sync

2024-10-09 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8329: -- Description: Currently we do not have an integration or functional test for validating hive sync with h

[jira] [Commented] (HUDI-7613) Check write/query with CustomKeyGenerator on Flink and Hive

2024-10-06 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17887207#comment-17887207 ] Lokesh Jain commented on HUDI-7613: --- Created: https://issues.apache.org/jira/browse/HUDI

[jira] [Created] (HUDI-8312) Support YYYY-MM-DD partition format with hive

2024-10-06 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8312: - Summary: Support -MM-DD partition format with hive Key: HUDI-8312 URL: https://issues.apache.org/jira/browse/HUDI-8312 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-8312) Support YYYY-MM-DD partition format with hive

2024-10-06 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8312: -- Fix Version/s: 1.1.0 > Support -MM-DD partition format with hive > -

[jira] [Updated] (HUDI-8311) Support YYYY/MM/DD partition format with hive

2024-10-06 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8311: -- Fix Version/s: 1.1.0 > Support /MM/DD partition format with hive > -

[jira] [Updated] (HUDI-8311) Support YYYY/MM/DD partition format with hive

2024-10-06 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8311: -- Summary: Support /MM/DD partition format with hive (was: Support multi part partition format with h

[jira] [Created] (HUDI-8311) Support multi part partition format with hive

2024-10-06 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8311: - Summary: Support multi part partition format with hive Key: HUDI-8311 URL: https://issues.apache.org/jira/browse/HUDI-8311 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-8302) Handle multiple secondary indexes

2024-10-04 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8302: -- Description: Currently there is only one secondary index supported. The Jira aims to support multiple s

[jira] [Updated] (HUDI-8302) Handle multiple secondary indexes

2024-10-04 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8302: -- Description: Currently there is only one secondary index supported. The Jira aims to support multiple s

[jira] [Updated] (HUDI-8302) Handle multiple secondary indexes

2024-10-04 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8302: -- Fix Version/s: 1.1.0 > Handle multiple secondary indexes > - > >

[jira] [Created] (HUDI-8302) Handle multiple secondary indexes

2024-10-04 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8302: - Summary: Handle multiple secondary indexes Key: HUDI-8302 URL: https://issues.apache.org/jira/browse/HUDI-8302 Project: Apache Hudi Issue Type: Bug Rep

[jira] [Comment Edited] (HUDI-7613) Check write/query with CustomKeyGenerator on Flink and Hive

2024-10-03 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17886821#comment-17886821 ] Lokesh Jain edited comment on HUDI-7613 at 10/4/24 1:45 AM: Hi

[jira] [Commented] (HUDI-7613) Check write/query with CustomKeyGenerator on Flink and Hive

2024-10-03 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17886821#comment-17886821 ] Lokesh Jain commented on HUDI-7613: --- Hive queries: hudi_table_2 is the synced table in h

[jira] [Updated] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-10-03 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8262: -- Reviewers: Sagar Sumit > Add validation for secondary index in HoodieMetadataTableValidator > --

[jira] [Updated] (HUDI-8177) Add validation of new index types in HoodieMetadataTableValidator

2024-10-03 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8177: -- Remaining Estimate: 9h (was: 8h) > Add validation of new index types in HoodieMetadataTableValidator >

[jira] [Updated] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-10-03 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8262: -- Remaining Estimate: 8h Original Estimate: 8h > Add validation for secondary index in HoodieMetadata

[jira] [Updated] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-10-03 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8262: -- Sprint: Hudi 1.0 Sprint 2024/9/30-10/6 (was: Hudi 1.0 Sprint2024/10/7-10/13) > Add validation for seco

[jira] [Updated] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-10-03 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8262: -- Status: In Progress (was: Open) > Add validation for secondary index in HoodieMetadataTableValidator >

[jira] [Updated] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-10-03 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8262: -- Status: Patch Available (was: In Progress) > Add validation for secondary index in HoodieMetadataTableV

[jira] [Updated] (HUDI-7613) Check write/query with CustomKeyGenerator on Flink and Hive

2024-10-03 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-7613: -- Status: Patch Available (was: In Progress) > Check write/query with CustomKeyGenerator on Flink and Hiv

[jira] [Commented] (HUDI-7613) Check write/query with CustomKeyGenerator on Flink and Hive

2024-10-02 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17886546#comment-17886546 ] Lokesh Jain commented on HUDI-7613: --- Hive test script: Tested using Docker demo {code:

[jira] [Updated] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-09-27 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8262: -- Story Points: 8 > Add validation for secondary index in HoodieMetadataTableValidator > -

[jira] [Updated] (HUDI-8263) Add validation for functional index in HoodieMetadataTableValidator

2024-09-27 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8263: -- Story Points: 8 > Add validation for functional index in HoodieMetadataTableValidator >

[jira] [Assigned] (HUDI-8263) Add validation for functional index in HoodieMetadataTableValidator

2024-09-26 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain reassigned HUDI-8263: - Assignee: Lokesh Jain > Add validation for functional index in HoodieMetadataTableValidator > ---

[jira] [Assigned] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-09-26 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain reassigned HUDI-8262: - Assignee: Lokesh Jain > Add validation for secondary index in HoodieMetadataTableValidator >

[jira] [Updated] (HUDI-8263) Add validation for functional index in HoodieMetadataTableValidator

2024-09-26 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8263: -- Fix Version/s: 1.0.0 > Add validation for functional index in HoodieMetadataTableValidator > ---

[jira] [Updated] (HUDI-8263) Add validation for functional index in HoodieMetadataTableValidator

2024-09-26 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8263: -- Description: The Jira aims to add validation for functional index in HoodieMetadataTableValidator. > Ad

[jira] [Updated] (HUDI-8263) Add validation for functional index in HoodieMetadataTableValidator

2024-09-26 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8263: -- Priority: Blocker (was: Major) > Add validation for functional index in HoodieMetadataTableValidator >

[jira] [Updated] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-09-26 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8262: -- Priority: Blocker (was: Major) > Add validation for secondary index in HoodieMetadataTableValidator > -

[jira] [Updated] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-09-26 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8262: -- Fix Version/s: 1.0.0 > Add validation for secondary index in HoodieMetadataTableValidator >

[jira] [Created] (HUDI-8263) Add validation for functional index in HoodieMetadataTableValidator

2024-09-26 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8263: - Summary: Add validation for functional index in HoodieMetadataTableValidator Key: HUDI-8263 URL: https://issues.apache.org/jira/browse/HUDI-8263 Project: Apache Hudi

[jira] [Updated] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-09-26 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8262: -- Description: The Jira aims to add validation for secondary index in HoodieMetadataTableValidator. > Add

[jira] [Updated] (HUDI-8262) Add validation for secondary index in HoodieMetadataTableValidator

2024-09-26 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8262: -- Summary: Add validation for secondary index in HoodieMetadataTableValidator (was: Add val) > Add valid

[jira] [Created] (HUDI-8262) Add val

2024-09-26 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8262: - Summary: Add val Key: HUDI-8262 URL: https://issues.apache.org/jira/browse/HUDI-8262 Project: Apache Hudi Issue Type: Bug Reporter: Lokesh Jain -

[jira] [Created] (HUDI-8260) Fix col stats metadata validation so that log files are also validated

2024-09-25 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8260: - Summary: Fix col stats metadata validation so that log files are also validated Key: HUDI-8260 URL: https://issues.apache.org/jira/browse/HUDI-8260 Project: Apache Hudi

[jira] [Updated] (HUDI-8208) Fix partition stats with compaction or clustering

2024-09-25 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8208: -- Description: Consider a partition with 10 file slices. If compaction triggered for 1 file slice fs1_1,

[jira] [Updated] (HUDI-8188) Add validation for partition stats index in HoodieMetadataTableValidator

2024-09-23 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8188: -- Status: Patch Available (was: In Progress) > Add validation for partition stats index in HoodieMetadata

[jira] [Updated] (HUDI-7613) Check write/query with CustomKeyGenerator on Flink and Hive

2024-09-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-7613: -- Status: Patch Available (was: In Progress) > Check write/query with CustomKeyGenerator on Flink and Hiv

[jira] [Updated] (HUDI-7613) Check write/query with CustomKeyGenerator on Flink and Hive

2024-09-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-7613: -- Status: In Progress (was: Open) > Check write/query with CustomKeyGenerator on Flink and Hive > ---

[jira] [Commented] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883679#comment-17883679 ] Lokesh Jain commented on HUDI-8220: --- Updated the exception message which is being seen.

[jira] [Updated] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8220: -- Description: If we create a table in flink with custom key generator it fails with error message. {code

[jira] [Commented] (HUDI-7613) Check write/query with CustomKeyGenerator on Flink and Hive

2024-09-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883365#comment-17883365 ] Lokesh Jain commented on HUDI-7613: --- Verified that hive sync is working with custom keyg

[jira] [Created] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-20 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8220: - Summary: CustomKeyGenerator can not be created with flink Key: HUDI-8220 URL: https://issues.apache.org/jira/browse/HUDI-8220 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8220: -- Fix Version/s: 1.0.0 > CustomKeyGenerator can not be created with flink > --

[jira] [Updated] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8220: -- Fix Version/s: (was: 1.0.0) (was: 0.15.1) > CustomKeyGenerator can not be cre

[jira] [Updated] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8220: -- Affects Version/s: 0.15.0 1.0.0 > CustomKeyGenerator can not be created with flin

[jira] [Updated] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8220: -- Affects Version/s: (was: 0.15.0) (was: 1.0.0) > CustomKeyGenerator can no

[jira] [Updated] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8220: -- Fix Version/s: 1.0.0 0.15.1 > CustomKeyGenerator can not be created with flink >

[jira] [Commented] (HUDI-7613) Check write/query with CustomKeyGenerator on Flink and Hive

2024-09-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883175#comment-17883175 ] Lokesh Jain commented on HUDI-7613: --- With flink https://issues.apache.org/jira/browse/HU

[jira] [Comment Edited] (HUDI-8208) Fix partition stats with compaction or clustering

2024-09-18 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17882772#comment-17882772 ] Lokesh Jain edited comment on HUDI-8208 at 9/18/24 5:46 PM: Th

[jira] [Commented] (HUDI-8208) Fix partition stats with compaction or clustering

2024-09-18 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17882772#comment-17882772 ] Lokesh Jain commented on HUDI-8208: --- The fix requires that partition stats should be new

[jira] [Updated] (HUDI-7280) Add/Drop/Rename table properties hoodie.properties

2024-09-18 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-7280: -- Story Points: 4 > Add/Drop/Rename table properties hoodie.properties >

[jira] [Updated] (HUDI-8208) Fix partition stats with compaction or clustering

2024-09-17 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8208: -- Status: In Progress (was: Open) > Fix partition stats with compaction or clustering > -

[jira] [Updated] (HUDI-8208) Fix partition stats with compaction or clustering

2024-09-17 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8208: -- Sprint: Hudi 1.0 Sprint 2024/09/16-22 > Fix partition stats with compaction or clustering >

[jira] [Created] (HUDI-8208) Fix partition stats with compaction or clustering

2024-09-17 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8208: - Summary: Fix partition stats with compaction or clustering Key: HUDI-8208 URL: https://issues.apache.org/jira/browse/HUDI-8208 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-8187) Hudi 1.0 reader should be able to read both 1.0 and 0.x tables with custom key generator

2024-09-12 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8187: -- Description: We changed the format of partition field table config "{{{}hoodie.table.partition.fields{}

[jira] [Commented] (HUDI-7524) Ensure existing hoodie.properties are not overwritten with HoodieTableMetaClient creation

2024-09-11 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881180#comment-17881180 ] Lokesh Jain commented on HUDI-7524: --- Closing the ticket based on Siva's comment on the P

[jira] [Closed] (HUDI-7524) Ensure existing hoodie.properties are not overwritten with HoodieTableMetaClient creation

2024-09-11 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain closed HUDI-7524. - Resolution: Won't Fix > Ensure existing hoodie.properties are not overwritten with > HoodieTableMetaClien

[jira] [Updated] (HUDI-8187) Hudi 1.0 reader should be able to read both 1.0 and 0.x tables with custom key generator

2024-09-11 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8187: -- Status: Patch Available (was: In Progress) > Hudi 1.0 reader should be able to read both 1.0 and 0.x ta

[jira] [Updated] (HUDI-8187) Hudi 1.0 reader should be able to read both 1.0 and 0.x tables with custom key generator

2024-09-11 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8187: -- Status: In Progress (was: Open) > Hudi 1.0 reader should be able to read both 1.0 and 0.x tables with c

[jira] [Updated] (HUDI-8188) Add validation for partition stats index in HoodieMetadataTableValidator

2024-09-10 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8188: -- Description: In this Jira, we are adding validation support for partition stat index in . Partition stat

[jira] [Updated] (HUDI-8188) Add validation for partition stats index in HoodieMetadataTableValidator

2024-09-10 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8188: -- Fix Version/s: 1.0.0 > Add validation for partition stats index in HoodieMetadataTableValidator > --

[jira] [Updated] (HUDI-8188) Add validation for partition stats index in HoodieMetadataTableValidator

2024-09-10 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8188: -- Description: In this Jira, we are adding validation support for partition stat index. Partition stat ind

[jira] [Created] (HUDI-8188) Add validation for partition stats index in HoodieMetadataTableValidator

2024-09-10 Thread Lokesh Jain (Jira)
Lokesh Jain created HUDI-8188: - Summary: Add validation for partition stats index in HoodieMetadataTableValidator Key: HUDI-8188 URL: https://issues.apache.org/jira/browse/HUDI-8188 Project: Apache Hudi

[jira] [Updated] (HUDI-8034) Support custom key generator with HoodieCatalogTable

2024-09-06 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8034: -- Status: In Progress (was: Reopened) > Support custom key generator with HoodieCatalogTable > --

[jira] [Reopened] (HUDI-8034) Support custom key generator with HoodieCatalogTable

2024-09-06 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain reopened HUDI-8034: --- > Support custom key generator with HoodieCatalogTable > -

[jira] [Updated] (HUDI-8110) Throw an error for time travel query on MDT

2024-08-23 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8110: -- Status: In Progress (was: Open) > Throw an error for time travel query on MDT > ---

[jira] [Updated] (HUDI-8034) Support custom key generator with HoodieCatalogTable

2024-08-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8034: -- Status: Patch Available (was: In Progress) > Support custom key generator with HoodieCatalogTable > ---

[jira] [Updated] (HUDI-8034) Support custom key generator with HoodieCatalogTable

2024-08-22 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8034: -- Summary: Support custom key generator with HoodieCatalogTable (was: KeyGenUtils#inferKeyGeneratorTypeFo

[jira] [Updated] (HUDI-8034) KeyGenUtils#inferKeyGeneratorTypeForAutoKeyGen should support custom key generator

2024-08-21 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8034: -- Status: In Progress (was: Open) > KeyGenUtils#inferKeyGeneratorTypeForAutoKeyGen should support custom

[jira] [Updated] (HUDI-7970) Move methods from HoodieTableConfigUtils to HoodieTableConfig

2024-08-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-7970: -- Sprint: Hudi 1.0 Sprint 2024/08/19-25 > Move methods from HoodieTableConfigUtils to HoodieTableConfig >

[jira] [Updated] (HUDI-7902) Partition fields in Table config should store partition field types for custom key generator

2024-08-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-7902: -- Status: Patch Available (was: In Progress) > Partition fields in Table config should store partition fi

[jira] [Updated] (HUDI-7902) Partition fields in Table config should store partition field types for custom key generator

2024-08-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-7902: -- Status: In Progress (was: Open) > Partition fields in Table config should store partition field types f

[jira] [Resolved] (HUDI-8044) Store timestamp keygen configs for custom key generator

2024-08-20 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain resolved HUDI-8044. --- > Store timestamp keygen configs for custom key generator > --

[jira] [Updated] (HUDI-8036) Handle partition schema for custom key gen in SparkHoodieTableFileIndex

2024-08-12 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8036: -- Status: Patch Available (was: In Progress) > Handle partition schema for custom key gen in SparkHoodieT

[jira] [Updated] (HUDI-8044) Store timestamp keygen configs for custom key generator

2024-08-07 Thread Lokesh Jain (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HUDI-8044: -- Status: Patch Available (was: In Progress) > Store timestamp keygen configs for custom key generator >

  1   2   3   4   >