[jira] [Updated] (HUDI-8277) Update wait time in ITTestDataStreamWrite#execute to prevent unit test failures

2024-09-30 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8277: - Fix Version/s: 1.0.0 > Update wait time in ITTestDataStreamWrite#execute to prevent unit test > failures

[jira] [Closed] (HUDI-8277) Update wait time in ITTestDataStreamWrite#execute to prevent unit test failures

2024-09-30 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8277. Resolution: Fixed Fixed via master branch: 67a060321bee90c180088a6d1c40ec354f07a187 > Update wait time in I

[jira] [Closed] (HUDI-8160) Verify the consistency of the user-defined schema and the existing hoodie scheme when creating the hoodie table

2024-09-29 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8160. Fix Version/s: 1.0.0 Assignee: Danny Chen Resolution: Fixed Fixed via master branch: 4e98278

[jira] [Commented] (HUDI-6909) Handle `_hoodie_operation` field in the new HoodieFileGroupReader

2024-09-29 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17885753#comment-17885753 ] Danny Chen commented on HUDI-6909: -- Fixed via master branch: 7bbe1a42406110f6982384ea6f97

[jira] [Closed] (HUDI-8266) CleanPlanner is not serializable

2024-09-27 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8266. Fix Version/s: 1.0.0 Resolution: Fixed > CleanPlanner is not serializable > -

[jira] [Commented] (HUDI-8266) CleanPlanner is not serializable

2024-09-27 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17885304#comment-17885304 ] Danny Chen commented on HUDI-8266: -- Fixed via master branch: 215deedab36bf24bba11472e8253

[jira] [Commented] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-26 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17885208#comment-17885208 ] Danny Chen commented on HUDI-8220: -- Moved it to 1.1.0 release because it looks like we ne

[jira] [Updated] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-26 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8220: - Fix Version/s: 1.1.0 (was: 1.0.0) > CustomKeyGenerator can not be created with flin

[jira] [Resolved] (HUDI-8161) Make spark-sql command 'desc' independent from schema evolution config

2024-09-26 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-8161. -- > Make spark-sql command 'desc' independent from schema evolution config > -

[jira] [Closed] (HUDI-8161) Make spark-sql command 'desc' independent from schema evolution config

2024-09-26 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8161. Resolution: Fixed Fixed via master branch: f2d90f5c36eb737704500b41f725b626bc7ad101 > Make spark-sql comman

[jira] [Updated] (HUDI-8161) Make spark-sql command 'desc' independent from schema evolution config

2024-09-26 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8161: - Fix Version/s: 1.0.0 > Make spark-sql command 'desc' independent from schema evolution config > --

[jira] [Updated] (HUDI-8164) Chain multiple indices support in HoodieFileIndex

2024-09-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8164: - Fix Version/s: 1.1.0 (was: 1.0.0) > Chain multiple indices support in HoodieFileInd

[jira] [Closed] (HUDI-8143) NB CC across Flink/Spark for multiple writers and all table services (clean, compaction, clustering, ...)

2024-09-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8143. Resolution: Fixed > NB CC across Flink/Spark for multiple writers and all table services (clean, > compacti

[jira] [Closed] (HUDI-8075) Revisit table service scheduling and execution with the completion time

2024-09-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8075. Resolution: Fixed > Revisit table service scheduling and execution with the completion time > --

[jira] [Closed] (HUDI-8215) Support composition compaction strategy

2024-09-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8215. Resolution: Fixed Fixed via master branch: e4cc7f05c9e1a860c9f95a914ff15cb3c2a4578c > Support composition c

[jira] [Commented] (HUDI-6930) Eliminate the shouldLock flag for TimeGenerator(phase2)

2024-09-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883645#comment-17883645 ] Danny Chen commented on HUDI-6930: -- Like described in the context, there is no good sulut

[jira] [Resolved] (HUDI-8143) NB CC across Flink/Spark for multiple writers and all table services (clean, compaction, clustering, ...)

2024-09-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-8143. -- > NB CC across Flink/Spark for multiple writers and all table services (clean, > compaction, clustering, ..

[jira] [Updated] (HUDI-8143) NB CC across Flink/Spark for multiple writers and all table services (clean, compaction, clustering, ...)

2024-09-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8143: - Status: In Progress (was: Open) > NB CC across Flink/Spark for multiple writers and all table services (c

[jira] [Updated] (HUDI-6930) Eliminate the shouldLock flag for TimeGenerator(phase2)

2024-09-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6930: - Summary: Eliminate the shouldLock flag for TimeGenerator(phase2) (was: Eliminate the shouldLock flag for

[jira] [Resolved] (HUDI-8075) Revisit table service scheduling and execution with the completion time

2024-09-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-8075. -- > Revisit table service scheduling and execution with the completion time >

[jira] [Commented] (HUDI-8182) Optimize schema evolution for Hive

2024-09-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883597#comment-17883597 ] Danny Chen commented on HUDI-8182: -- Fixed via master branch: 77eb9e58287b0cb1ad142364489e

[jira] [Resolved] (HUDI-8182) Optimize schema evolution for Hive

2024-09-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen resolved HUDI-8182. -- > Optimize schema evolution for Hive > -- > > Key: HUDI-8182

[jira] [Updated] (HUDI-8182) Optimize schema evolution for Hive

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8182: - Sprint: Hudi 1.0 Sprint 2024/09/16-22 > Optimize schema evolution for Hive > -

[jira] [Updated] (HUDI-8182) Optimize schema evolution for Hive

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8182: - Fix Version/s: 1.0.0 > Optimize schema evolution for Hive > -- > >

[jira] [Assigned] (HUDI-8182) Optimize schema evolution for Hive

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-8182: Assignee: Danny Chen > Optimize schema evolution for Hive > -- > >

[jira] [Updated] (HUDI-8182) Optimize schema evolution for Hive

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8182: - Status: In Progress (was: Open) > Optimize schema evolution for Hive > --

[jira] [Assigned] (HUDI-8215) Support composition compaction strategy

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-8215: Assignee: Danny Chen > Support composition compaction strategy > --

[jira] [Updated] (HUDI-8215) Support composition compaction strategy

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8215: - Fix Version/s: 1.0.0 > Support composition compaction strategy > --- >

[jira] [Commented] (HUDI-8075) Revisit table service scheduling and execution with the completion time

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883439#comment-17883439 ] Danny Chen commented on HUDI-8075: -- I went through the cleaning service logic again with

[jira] [Updated] (HUDI-7545) Concurrency control for LSM timeline management and writing.(phase-2)

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7545: - Fix Version/s: 1.1.0 (was: 1.0.0) > Concurrency control for LSM timeline management

[jira] [Commented] (HUDI-7545) Concurrency control for LSM timeline management and writing.(phase-2)

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883438#comment-17883438 ] Danny Chen commented on HUDI-7545: -- Currently, we always add a lock for archiving(again t

[jira] [Updated] (HUDI-7545) Concurrency control for LSM timeline management and writing.(phase-2)

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7545: - Summary: Concurrency control for LSM timeline management and writing.(phase-2) (was: Concurrency control

[jira] [Comment Edited] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883202#comment-17883202 ] Danny Chen edited comment on HUDI-8220 at 9/20/24 9:34 AM: --- Let'

[jira] [Commented] (HUDI-8220) CustomKeyGenerator can not be created with flink

2024-09-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883202#comment-17883202 ] Danny Chen commented on HUDI-8220: -- Let's revisit this option value, it looks like a beak

[jira] [Closed] (HUDI-8214) Support specify partitions with regex for compaction

2024-09-19 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8214. Fix Version/s: 1.0.0 Reviewers: Danny Chen Resolution: Fixed Fixed via master branch: b7320ec

[jira] [Closed] (HUDI-7609) Spark cannot write the hudi table containing array type created by flink

2024-09-19 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7609. Resolution: Fixed Fixed via master branch: ca568befc1b6cc13f647d36a201806447180f044 > Spark cannot write th

[jira] [Closed] (HUDI-8191) Fix MockStateInitializationContext#getKeyedStateStore to use non-KeyedStateStore

2024-09-18 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8191. Resolution: Fixed  Fixed via master branch: b9e40df9b44b344953e156a1e35056fe739f8fda > Fix MockStateInitial

[jira] [Closed] (HUDI-5530) Some warning when mvn build.

2024-09-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-5530. Resolution: Fixed  Fixed via master branch: 75ab91c58e538e206ab64e57befee50fa61be933 > Some warning when mv

[jira] [Updated] (HUDI-8155) Review TrueTime generator APIs

2024-09-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8155: - Status: Patch Available (was: In Progress) > Review TrueTime generator APIs > ---

[jira] [Updated] (HUDI-8155) Review TrueTime generator APIs

2024-09-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8155: - Status: In Progress (was: Open) > Review TrueTime generator APIs > -- > >

[jira] [Updated] (HUDI-8143) NB CC across Flink/Spark for multiple writers and all table services (clean, compaction, clustering, ...)

2024-09-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8143: - Description: We should suggest the users to always enable just one table service for multiple jobs. 1. C

[jira] [Updated] (HUDI-8075) Revisit table service scheduling and execution with the completion time

2024-09-16 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8075: - Description: * Cleaning: need to fix the retained-to instant to be a completion time; * Clustering: shoul

[jira] [Commented] (HUDI-7174) Fix ITTestHoodieDataSource.testMultipleLogBlocksWithDataSkipping:1645

2024-09-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881881#comment-17881881 ] Danny Chen commented on HUDI-7174: -- This test is not needed anymore because master has re

[jira] [Closed] (HUDI-7174) Fix ITTestHoodieDataSource.testMultipleLogBlocksWithDataSkipping:1645

2024-09-15 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7174. Resolution: Won't Fix > FixITTestHoodieDataSource.testMultipleLogBlocksWithDataSkipping:1645 >

[jira] [Updated] (HUDI-6700) Archiving should be time based, not this min-max and not per instant. Lets treat it like a log (Phase 2)

2024-09-12 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6700: - Labels: 1.0 (was: hudi-1.0.0-beta2) > Archiving should be time based, not this min-max and not per instan

[jira] [Closed] (HUDI-7867) Data deduplication caused by drawback in the delete invalid files before commit

2024-09-09 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7867. Fix Version/s: 1.0.0 0.15.1 Reviewers: Danny Chen Assignee: Jing Zhang

[jira] [Created] (HUDI-8165) Implements LRU eviction strategy for in-memory map of ExternalSpillableMap

2024-09-04 Thread Danny Chen (Jira)
Danny Chen created HUDI-8165: Summary: Implements LRU eviction strategy for in-memory map of ExternalSpillableMap Key: HUDI-8165 URL: https://issues.apache.org/jira/browse/HUDI-8165 Project: Apache Hudi

[jira] [Closed] (HUDI-8109) Fix the error related to locking during clustering/compaction when calling the file system in OSS.

2024-09-03 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8109. Fix Version/s: 1.0.0 Assignee: Danny Chen Resolution: Fixed Fixed via master branch: 52fd36d

[jira] [Closed] (HUDI-3488) The flink small file list should exclude file slices with pending compaction

2024-09-03 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-3488. Resolution: Fixed > The flink small file list should exclude file slices with pending compaction > -

[jira] [Updated] (HUDI-8142) Multi-writer unique key enforcement for OCC/NBCC

2024-09-03 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8142: - Story Points: 4 > Multi-writer unique key enforcement for OCC/NBCC > -

[jira] [Updated] (HUDI-7545) Concurrency control for LSM timeline management and writing.

2024-09-03 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7545: - Story Points: 4 > Concurrency control for LSM timeline management and writing. > -

[jira] [Updated] (HUDI-8136) New instant time generation for Flink streaming pipeline

2024-08-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8136: - Description: h2. Design The new design moves the instant time generation from being blocked on the check

[jira] [Updated] (HUDI-8136) New instant time generation for Flink streaming pipeline

2024-08-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8136: - Description: h2. Design The new design moves the instant time generation from being blocked on the checkp

[jira] [Created] (HUDI-8136) New instant time generation for Flink streaming pipeline

2024-08-28 Thread Danny Chen (Jira)
Danny Chen created HUDI-8136: Summary: New instant time generation for Flink streaming pipeline Key: HUDI-8136 URL: https://issues.apache.org/jira/browse/HUDI-8136 Project: Apache Hudi Issue Type

[jira] [Assigned] (HUDI-8136) New instant time generation for Flink streaming pipeline

2024-08-28 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-8136: Assignee: Danny Chen > New instant time generation for Flink streaming pipeline > -

[jira] [Created] (HUDI-8121) Flink implements the skip merge queries using the file group reader

2024-08-25 Thread Danny Chen (Jira)
Danny Chen created HUDI-8121: Summary: Flink implements the skip merge queries using the file group reader Key: HUDI-8121 URL: https://issues.apache.org/jira/browse/HUDI-8121 Project: Apache Hudi

[jira] [Created] (HUDI-8120) Flink implements the read optimized queries using the file group reader

2024-08-25 Thread Danny Chen (Jira)
Danny Chen created HUDI-8120: Summary: Flink implements the read optimized queries using the file group reader Key: HUDI-8120 URL: https://issues.apache.org/jira/browse/HUDI-8120 Project: Apache Hudi

[jira] [Created] (HUDI-8119) Flink implements the snapshot queries using the file group reader

2024-08-25 Thread Danny Chen (Jira)
Danny Chen created HUDI-8119: Summary: Flink implements the snapshot queries using the file group reader Key: HUDI-8119 URL: https://issues.apache.org/jira/browse/HUDI-8119 Project: Apache Hudi

[jira] [Created] (HUDI-8117) Implement the HoodieFlinkRecord akka to the HoodieSparkRecord

2024-08-25 Thread Danny Chen (Jira)
Danny Chen created HUDI-8117: Summary: Implement the HoodieFlinkRecord akka to the HoodieSparkRecord Key: HUDI-8117 URL: https://issues.apache.org/jira/browse/HUDI-8117 Project: Apache Hudi Issu

[jira] [Created] (HUDI-8118) Implement the Flink specific FileGroupReader with the HoodieFlinkRecord

2024-08-25 Thread Danny Chen (Jira)
Danny Chen created HUDI-8118: Summary: Implement the Flink specific FileGroupReader with the HoodieFlinkRecord Key: HUDI-8118 URL: https://issues.apache.org/jira/browse/HUDI-8118 Project: Apache Hudi

[jira] [Updated] (HUDI-6788) Integrate FileGroupReader with MergeOnReadInputFormat for Flink

2024-08-25 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6788: - Description: The existing MergeOnReadInputFormat implements different iterators for all kinds of read mor

[jira] [Updated] (HUDI-6788) Integrate FileGroupReader with MergeOnReadInputFormat for Flink

2024-08-25 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6788: - Description: The existing  > Integrate FileGroupReader with MergeOnReadInputFormat for Flink > ---

[jira] [Assigned] (HUDI-8078) Persisting writestatus to optimize the writes

2024-08-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-8078: Assignee: Danny Chen > Persisting writestatus to optimize the writes > ---

[jira] [Closed] (HUDI-8078) Persisting writestatus to optimize the writes

2024-08-24 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8078. Resolution: Fixed Fixed via master branch: e0ef86421993c1664da5cbb3ab7de7e87f16cb49 > Persisting writestatu

[jira] [Closed] (HUDI-8070) Support Flink 1.19 in Hudi

2024-08-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8070. Resolution: Fixed Fixed via master branch: d550b46d72488f92abfb234f919506e76900d69f > Support Flink 1.19 in

[jira] [Updated] (HUDI-8112) Fix TestHoodieActiveTimeline#testTimelineGetOperations test logic error

2024-08-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8112: - Fix Version/s: 1.0.0 > Fix TestHoodieActiveTimeline#testTimelineGetOperations test logic error > -

[jira] [Closed] (HUDI-8112) Fix TestHoodieActiveTimeline#testTimelineGetOperations test logic error

2024-08-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8112. Resolution: Fixed Fixed via master branch: e1f70fda5890cff44b3e44eb362c5a411f4174f3 > Fix TestHoodieActiveT

[jira] [Closed] (HUDI-8104) Hive on spark select mor rt table, org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit cannot be cast to org.apache.hadoop.hive.shims.HadoopShimsSecure

2024-08-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8104. Resolution: Fixed Fixed via master branch: b6bad403fa97479c44112b741007a08102617ab3 > Hive on spark select

[jira] [Updated] (HUDI-8104) Hive on spark select mor rt table, org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit cannot be cast to org.apache.hadoop.hive.shims.HadoopShimsSecure

2024-08-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8104: - Fix Version/s: 1.0.0 > Hive on spark select mor rt table, > org.apache.hudi.hadoop.hive.HoodieCombineReal

[jira] [Closed] (HUDI-8097) Schema evolution setting from hudi-defaults.conf is ignored while altering column in Spark

2024-08-19 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8097. Resolution: Fixed Fixed via master branch: e4e5bf0232093bfba25f1769fb63258e9f816a8f > Schema evolution sett

[jira] [Updated] (HUDI-8097) Schema evolution setting from hudi-defaults.conf is ignored while altering column in Spark

2024-08-19 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8097: - Status: Open (was: In Progress) > Schema evolution setting from hudi-defaults.conf is ignored while alter

[jira] [Updated] (HUDI-8097) Schema evolution setting from hudi-defaults.conf is ignored while altering column in Spark

2024-08-19 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8097: - Fix Version/s: 1.0.0 > Schema evolution setting from hudi-defaults.conf is ignored while altering > colum

[jira] [Closed] (HUDI-7873) Remove getStorage method from HoodieReaderContext

2024-08-14 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7873. Fix Version/s: 1.0.0 Resolution: Fixed Fixed via master branch: 64f546b8f0cae70793a6150170a649bad8e0e

[jira] [Updated] (HUDI-8077) Fix the incremental cleaning to base on completion time

2024-08-13 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8077: - Description: Currently, the incremental cleaning will remember a marker instant of last retained in the c

[jira] [Created] (HUDI-8077) Fix the incremental cleaning to base on completion time

2024-08-13 Thread Danny Chen (Jira)
Danny Chen created HUDI-8077: Summary: Fix the incremental cleaning to base on completion time Key: HUDI-8077 URL: https://issues.apache.org/jira/browse/HUDI-8077 Project: Apache Hudi Issue Type:

[jira] [Closed] (HUDI-8074) Improve comaction operator shuffle rebanlance

2024-08-13 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8074. Resolution: Fixed Fixed via master branch: 35c00daaf871a6c1b87d6a440832d60f9b26ee14 > Improve comaction ope

[jira] [Created] (HUDI-8074) Improve comaction operator shuffle rebanlance

2024-08-13 Thread Danny Chen (Jira)
Danny Chen created HUDI-8074: Summary: Improve comaction operator shuffle rebanlance Key: HUDI-8074 URL: https://issues.apache.org/jira/browse/HUDI-8074 Project: Apache Hudi Issue Type: Improvem

[jira] [Assigned] (HUDI-7930) After Compaction RuntimeException: Unsupported type in the list: optional binary xxx (STRING)

2024-08-05 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-7930: Assignee: Danny Chen > After Compaction RuntimeException: Unsupported type in the list: optional >

[jira] [Updated] (HUDI-7930) Flink Support for Array of Row and Map of Row value

2024-08-05 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7930: - Summary: Flink Support for Array of Row and Map of Row value (was: After Compaction RuntimeException: Uns

[jira] [Assigned] (HUDI-8041) Support projection push down for lookup join

2024-08-02 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-8041: Assignee: Danny Chen > Support projection push down for lookup join > -

[jira] [Created] (HUDI-8041) Support projection push down for lookup join

2024-08-02 Thread Danny Chen (Jira)
Danny Chen created HUDI-8041: Summary: Support projection push down for lookup join Key: HUDI-8041 URL: https://issues.apache.org/jira/browse/HUDI-8041 Project: Apache Hudi Issue Type: Improvemen

[jira] [Closed] (HUDI-8040) Fix SimpleConcurrentFileWritesConflictResolutionStrategy get pending clustering wrong

2024-08-02 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8040. Resolution: Fixed Fixed via master branch: 0fbf31281964a74e1d1595bc830cf04e8196063e > Fix SimpleConcurrentF

[jira] [Assigned] (HUDI-8040) Fix SimpleConcurrentFileWritesConflictResolutionStrategy get pending clustering wrong

2024-08-02 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-8040: Assignee: Danny Chen > Fix SimpleConcurrentFileWritesConflictResolutionStrategy get pending > clus

[jira] [Created] (HUDI-8040) Fix SimpleConcurrentFileWritesConflictResolutionStrategy get pending clustering wrong

2024-08-02 Thread Danny Chen (Jira)
Danny Chen created HUDI-8040: Summary: Fix SimpleConcurrentFileWritesConflictResolutionStrategy get pending clustering wrong Key: HUDI-8040 URL: https://issues.apache.org/jira/browse/HUDI-8040 Project: Ap

[jira] [Updated] (HUDI-5412) Send the boostrap event if the JM also rebooted

2024-07-31 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-5412: - Fix Version/s: (was: 0.12.2) > Send the boostrap event if the JM also rebooted > -

[jira] [Closed] (HUDI-7408) LSM tree writer failed with compaction

2024-07-30 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7408. Resolution: Fixed Fixed via master branch: https://github.com/apache/hudi/pull/10883 > LSM tree writer fail

[jira] [Assigned] (HUDI-8033) RFC-81: Log Compaction with Merge Sort

2024-07-30 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen reassigned HUDI-8033: Assignee: Danny Chen > RFC-81: Log Compaction with Merge Sort > ---

[jira] [Closed] (HUDI-6453) Cascade schema changes to partition level

2024-07-29 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6453. Fix Version/s: 1.0.0 Resolution: Fixed Fixed via master branch: 2024d17704e6856174fc6b56ba51cd2d1b93b

[jira] [Closed] (HUDI-4625) Clean up KafkaOffsetGen

2024-07-29 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-4625. Fix Version/s: 1.0.0 Resolution: Fixed Fixed via master branch: 6485fa5c8a79f3199e24dfdf721d9d1993f88

[jira] [Updated] (HUDI-4625) Clean up KafkaOffsetGen

2024-07-29 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-4625: - Status: Open (was: In Progress) > Clean up KafkaOffsetGen > --- > > K

[jira] [Created] (HUDI-8030) Fix for add missing table configurations to payloadProps

2024-07-28 Thread Danny Chen (Jira)
Danny Chen created HUDI-8030: Summary: Fix for add missing table configurations to payloadProps Key: HUDI-8030 URL: https://issues.apache.org/jira/browse/HUDI-8030 Project: Apache Hudi Issue Type

[jira] [Updated] (HUDI-6191) Improve passing the debezium checkpoint values to start job from offset

2024-07-25 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-6191: - Status: Open (was: In Progress) > Improve passing the debezium checkpoint values to start job from offset

[jira] [Closed] (HUDI-6191) Improve passing the debezium checkpoint values to start job from offset

2024-07-25 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-6191. Resolution: Fixed Fixed via master branch: f54b9bb8dd7c77ba159f71feaca6f8475c15b535 > Improve passing the d

[jira] [Closed] (HUDI-8009) Optimize the code of HoodieTable#getPendingCommitTimeline

2024-07-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8009. Resolution: Fixed Fixed via master branch: 4f50fde3da345bea989b26ae1e7944d94e426dfd > Optimize the code of

[jira] [Updated] (HUDI-8009) Optimize the code of HoodieTable#getPendingCommitTimeline

2024-07-23 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8009: - Fix Version/s: 1.0.0 > Optimize the code of HoodieTable#getPendingCommitTimeline > ---

[jira] [Closed] (HUDI-8006) glue sync may not update columns

2024-07-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8006. Resolution: Fixed Fixed via master branch: a0bae464592f44cfa7463d9d09aa7235fd63b5fe > glue sync may not upd

[jira] [Updated] (HUDI-8006) glue sync may not update columns

2024-07-22 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8006: - Fix Version/s: 1.1.0 > glue sync may not update columns > - > >

[jira] [Updated] (HUDI-8007) ShowInvalidParquetProcedure support delete by parameter

2024-07-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-8007: - Fix Version/s: 1.0.0 > ShowInvalidParquetProcedure support delete by parameter > -

[jira] [Closed] (HUDI-8007) ShowInvalidParquetProcedure support delete by parameter

2024-07-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8007. Resolution: Fixed Fixed via master branch: 2af5f67bac39dab6814e790cd6090189e5cbc3fd > ShowInvalidParquetPro

[jira] [Closed] (HUDI-8001) Insert overwrite failed due to missing 'path' property when using Spark 3.5.1 and Hudi 1.0.0

2024-07-20 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-8001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-8001. Resolution: Fixed Fixed via master branch: 3349839f0297b6b0b0933a51f9e2a9e793f03642 > Insert overwrite fail

  1   2   3   4   5   6   7   8   9   10   >