[jira] [Updated] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout
[ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-3356: -- Sprint: Hudi-Sprint-Jan-31, Hudi-Sprint-Feb-7, Hudi-Sprint-Feb-14, Hudi-Sprint-Feb-22 (was: Hudi-Sprint-Jan-31, Hudi-Sprint-Feb-7, Hudi-Sprint-Feb-14) > Conversion of write stats to metadata index records should use HoodieData > throughout > > > Key: HUDI-3356 > URL: https://issues.apache.org/jira/browse/HUDI-3356 > Project: Apache Hudi > Issue Type: Task > Components: writer-core >Reporter: Manoj Govindassamy >Assignee: Manoj Govindassamy >Priority: Blocker > Labels: pull-request-available > Fix For: 0.11.0 > > > HoodieMetadataTableUtil convertMetadataToRecords() converts all write stats > to metadata index records to List of HoodieRecords before passing on them to > engine specific commit() to prep records. This can OOM driver. We need to use > HoodieData throughout. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout
[ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-3356: - Sprint: Hudi-Sprint-Jan-31, Hudi-Sprint-Feb-7, Hudi-Sprint-Feb-14 (was: Hudi-Sprint-Jan-31, Hudi-Sprint-Feb-7) > Conversion of write stats to metadata index records should use HoodieData > throughout > > > Key: HUDI-3356 > URL: https://issues.apache.org/jira/browse/HUDI-3356 > Project: Apache Hudi > Issue Type: Task > Components: writer-core >Reporter: Manoj Govindassamy >Assignee: Manoj Govindassamy >Priority: Blocker > Labels: pull-request-available > Fix For: 0.11.0 > > > HoodieMetadataTableUtil convertMetadataToRecords() converts all write stats > to metadata index records to List of HoodieRecords before passing on them to > engine specific commit() to prep records. This can OOM driver. We need to use > HoodieData throughout. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout
[ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-3356: -- Sprint: Hudi-Sprint-Jan-31, Hudi-Sprint-Feb-7 (was: Hudi-Sprint-Jan-31) > Conversion of write stats to metadata index records should use HoodieData > throughout > > > Key: HUDI-3356 > URL: https://issues.apache.org/jira/browse/HUDI-3356 > Project: Apache Hudi > Issue Type: Task > Components: writer-core >Reporter: Manoj Govindassamy >Assignee: Manoj Govindassamy >Priority: Blocker > Labels: pull-request-available > Fix For: 0.11.0 > > > HoodieMetadataTableUtil convertMetadataToRecords() converts all write stats > to metadata index records to List of HoodieRecords before passing on them to > engine specific commit() to prep records. This can OOM driver. We need to use > HoodieData throughout. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout
[ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-3356: - Story Points: 1 (was: 2) > Conversion of write stats to metadata index records should use HoodieData > throughout > > > Key: HUDI-3356 > URL: https://issues.apache.org/jira/browse/HUDI-3356 > Project: Apache Hudi > Issue Type: Task > Components: writer-core >Reporter: Manoj Govindassamy >Assignee: Manoj Govindassamy >Priority: Blocker > Labels: pull-request-available > Fix For: 0.11.0 > > > HoodieMetadataTableUtil convertMetadataToRecords() converts all write stats > to metadata index records to List of HoodieRecords before passing on them to > engine specific commit() to prep records. This can OOM driver. We need to use > HoodieData throughout. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout
[ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-3356: -- Status: Patch Available (was: In Progress) > Conversion of write stats to metadata index records should use HoodieData > throughout > > > Key: HUDI-3356 > URL: https://issues.apache.org/jira/browse/HUDI-3356 > Project: Apache Hudi > Issue Type: Task > Components: writer-core >Reporter: Manoj Govindassamy >Assignee: Manoj Govindassamy >Priority: Blocker > Labels: pull-request-available > Fix For: 0.11.0 > > > HoodieMetadataTableUtil convertMetadataToRecords() converts all write stats > to metadata index records to List of HoodieRecords before passing on them to > engine specific commit() to prep records. This can OOM driver. We need to use > HoodieData throughout. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout
[ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-3356: - Reviewers: Sagar Sumit > Conversion of write stats to metadata index records should use HoodieData > throughout > > > Key: HUDI-3356 > URL: https://issues.apache.org/jira/browse/HUDI-3356 > Project: Apache Hudi > Issue Type: Task > Components: writer-core >Reporter: Manoj Govindassamy >Assignee: Manoj Govindassamy >Priority: Blocker > Labels: pull-request-available > Fix For: 0.11.0 > > > HoodieMetadataTableUtil convertMetadataToRecords() converts all write stats > to metadata index records to List of HoodieRecords before passing on them to > engine specific commit() to prep records. This can OOM driver. We need to use > HoodieData throughout. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout
[ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-3356: - Labels: pull-request-available (was: ) > Conversion of write stats to metadata index records should use HoodieData > throughout > > > Key: HUDI-3356 > URL: https://issues.apache.org/jira/browse/HUDI-3356 > Project: Apache Hudi > Issue Type: Task > Components: writer-core >Reporter: Manoj Govindassamy >Assignee: Manoj Govindassamy >Priority: Blocker > Labels: pull-request-available > Fix For: 0.11.0 > > > HoodieMetadataTableUtil convertMetadataToRecords() converts all write stats > to metadata index records to List of HoodieRecords before passing on them to > engine specific commit() to prep records. This can OOM driver. We need to use > HoodieData throughout. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout
[ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HUDI-3356: - Status: In Progress (was: Open) > Conversion of write stats to metadata index records should use HoodieData > throughout > > > Key: HUDI-3356 > URL: https://issues.apache.org/jira/browse/HUDI-3356 > Project: Apache Hudi > Issue Type: Task > Components: writer-core >Reporter: Manoj Govindassamy >Assignee: Manoj Govindassamy >Priority: Blocker > Fix For: 0.11.0 > > > HoodieMetadataTableUtil convertMetadataToRecords() converts all write stats > to metadata index records to List of HoodieRecords before passing on them to > engine specific commit() to prep records. This can OOM driver. We need to use > HoodieData throughout. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HUDI-3356) Conversion of write stats to metadata index records should use HoodieData throughout
[ https://issues.apache.org/jira/browse/HUDI-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Mahindra updated HUDI-3356: -- Sprint: Hudi-Sprint-Jan-31 > Conversion of write stats to metadata index records should use HoodieData > throughout > > > Key: HUDI-3356 > URL: https://issues.apache.org/jira/browse/HUDI-3356 > Project: Apache Hudi > Issue Type: Task > Components: writer-core >Reporter: Manoj Govindassamy >Assignee: Manoj Govindassamy >Priority: Blocker > Fix For: 0.11.0 > > > HoodieMetadataTableUtil convertMetadataToRecords() converts all write stats > to metadata index records to List of HoodieRecords before passing on them to > engine specific commit() to prep records. This can OOM driver. We need to use > HoodieData throughout. -- This message was sent by Atlassian Jira (v8.20.1#820001)
