Re: [PR] [HUDI-7414] Remove redundant base path config in BQ sync [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11395: URL: https://github.com/apache/hudi/pull/11395#issuecomment-2148961247 ## CI report: * 158fcffa7e931e07c8f718138c73f84ae05a03ef Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7414] Remove redundant base path config in BQ sync [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11395: URL: https://github.com/apache/hudi/pull/11395#issuecomment-2148886370 ## CI report: * 158fcffa7e931e07c8f718138c73f84ae05a03ef Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7414] Remove redundant base path config in BQ sync [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11395: URL: https://github.com/apache/hudi/pull/11395#issuecomment-2148877843 ## CI report: * 158fcffa7e931e07c8f718138c73f84ae05a03ef UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[jira] [Updated] (HUDI-7414) Remove hoodie.gcp.bigquery.sync.base_path reference in the gcp docs

2024-06-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7414: - Labels: pull-request-available (was: ) > Remove hoodie.gcp.bigquery.sync.base_path reference in t

[PR] [HUDI-7414] Remove redundant base path config in BQ sync [hudi]

2024-06-04 Thread via GitHub
xushiyan opened a new pull request, #11395: URL: https://github.com/apache/hudi/pull/11395 ### Change Logs The base path config is not used by big query sync client. The meta client created for bigquery sync uses `hoodie.datasource.meta.sync.base.path`. Removing this avoids user conf

[jira] [Closed] (HUDI-7828) Support Flink 1.18.1

2024-06-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen closed HUDI-7828. Resolution: Fixed Fixed via master branch: dfbbd1890a0a0076eac390e05636a36c8d0da0b4 > Support Flink 1.18.1

[jira] [Updated] (HUDI-7828) Support Flink 1.18.1

2024-06-04 Thread Danny Chen (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-7828: - Fix Version/s: 1.0.0 > Support Flink 1.18.1 > > > Key: HUDI-7828 >

(hudi) branch master updated (d964895cf0b -> dfbbd1890a0)

2024-06-04 Thread danny0405
This is an automated email from the ASF dual-hosted git repository. danny0405 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git from d964895cf0b [HUDI-7713] Enforce ordering of fields during schema reconciliation (#11154) add dfbbd1890a0 [HUDI-

Re: [PR] [HUDI-7828] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
danny0405 merged PR #11394: URL: https://github.com/apache/hudi/pull/11394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apac

[jira] [Updated] (HUDI-7100) Data loss when using insert_overwrite_table with insert.drop.duplicates

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7100: Fix Version/s: (was: 0.13.2) > Data loss when using insert_overwrite_table with insert.drop.duplicates >

[jira] [Updated] (HUDI-6217) Spark reads the deleted data whose _hoodie_operation is D

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6217: Fix Version/s: (was: 0.13.2) > Spark reads the deleted data whose _hoodie_operation is D > -

[jira] [Updated] (HUDI-6946) Data Duplicates with range pruning while using hoodie.bloom.index.use.metadata

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6946: Fix Version/s: (was: 0.13.2) > Data Duplicates with range pruning while using hoodie.bloom.index.use.met

[jira] [Updated] (HUDI-6675) Clean action will delete the whole table

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6675: Fix Version/s: (was: 0.12.4) > Clean action will delete the whole table > --

[jira] [Updated] (HUDI-6675) Clean action will delete the whole table

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6675: Fix Version/s: (was: 0.13.2) > Clean action will delete the whole table > --

[jira] [Updated] (HUDI-6217) Spark reads the deleted data whose _hoodie_operation is D

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6217: Fix Version/s: 0.14.1 (was: 0.12.4) > Spark reads the deleted data whose _hoodie_oper

[jira] [Updated] (HUDI-7100) Data loss when using insert_overwrite_table with insert.drop.duplicates

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7100: Fix Version/s: (was: 0.12.4) > Data loss when using insert_overwrite_table with insert.drop.duplicates >

[jira] [Updated] (HUDI-6946) Data Duplicates with range pruning while using hoodie.bloom.index.use.metadata

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6946: Fix Version/s: (was: 0.12.4) > Data Duplicates with range pruning while using hoodie.bloom.index.use.met

[jira] [Updated] (HUDI-7675) Don't set default value for primary key when get schema from hms

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7675: Fix Version/s: 0.15.0 (was: 1.15) > Don't set default value for primary key when get

Re: [PR] [HUDI-7828] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11394: URL: https://github.com/apache/hudi/pull/11394#issuecomment-2148818171 ## CI report: * 31b3d01c11b9ee4975283e51d0ef75c9f2ebd03f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

[jira] [Closed] (HUDI-7289) Fix parameters for Big Query Sync

2024-06-04 Thread Shiyan Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shiyan Xu closed HUDI-7289. --- Resolution: Fixed > Fix parameters for Big Query Sync > - > >

[jira] [Assigned] (HUDI-7289) Fix parameters for Big Query Sync

2024-06-04 Thread Shiyan Xu (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shiyan Xu reassigned HUDI-7289: --- Assignee: nadine (was: Shiyan Xu) > Fix parameters for Big Query Sync >

Re: [PR] [HUDI-7828][branch-0.x] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11393: URL: https://github.com/apache/hudi/pull/11393#issuecomment-2148784686 ## CI report: * 31215b95ff44375962da231d98de259a46d8d016 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7763] Fix that multiple jmx reporter can exist if metadata enables [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11226: URL: https://github.com/apache/hudi/pull/11226#issuecomment-2148784413 ## CI report: * 88fc6863096ff3b499d9b1a52129643e5db6a94a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

[jira] [Assigned] (HUDI-7828) Support Flink 1.18.1

2024-06-04 Thread Shawn Chang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Chang reassigned HUDI-7828: - Assignee: Shawn Chang > Support Flink 1.18.1 > > > Key: HUDI

Re: [PR] [HUDI-7763] Fix that multiple jmx reporter can exist if metadata enables [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11226: URL: https://github.com/apache/hudi/pull/11226#issuecomment-2148778911 ## CI report: * c7b402870d78e079662a5d810f7484e39dc20f83 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23

[jira] [Closed] (HUDI-7782) Task not serializable due to DynamoDBBasedLockProvider and HiveMetastoreBasedLockProvider in clean action

2024-06-04 Thread Vova Kolmakov (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vova Kolmakov closed HUDI-7782. --- Resolution: Fixed Fixed via master branch: eb63e1ffa1a99aaf489a18bbca830d6290c04bba > Task not serial

Re: [PR] [HUDI-7828] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11394: URL: https://github.com/apache/hudi/pull/11394#issuecomment-2148741418 ## CI report: * 31b3d01c11b9ee4975283e51d0ef75c9f2ebd03f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7828][branch-0.x] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11393: URL: https://github.com/apache/hudi/pull/11393#issuecomment-2148735300 ## CI report: * 31215b95ff44375962da231d98de259a46d8d016 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24

Re: [PR] [HUDI-7828] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11394: URL: https://github.com/apache/hudi/pull/11394#issuecomment-2148735328 ## CI report: * 31b3d01c11b9ee4975283e51d0ef75c9f2ebd03f UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

Re: [PR] [HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2148734691 ## CI report: * c98242b22fb2518c0cc93c037df558037030500f UNKNOWN * 11862a3bd3b84cb12b0abcf8a399d2bfb56870b3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

Re: [PR] [HUDI-7828][branch-0.x] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #11393: URL: https://github.com/apache/hudi/pull/11393#issuecomment-2148728773 ## CI report: * 31215b95ff44375962da231d98de259a46d8d016 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run th

[jira] [Updated] (HUDI-7555) Revisit core logic and reported issues of table services

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7555: Status: Open (was: In Progress) > Revisit core logic and reported issues of table services > --

[jira] [Updated] (HUDI-7823) Simplify dependency management on exclusions

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7823: Status: In Progress (was: Open) > Simplify dependency management on exclusions > --

[jira] [Updated] (HUDI-7814) Exclude unused transitive dependencies that introduce vulnerabilities

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7814: Story Points: 5 > Exclude unused transitive dependencies that introduce vulnerabilities > --

[jira] [Updated] (HUDI-7695) Add docs on Spark 3.5 and Scala 2.13

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7695: Status: Open (was: In Progress) > Add docs on Spark 3.5 and Scala 2.13 > --

[jira] [Updated] (HUDI-7823) Simplify dependency management on exclusions

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7823: Sprint: 2024/06/03-16 > Simplify dependency management on exclusions > -

[jira] [Updated] (HUDI-7823) Simplify dependency management on exclusions

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7823: Story Points: 10 > Simplify dependency management on exclusions > --

[jira] [Updated] (HUDI-7823) Simplify dependency management on exclusions

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7823: Fix Version/s: 1.0.0 > Simplify dependency management on exclusions > --

[jira] [Assigned] (HUDI-7823) Simplify dependency management on exclusions

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo reassigned HUDI-7823: --- Assignee: Ethan Guo > Simplify dependency management on exclusions >

[jira] [Updated] (HUDI-7823) Simplify dependency management on exclusions

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7823: Fix Version/s: 0.16.0 > Simplify dependency management on exclusions > -

[PR] [HUDI-7828] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
CTTY opened a new pull request, #11394: URL: https://github.com/apache/hudi/pull/11394 ### Change Logs Support Flink 1.18.1, #11393 for branch-0.x ### Impact none ### Risk level (write none, low medium or high below) low ### Documentation Update

Re: [PR] [HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2148690599 ## CI report: * c98242b22fb2518c0cc93c037df558037030500f UNKNOWN * 36d0b151cbd361eb0dc6444e800ba65ccf4beaa7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

[PR] [HUDI-7828][branch-0.x] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
CTTY opened a new pull request, #11393: URL: https://github.com/apache/hudi/pull/11393 ### Change Logs Adapt to Flink 1.18.1's API changes: https://github.com/apache/flink/pull/18304 ### Impact None ### Risk level (write none, low medium or high below) Low

Re: [PR] [HUDI-7828] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
CTTY closed pull request #11392: [HUDI-7828] Support Flink 1.18.1 URL: https://github.com/apache/hudi/pull/11392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] [HUDI-7567] Add schema evolution to the filegroup reader [hudi]

2024-06-04 Thread via GitHub
hudi-bot commented on PR #10957: URL: https://github.com/apache/hudi/pull/10957#issuecomment-2148683919 ## CI report: * c98242b22fb2518c0cc93c037df558037030500f UNKNOWN * 36d0b151cbd361eb0dc6444e800ba65ccf4beaa7 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4

[jira] [Updated] (HUDI-7828) Support Flink 1.18.1

2024-06-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-7828: - Labels: pull-request-available (was: ) > Support Flink 1.18.1 > > >

[PR] [HUDI-7828] Support Flink 1.18.1 [hudi]

2024-06-04 Thread via GitHub
CTTY opened a new pull request, #11392: URL: https://github.com/apache/hudi/pull/11392 ### Change Logs Adapt to Flink 1.18.1's API changes: https://github.com/apache/flink/pull/18304 ### Impact None ### Risk level (write none, low medium or high below) Medi

[jira] [Updated] (HUDI-7203) Reuse Hudi CompressionCodec enums

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7203: Sprint: Sprint 2024-03-25, Sprint 2024-04-26 (was: Sprint 2024-03-25, Sprint 2024-04-26, 2024/06/03-16) >

[jira] [Updated] (HUDI-7202) Consolidate IO util methods

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7202: Sprint: Sprint 2024-03-25, Sprint 2024-04-26 (was: Sprint 2024-03-25, Sprint 2024-04-26, 2024/06/03-16) >

[jira] [Updated] (HUDI-7220) Benchmark new HFile reader

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7220: Sprint: Sprint 2024-03-25, Sprint 2024-04-26 (was: Sprint 2024-03-25, Sprint 2024-04-26, 2024/06/03-16) >

[jira] [Updated] (HUDI-7219) Implement storage and HFile block cache in the same JVM

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7219: Sprint: Sprint 2024-03-25, Sprint 2024-04-26 (was: Sprint 2024-03-25, Sprint 2024-04-26, 2024/06/03-16) >

[jira] [Updated] (HUDI-5757) Add Log Compaction to Write Operation docs

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5757: Sprint: Sprint 2023-01-31, Sprint 2023-02-14, Sprint 2023-02-28, Sprint 2023-03-14, Sprint 2024-03-25, Sprin

[jira] [Created] (HUDI-7828) Support Flink 1.18.1

2024-06-04 Thread Shawn Chang (Jira)
Shawn Chang created HUDI-7828: - Summary: Support Flink 1.18.1 Key: HUDI-7828 URL: https://issues.apache.org/jira/browse/HUDI-7828 Project: Apache Hudi Issue Type: Improvement Reporter

[jira] [Updated] (HUDI-7539) Use .compaction for compaction action consistently

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7539: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Use .compaction for compaction acti

[jira] [Updated] (HUDI-6798) Implement event-time-based merging mode in FileGroupReader

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6798: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7280) Add/Drop/Rename table properties hoodie.properties

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7280: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Add/Drop/Rename table properties ho

[jira] [Updated] (HUDI-7692) Move MDT partiiton type code in HoodieMetadataPaylaod to MetadataPartitionType

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7692: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Move MDT partiiton type code in Hoo

[jira] [Updated] (HUDI-7484) Fix partitioning style when partition is inferred from partitionBy

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7484: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7593) Create COW record reader based on HoodieStorage abstraction for Trino

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7593: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7221) Move Hudi Option class from hudi-common to hudi-io module

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7221: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7202) Consolidate IO util methods

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7202: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7695) Add docs on Spark 3.5 and Scala 2.13

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7695: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Add docs on Spark 3.5 and Scala 2.1

[jira] [Updated] (HUDI-7769) Fix Hudi CDC read with legacy parquet file format on Spark

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7769: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Fix Hudi CDC read with legacy parqu

[jira] [Updated] (HUDI-7420) Parallelize the process of constructing `logFilesMarkerPath` in CommitMetadatautils#reconcileMetadataForMissingFiles

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7420: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-1739) Standardize usage of replacecommit files across the code base

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-1739: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Standardize usage of replacecommit

[jira] [Updated] (HUDI-4732) Leverage Schema Registry for reading proto messages from kafka

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-4732: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Leverage Schema Registry for readin

[jira] [Updated] (HUDI-7585) Avoid reading log files for resolving schema for _hoodie_operation field

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7585: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7591) Implement InlineFS in HoodieStorage

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7591: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-6791) Integrate FileGroupReader with NewHoodieParquetFileFormat for Spark CDC Query

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6791: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7220) Benchmark new HFile reader

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7220: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7696) Consolidate convertFilesToPartitionStatsRecords and convertMetadataToPartitionStatsRecords

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7696: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Consolidate convertFilesToPartition

[jira] [Updated] (HUDI-7634) Rename HoodieStorage APIs

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7634: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7497) Add a global timeline mingled with active and archived instants

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7497: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7795) Fix loading of input splits from look up table reader

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7795: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Fix loading of input splits from lo

[jira] [Updated] (HUDI-7547) Simplification of archival, savepoint, cleaning interplays

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7547: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Closed] (HUDI-7807) spark-sql updates for a pk less table fails w/ partitioned table

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo closed HUDI-7807. --- Resolution: Fixed > spark-sql updates for a pk less table fails w/ partitioned table > --

[jira] [Updated] (HUDI-7596) Enable Jacoco code coverage report across multiple modules

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7596: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7028) Fix Spark Quick Start

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7028: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7157) Support filter pushdown for positional merging in Spark 3.5

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7157: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-6706) Put up a 1.0 tech specs doc, consolidating all storage changes

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6706: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-5062) Track and announce breaking changes in 1.0.0

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-5062: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7691) Move MDT partition type related logic in HoodieBackedTableMetadataWriter to MetadataPartitionType

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7691: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Move MDT partition type related log

[jira] [Updated] (HUDI-6794) Support completion-time-based file slice in FileGroupReader

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6794: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7408) LSM tree writer failed with compaction

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7408: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7527) Include instants outside commits and compaction for generating the latest instant and timeline hash for timeline server requests

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7527: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-6713) Redesign CDC workload to include partition column for partition pruning

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6713: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7395) Fix computation for metrics in HoodieMetadataMetrics

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7395: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Fix computation for metrics in Hood

[jira] [Updated] (HUDI-6596) Propose rollback implementation changes to guard against concurrent jobs

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6596: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7269) Fallback to key-based merging if there is no positions in log header

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7269: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7528) Fix RowCustomColumnsSortPartitioner to use repartition instead of coalesce

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7528: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Fix RowCustomColumnsSortPartitioner

[jira] [Updated] (HUDI-7700) Support query hint to inject indexes in query plans

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7700: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Support query hint to inject indexe

[jira] [Updated] (HUDI-7594) Create MOR record reader based on HoodieStorage abstraction for Trino

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7594: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7219) Implement storage and HFile block cache in the same JVM

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7219: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-6802) Use completion time in Spark FileIndex for listing

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6802: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-6768) Revisit HoodieRecord design and how it affects e2e row writing

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6768: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7818) Flink Table planner not loading problem

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7818: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Flink Table planner not loading pro

[jira] [Updated] (HUDI-6778) Track schema in metadata table

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-6778: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7544) Harden, Stress and Performance test the LSM timeline on cloud storage

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7544: Sprint: Sprint 2024-03-25, Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2024-03-25, Sprint 2023-04-26)

[jira] [Updated] (HUDI-7661) Create index readme to show how a new index implementation can be added

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7661: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > Create index readme to show how a n

[jira] [Updated] (HUDI-7546) TLA+ Spec for Hudi CC

2024-06-04 Thread Ethan Guo (Jira)
[ https://issues.apache.org/jira/browse/HUDI-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Guo updated HUDI-7546: Sprint: Sprint 2023-04-26, Sprint 2023-04-28 (was: Sprint 2023-04-26) > TLA+ Spec for Hudi CC > ---

  1   2   3   4   5   >