[jira] [Assigned] (HUDI-5948) Apply maven CI friendly version

2023-03-17 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-5948: Assignee: Yann Byron > Apply maven CI friendly version > --- > >

[jira] [Created] (HUDI-5948) Apply maven CI friendly version

2023-03-17 Thread Yann Byron (Jira)
Yann Byron created HUDI-5948: Summary: Apply maven CI friendly version Key: HUDI-5948 URL: https://issues.apache.org/jira/browse/HUDI-5948 Project: Apache Hudi Issue Type: Improvement C

[jira] [Created] (HUDI-5702) Avoid to write useless CDC when compaction

2023-02-04 Thread Yann Byron (Jira)
Yann Byron created HUDI-5702: Summary: Avoid to write useless CDC when compaction Key: HUDI-5702 URL: https://issues.apache.org/jira/browse/HUDI-5702 Project: Apache Hudi Issue Type: Improvement

[jira] [Created] (HUDI-5701) Fail to compact mor table when enable cdc

2023-02-04 Thread Yann Byron (Jira)
Yann Byron created HUDI-5701: Summary: Fail to compact mor table when enable cdc Key: HUDI-5701 URL: https://issues.apache.org/jira/browse/HUDI-5701 Project: Apache Hudi Issue Type: Bug

[jira] [Assigned] (HUDI-5634) Imporve cdc-related codes

2023-01-27 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-5634: Assignee: Yann Byron > Imporve cdc-related codes > - > > Ke

[jira] [Created] (HUDI-5634) Imporve cdc-related codes

2023-01-27 Thread Yann Byron (Jira)
Yann Byron created HUDI-5634: Summary: Imporve cdc-related codes Key: HUDI-5634 URL: https://issues.apache.org/jira/browse/HUDI-5634 Project: Apache Hudi Issue Type: Improvement Compone

[jira] [Assigned] (HUDI-5629) Clean cdc log files when disable cdc

2023-01-27 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-5629: Assignee: Yann Byron > Clean cdc log files when disable cdc >

[jira] [Updated] (HUDI-5629) Clean cdc log files when disable cdc

2023-01-27 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-5629: - Reviewers: Raymond Xu > Clean cdc log files when disable cdc > > >

[jira] [Created] (HUDI-5629) Clean cdc log files when disable cdc

2023-01-27 Thread Yann Byron (Jira)
Yann Byron created HUDI-5629: Summary: Clean cdc log files when disable cdc Key: HUDI-5629 URL: https://issues.apache.org/jira/browse/HUDI-5629 Project: Apache Hudi Issue Type: Improvement

[jira] [Assigned] (HUDI-5340) Spark SQL supports Table-Valued Function to extend more query syntax

2023-01-27 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-5340: Assignee: Yann Byron > Spark SQL supports Table-Valued Function to extend more query syntax > -

[jira] [Created] (HUDI-5340) Spark SQL supports Table-Valued Function to extend more query syntax

2022-12-06 Thread Yann Byron (Jira)
Yann Byron created HUDI-5340: Summary: Spark SQL supports Table-Valued Function to extend more query syntax Key: HUDI-5340 URL: https://issues.apache.org/jira/browse/HUDI-5340 Project: Apache Hudi

[jira] [Created] (HUDI-5279) move logic for deleting active instant to HoodieActiveTimeline

2022-11-25 Thread Yann Byron (Jira)
Yann Byron created HUDI-5279: Summary: move logic for deleting active instant to HoodieActiveTimeline Key: HUDI-5279 URL: https://issues.apache.org/jira/browse/HUDI-5279 Project: Apache Hudi Iss

[jira] [Created] (HUDI-5241) Optimize HoodieDefaultTimeline API

2022-11-17 Thread Yann Byron (Jira)
Yann Byron created HUDI-5241: Summary: Optimize HoodieDefaultTimeline API Key: HUDI-5241 URL: https://issues.apache.org/jira/browse/HUDI-5241 Project: Apache Hudi Issue Type: Improvement

[jira] [Assigned] (HUDI-5082) Improve the cdc log file name format

2022-10-23 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-5082: Assignee: Yann Byron > Improve the cdc log file name format >

[jira] [Created] (HUDI-5082) Improve the cdc log file name format

2022-10-23 Thread Yann Byron (Jira)
Yann Byron created HUDI-5082: Summary: Improve the cdc log file name format Key: HUDI-5082 URL: https://issues.apache.org/jira/browse/HUDI-5082 Project: Apache Hudi Issue Type: Improvement

[jira] [Resolved] (HUDI-4949) Optimize cdc read to avoid problems that caused by reusing buffer underlying the Row

2022-10-07 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron resolved HUDI-4949. -- > Optimize cdc read to avoid problems that caused by reusing buffer underlying > the Row >

[jira] [Updated] (HUDI-4949) Optimize cdc read to avoid problems that caused by reusing buffer underlying the Row

2022-10-07 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-4949: - Fix Version/s: 0.13.0 > Optimize cdc read to avoid problems that caused by reusing buffer underlying > th

[jira] [Assigned] (HUDI-4949) Optimize cdc read to avoid problems that caused by reusing buffer underlying the Row

2022-10-07 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-4949: Assignee: Yann Byron > Optimize cdc read to avoid problems that caused by reusing buffer underlying

[jira] [Closed] (HUDI-4915) Spark Avro SerDe returns wrong result upon multiple calls

2022-10-07 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-4915. Resolution: Won't Fix > Spark Avro SerDe returns wrong result upon multiple calls >

[jira] [Created] (HUDI-4949) Optimize cdc read to avoid problems that caused by reusing buffer underlying the Row

2022-09-28 Thread Yann Byron (Jira)
Yann Byron created HUDI-4949: Summary: Optimize cdc read to avoid problems that caused by reusing buffer underlying the Row Key: HUDI-4949 URL: https://issues.apache.org/jira/browse/HUDI-4949 Project: Apa

[jira] [Created] (HUDI-4948) Support flush and rollover for CDC Write

2022-09-28 Thread Yann Byron (Jira)
Yann Byron created HUDI-4948: Summary: Support flush and rollover for CDC Write Key: HUDI-4948 URL: https://issues.apache.org/jira/browse/HUDI-4948 Project: Apache Hudi Issue Type: Improvement

[jira] [Updated] (HUDI-4915) Improve Spark Avro SerDe

2022-09-25 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-4915: - Description: Currently, Spark Avro serializer/deserializer has a bug that it will return the same object

[jira] [Created] (HUDI-4915) Improve Spark Avro SerDe

2022-09-25 Thread Yann Byron (Jira)
Yann Byron created HUDI-4915: Summary: Improve Spark Avro SerDe Key: HUDI-4915 URL: https://issues.apache.org/jira/browse/HUDI-4915 Project: Apache Hudi Issue Type: Improvement Componen

[jira] [Created] (HUDI-4887) Use Avro as the persisted cdc data format instead of Json

2022-09-20 Thread Yann Byron (Jira)
Yann Byron created HUDI-4887: Summary: Use Avro as the persisted cdc data format instead of Json Key: HUDI-4887 URL: https://issues.apache.org/jira/browse/HUDI-4887 Project: Apache Hudi Issue Typ

[jira] [Created] (HUDI-4822) Extract the baseFile and logFIles from HoodieDeltaWriteStat in the right way

2022-09-09 Thread Yann Byron (Jira)
Yann Byron created HUDI-4822: Summary: Extract the baseFile and logFIles from HoodieDeltaWriteStat in the right way Key: HUDI-4822 URL: https://issues.apache.org/jira/browse/HUDI-4822 Project: Apache Hudi

[jira] [Resolved] (HUDI-4703) use the corresponding schema (not the latest schema) to response the time travel query

2022-09-06 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron resolved HUDI-4703. -- > use the corresponding schema (not the latest schema) to response the time > travel query > -

[jira] [Updated] (HUDI-4703) use the corresponding schema (not the latest schema) to response the time travel query

2022-09-06 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-4703: - Status: In Progress (was: Open) > use the corresponding schema (not the latest schema) to response the ti

[jira] [Updated] (HUDI-4703) use the corresponding schema (not the latest schema) to response the time travel query

2022-09-06 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-4703: - Status: Patch Available (was: In Progress) > use the corresponding schema (not the latest schema) to resp

[jira] [Assigned] (HUDI-4703) use the corresponding schema (not the latest schema) to response the time travel query

2022-09-06 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-4703: Assignee: Yann Byron > use the corresponding schema (not the latest schema) to response the time >

[jira] [Created] (HUDI-4705) Support Write-on-compaction mode when query cdc on MOR tables

2022-08-24 Thread Yann Byron (Jira)
Yann Byron created HUDI-4705: Summary: Support Write-on-compaction mode when query cdc on MOR tables Key: HUDI-4705 URL: https://issues.apache.org/jira/browse/HUDI-4705 Project: Apache Hudi Issu

[jira] [Created] (HUDI-4703) use the corresponding schema (not the latest schema) to response the time travel query

2022-08-23 Thread Yann Byron (Jira)
Yann Byron created HUDI-4703: Summary: use the corresponding schema (not the latest schema) to response the time travel query Key: HUDI-4703 URL: https://issues.apache.org/jira/browse/HUDI-4703 Project:

[jira] [Closed] (HUDI-3690) use all the coming records to update the existing

2022-08-04 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-3690. Resolution: Duplicate > use all the coming records to update the existing >

[jira] [Commented] (HUDI-3690) use all the coming records to update the existing

2022-08-04 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575312#comment-17575312 ] Yann Byron commented on HUDI-3690: -- [~Pratyaksh] yes. i close this, and link to that one.

[jira] [Assigned] (HUDI-4514) optimize CTAS or saveAsTable in different modes

2022-08-01 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-4514: Assignee: Yann Byron > optimize CTAS or saveAsTable in different modes > --

[jira] [Created] (HUDI-4514) optimize CTAS or saveAsTable in different modes

2022-08-01 Thread Yann Byron (Jira)
Yann Byron created HUDI-4514: Summary: optimize CTAS or saveAsTable in different modes Key: HUDI-4514 URL: https://issues.apache.org/jira/browse/HUDI-4514 Project: Apache Hudi Issue Type: Improve

[jira] [Created] (HUDI-4503) support for parsing identifier with catalog

2022-07-29 Thread Yann Byron (Jira)
Yann Byron created HUDI-4503: Summary: support for parsing identifier with catalog Key: HUDI-4503 URL: https://issues.apache.org/jira/browse/HUDI-4503 Project: Apache Hudi Issue Type: Improvement

[jira] [Assigned] (HUDI-4503) support for parsing identifier with catalog

2022-07-29 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-4503: Assignee: Yann Byron > support for parsing identifier with catalog > --

[jira] [Updated] (HUDI-4494) keep the fields' order when data is written out of order

2022-07-28 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-4494: - Fix Version/s: 0.12.0 > keep the fields' order when data is written out of order > --

[jira] [Assigned] (HUDI-4487) support to create ro/rt table by spark sql

2022-07-28 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-4487: Assignee: Yann Byron > support to create ro/rt table by spark sql > ---

[jira] [Updated] (HUDI-4487) support to create ro/rt table by spark sql

2022-07-28 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-4487: - Fix Version/s: 0.12.0 > support to create ro/rt table by spark sql > -

[jira] [Updated] (HUDI-4494) keep the fields' order when data is written out of order

2022-07-28 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-4494: - Summary: keep the fields' order when data is written out of order (was: Sort data by the schema order wh

[jira] [Created] (HUDI-4494) Sort data by the schema order when call "Insert"

2022-07-28 Thread Yann Byron (Jira)
Yann Byron created HUDI-4494: Summary: Sort data by the schema order when call "Insert" Key: HUDI-4494 URL: https://issues.apache.org/jira/browse/HUDI-4494 Project: Apache Hudi Issue Type: Improv

[jira] [Updated] (HUDI-4487) support to create ro/rt table by spark sql

2022-07-27 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-4487: - Description: Currently, if the ro/rt table is missing, user just create these only by hudi cli, and provi

[jira] [Updated] (HUDI-4487) support to create ro/rt table by spark sql

2022-07-27 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-4487: - Description: Currently, if the ro/rt table is missing, user just create these only by hudi cli, and provi

[jira] [Created] (HUDI-4487) support to create ro/rt table by spark sql

2022-07-27 Thread Yann Byron (Jira)
Yann Byron created HUDI-4487: Summary: support to create ro/rt table by spark sql Key: HUDI-4487 URL: https://issues.apache.org/jira/browse/HUDI-4487 Project: Apache Hudi Issue Type: Improvement

[jira] [Created] (HUDI-4486) validate the coming configs and table name when create an external table

2022-07-27 Thread Yann Byron (Jira)
Yann Byron created HUDI-4486: Summary: validate the coming configs and table name when create an external table Key: HUDI-4486 URL: https://issues.apache.org/jira/browse/HUDI-4486 Project: Apache Hudi

[jira] [Assigned] (HUDI-1329) Support async compaction in spark DF write()

2022-04-20 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-1329: Assignee: (was: Yann Byron) > Support async compaction in spark DF write() > --

[jira] [Created] (HUDI-3690) use all the coming records to update the existing

2022-03-22 Thread Yann Byron (Jira)
Yann Byron created HUDI-3690: Summary: use all the coming records to update the existing Key: HUDI-3690 URL: https://issues.apache.org/jira/browse/HUDI-3690 Project: Apache Hudi Issue Type: New F

[jira] [Assigned] (HUDI-3232) support reload timeline Incrementally

2022-03-10 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-3232: Assignee: (was: Yann Byron) > support reload timeline Incrementally > -

[jira] [Assigned] (HUDI-3232) support reload timeline Incrementally

2022-03-10 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-3232: Assignee: Yann Byron > support reload timeline Incrementally >

[jira] [Updated] (HUDI-3232) support reload timeline Incrementally

2022-03-10 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3232: - Labels: (was: pull-request-available) > support reload timeline Incrementally >

[jira] [Closed] (HUDI-1689) Support Multipath query for HoodieFileIndex

2022-03-10 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-1689. Resolution: Won't Do > Support Multipath query for HoodieFileIndex > ---

[jira] [Commented] (HUDI-3214) Optimize auto partition in spark

2022-02-28 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499274#comment-17499274 ] Yann Byron commented on HUDI-3214: -- [~xushiyan] [~shivnarayan] I think no new configs or

[jira] [Closed] (HUDI-1869) Upgrading Spark3 To 3.1

2022-02-23 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-1869. > Upgrading Spark3 To 3.1 > --- > > Key: HUDI-1869 > URL: ht

[jira] [Closed] (HUDI-1832) Support Hoodie CLI Command In Spark SQL

2022-02-23 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-1832. Resolution: Duplicate These will be supported by call produce command. > Support Hoodie CLI Command In Spar

[jira] [Closed] (HUDI-2482) Support drop partitions SQL

2022-02-23 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-2482. > Support drop partitions SQL > --- > > Key: HUDI-2482 >

[jira] [Closed] (HUDI-2456) Support show partitions SQL

2022-02-23 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-2456. > Support show partitions SQL > --- > > Key: HUDI-2456 >

[jira] [Closed] (HUDI-2538) Persist configs to hoodie.properties on the first write

2022-02-23 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-2538. > Persist configs to hoodie.properties on the first write >

[jira] [Comment Edited] (HUDI-3201) Make partition auto discovery configurable

2022-02-23 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497209#comment-17497209 ] Yann Byron edited comment on HUDI-3201 at 2/24/22, 6:49 AM: h4

[jira] [Commented] (HUDI-3201) Make partition auto discovery configurable

2022-02-23 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17497209#comment-17497209 ] Yann Byron commented on HUDI-3201: -- h4. `hoodie.datasource.write.partitionpath.urlencode

[jira] [Closed] (HUDI-3201) Make partition auto discovery configurable

2022-02-23 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-3201. Resolution: Fixed > Make partition auto discovery configurable > --

[jira] [Closed] (HUDI-3202) Add keygen to support partition discovery

2022-02-23 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-3202. Resolution: Won't Do not necessary to add another keygen for this. this behavior can be controlled by `hood

[jira] [Closed] (HUDI-3423) Upgrade Spark to 3.2.1

2022-02-22 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron closed HUDI-3423. Resolution: Fixed > Upgrade Spark to 3.2.1 > -- > > Key: HUDI-3423 >

[jira] [Assigned] (HUDI-3423) Upgrade Spark to 3.2.1

2022-02-22 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-3423: Assignee: Yann Byron > Upgrade Spark to 3.2.1 > -- > > Key: HUD

[jira] [Created] (HUDI-3478) support Changing Data Capture for Hudi

2022-02-22 Thread Yann Byron (Jira)
Yann Byron created HUDI-3478: Summary: support Changing Data Capture for Hudi Key: HUDI-3478 URL: https://issues.apache.org/jira/browse/HUDI-3478 Project: Apache Hudi Issue Type: New Feature

[jira] [Assigned] (HUDI-3478) support Changing Data Capture for Hudi

2022-02-22 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-3478: Assignee: Yann Byron > support Changing Data Capture for Hudi > ---

[jira] [Created] (HUDI-3423) Upgrade Spark to 3.2.1

2022-02-14 Thread Yann Byron (Jira)
Yann Byron created HUDI-3423: Summary: Upgrade Spark to 3.2.1 Key: HUDI-3423 URL: https://issues.apache.org/jira/browse/HUDI-3423 Project: Apache Hudi Issue Type: Improvement Components

[jira] [Assigned] (HUDI-3200) File Index config affects partition fields shown in printSchema results

2022-02-11 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-3200: Assignee: Yann Byron > File Index config affects partition fields shown in printSchema results > --

[jira] [Updated] (HUDI-3402) Set hoodie.parquet.outputtimestamptype to TIMESTAMP_MICROS by default

2022-02-10 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3402: - Status: In Progress (was: Open) > Set hoodie.parquet.outputtimestamptype to TIMESTAMP_MICROS by default >

[jira] [Updated] (HUDI-3402) Set hoodie.parquet.outputtimestamptype to TIMESTAMP_MICROS by default

2022-02-10 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3402: - Status: Patch Available (was: In Progress) > Set hoodie.parquet.outputtimestamptype to TIMESTAMP_MICROS b

[jira] [Assigned] (HUDI-3402) Set hoodie.parquet.outputtimestamptype to TIMESTAMP_MICROS by default

2022-02-10 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-3402: Assignee: Yann Byron > Set hoodie.parquet.outputtimestamptype to TIMESTAMP_MICROS by default >

[jira] [Updated] (HUDI-3333) getNestedFieldVal breaks with Spark 3.2

2022-02-10 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-: - Status: Patch Available (was: In Progress) > getNestedFieldVal breaks with Spark 3.2 > --

[jira] [Updated] (HUDI-3200) File Index config affects partition fields shown in printSchema results

2022-02-09 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3200: - Status: In Progress (was: Open) > File Index config affects partition fields shown in printSchema results

[jira] [Updated] (HUDI-3338) Use custom relation instead of HadoopFsRelation

2022-02-09 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3338: - Status: Patch Available (was: In Progress) > Use custom relation instead of HadoopFsRelation > --

[jira] [Updated] (HUDI-2610) Fix Spark version info for hudi table CTAS from another hudi table

2022-02-09 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-2610: - Status: In Progress (was: Open) > Fix Spark version info for hudi table CTAS from another hudi table > --

[jira] [Updated] (HUDI-3338) Use custom relation instead of HadoopFsRelation

2022-02-09 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3338: - Status: In Progress (was: Open) > Use custom relation instead of HadoopFsRelation > -

[jira] [Updated] (HUDI-3204) spark on TimestampBasedKeyGenerator has no result when query by partition column

2022-02-09 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3204: - Status: Patch Available (was: In Progress) > spark on TimestampBasedKeyGenerator has no result when query

[jira] [Assigned] (HUDI-3333) getNestedFieldVal breaks with Spark 3.2

2022-02-09 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-: Assignee: Yann Byron > getNestedFieldVal breaks with Spark 3.2 > --

[jira] [Updated] (HUDI-3333) getNestedFieldVal breaks with Spark 3.2

2022-02-09 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-: - Status: In Progress (was: Open) > getNestedFieldVal breaks with Spark 3.2 > -

[jira] [Created] (HUDI-3403) Manage immutable hudi Configurations

2022-02-09 Thread Yann Byron (Jira)
Yann Byron created HUDI-3403: Summary: Manage immutable hudi Configurations Key: HUDI-3403 URL: https://issues.apache.org/jira/browse/HUDI-3403 Project: Apache Hudi Issue Type: Improvement

[jira] [Created] (HUDI-3402) Set hoodie.parquet.outputtimestamptype to TIMESTAMP_MICROS by default

2022-02-09 Thread Yann Byron (Jira)
Yann Byron created HUDI-3402: Summary: Set hoodie.parquet.outputtimestamptype to TIMESTAMP_MICROS by default Key: HUDI-3402 URL: https://issues.apache.org/jira/browse/HUDI-3402 Project: Apache Hudi

[jira] [Comment Edited] (HUDI-2972) Support different Spark internal Timestamp and Date types

2022-02-05 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487436#comment-17487436 ] Yann Byron edited comment on HUDI-2972 at 2/5/22, 9:01 AM: --- [~ry

[jira] [Commented] (HUDI-2972) Support different Spark internal Timestamp and Date types

2022-02-05 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487436#comment-17487436 ] Yann Byron commented on HUDI-2972: -- [~ryanpife] can you retry by hudi master branch which

[jira] [Comment Edited] (HUDI-3314) support merge into with no-pk condition

2022-02-03 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486800#comment-17486800 ] Yann Byron edited comment on HUDI-3314 at 2/4/22, 3:33 AM: --- [~sh

[jira] [Commented] (HUDI-3314) support merge into with no-pk condition

2022-02-03 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486800#comment-17486800 ] Yann Byron commented on HUDI-3314: -- [~shivnarayan] This is a little complicated to implem

[jira] [Updated] (HUDI-3338) Use custom relation instead of HadoopFsRelation

2022-01-27 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3338: - Description: For HUDI-3204, COW table and MOR table in read_optimized query mode should return the '-

[jira] [Created] (HUDI-3338) Use custom relation instead of HadoopFsRelation

2022-01-27 Thread Yann Byron (Jira)
Yann Byron created HUDI-3338: Summary: Use custom relation instead of HadoopFsRelation Key: HUDI-3338 URL: https://issues.apache.org/jira/browse/HUDI-3338 Project: Apache Hudi Issue Type: Improve

[jira] [Commented] (HUDI-3204) spark on TimestampBasedKeyGenerator has no result when query by partition column

2022-01-26 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482858#comment-17482858 ] Yann Byron commented on HUDI-3204: -- [~shivnarayan] [~taisenki]  i also agree that the re

[jira] [Comment Edited] (HUDI-3232) support reload timeline Incrementally

2022-01-24 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481615#comment-17481615 ] Yann Byron edited comment on HUDI-3232 at 1/25/22, 7:54 AM: [~

[jira] [Commented] (HUDI-3232) support reload timeline Incrementally

2022-01-24 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481615#comment-17481615 ] Yann Byron commented on HUDI-3232: -- [~shivnarayan] yep, it's very like you comment above

[jira] [Updated] (HUDI-3314) support merge into with no-pk condition

2022-01-24 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3314: - Fix Version/s: 0.12.0 (was: 0.11.0) Reviewers: Raymond Xu, Vinoth Chandar >

[jira] [Created] (HUDI-3314) support merge into with no-pk condition

2022-01-24 Thread Yann Byron (Jira)
Yann Byron created HUDI-3314: Summary: support merge into with no-pk condition Key: HUDI-3314 URL: https://issues.apache.org/jira/browse/HUDI-3314 Project: Apache Hudi Issue Type: Improvement

[jira] [Commented] (HUDI-2968) Support Delete/Update using non-pk fields

2022-01-24 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480988#comment-17480988 ] Yann Byron commented on HUDI-2968: -- [~vinoth]  if source has fields which same as target

[jira] [Updated] (HUDI-3237) ALTER TABLE column type change fails select query

2022-01-24 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3237: - Status: In Progress (was: Open) > ALTER TABLE column type change fails select query > ---

[jira] [Updated] (HUDI-3253) preferred to use table's location

2022-01-18 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3253: - Status: In Progress (was: Open) > preferred to use table's location > - >

[jira] [Updated] (HUDI-3253) preferred to use table's location

2022-01-15 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron updated HUDI-3253: - Description: When we create a hudi table with specified location which isn't the subpath of the current d

[jira] [Created] (HUDI-3253) preferred to use table's location

2022-01-15 Thread Yann Byron (Jira)
Yann Byron created HUDI-3253: Summary: preferred to use table's location Key: HUDI-3253 URL: https://issues.apache.org/jira/browse/HUDI-3253 Project: Apache Hudi Issue Type: Bug Compone

[jira] [Assigned] (HUDI-3253) preferred to use table's location

2022-01-15 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yann Byron reassigned HUDI-3253: Assignee: Yann Byron > preferred to use table's location > - > >

[jira] [Commented] (HUDI-3240) ALTER TABLE rename breaks with managed table in Spark 2.4

2022-01-15 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17476630#comment-17476630 ] Yann Byron commented on HUDI-3240: -- [~xushiyan] If this's a managed table, `rename` oper

[jira] [Comment Edited] (HUDI-3237) ALTER TABLE column type change fails select query

2022-01-13 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475182#comment-17475182 ] Yann Byron edited comment on HUDI-3237 at 1/13/22, 10:48 AM: -

  1   2   3   >