[
https://issues.apache.org/jira/browse/HUDI-8817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916203#comment-17916203
]
Davis Zhang commented on HUDI-8817:
-----------------------------------
relevant discussion with Sagar and Ethan. will post their reply here as well
please review the backward compatibility questions: * By default, with 1.x hudi
binary, what's the table version expected to be for new table creation?
** Current system behavior: version is 8.
* With 1.x hudi binary, do we allow user to create version 6 table by setting
hoodie.writer.table.version=6?
** Current system behavior: yes.
* In order to use backward compatible writer with 1.x, do we need any explicit
configurations like write.table.version=6?
** Current system behavior: No
* In 1.x, if we set both auto upgrade flag as well as the
hoodie.writer.table.version, what's the version used by table creation?
** Current system behavior: Table is created with version 6. Then on next
write it is upgraded to 8.
* For 1.x backward compatible readers to work, MDT should be disabled. (Saw
that in the migration doc as well)
> Hudi 1.0.0 cannot create older version table
> --------------------------------------------
>
> Key: HUDI-8817
> URL: https://issues.apache.org/jira/browse/HUDI-8817
> Project: Apache Hudi
> Issue Type: Sub-task
> Affects Versions: 1.0.0
> Reporter: Shawn Chang
> Assignee: Davis Zhang
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.0.1
>
> Time Spent: 3.5h
> Remaining Estimate: 8h
>
> When using Hudi 1.0 + backward writer, it still creates version 8 table which
> cannot be read by an older version Hudi reader.
> Reproduction steps:
> 1. Create the table with SQL, specifying the table version to be 6 (I've
> tested the same with DF and Hudi 1.0 also cannot create version 6 table)
> {code:java}
> CREATE TABLE hudi_14_table_sql_005 (
> event_id INT,
> event_date STRING,
> event_name STRING,
> event_ts STRING,
> event_type STRING
> ) USING hudi
> OPTIONS(
> type = 'cow', -- or 'mor'
> primaryKey = 'event_id,event_date',
> preCombileField = 'event_ts',
> hoodie.write.table.version = 6
> )
> PARTITIONED BY (event_type)
> LOCATION 's3://<some_bucket>/hudi_14_table_sql_005';
> {code}
> 2. Check `hoodie.properties` under the table's S3 location, and you should
> see `hoodie.table.version=8`
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
