[
https://issues.apache.org/jira/browse/HUDI-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
du.junling updated HUDI-4237:
-----------------------------
Description:
Create a non-partition hudi table in Spark，it will store
hoodie.table.partition.fields with an empty value in hiveMetastore.
This is unexpected behavior.
Steps to reproduce the behavior:
# Create a non-partition hudi table in Spark
create table hudi_mor_tbl (
id int,
name string,
price double,
ts bigint
) using hudi
tblproperties (
type = 'mor',
primaryKey = 'id',
preCombineField = 'ts'
） # insert data one row to it.
insert into hudi_mor_tbl select 1, 'a1', 20, 1000; # cat hoodie.properties in
table's base path, it include partition.fields key with an empty value
hoodie.table.partition.fields=
# check spark.sql.sources.schema.partCol.0 that stored in table TABLE_PARAMS
of the HiveMetaStore .
| 50 | spark.sql.sources.schema.partCol.0 |it has a value "".
*Expected behavior*
this is no hoodie.table.partition.fields in hoodie.properties and
spark.sql.sources.schema.partCol.0 in HiveMetastore
*Environment Description*
* Hudi version : 0.10.0
* Spark version : 3.2.1
* Hive version : 3.1.2
* Hadoop version : 3.3.1
* Storage (HDFS/S3/GCS..) : HDFS
* Running on Docker? (yes/no) : no
> spark.sql.sources.schema.partCol.0 is non-empty in HiveMetaStore when create
> non-partition hudi table in Spark
> --------------------------------------------------------------------------------------------------------------
>
> Key: HUDI-4237
> URL: https://issues.apache.org/jira/browse/HUDI-4237
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: du.junling
> Priority: Major
>
> Create a non-partition hudi table in Spark，it will store
> hoodie.table.partition.fields with an empty value in hiveMetastore.
> This is unexpected behavior.
>
> Steps to reproduce the behavior:
> # Create a non-partition hudi table in Spark
> create table hudi_mor_tbl (
> id int,
> name string,
> price double,
> ts bigint
> ) using hudi
> tblproperties (
> type = 'mor',
> primaryKey = 'id',
> preCombineField = 'ts'
> ） # insert data one row to it.
> insert into hudi_mor_tbl select 1, 'a1', 20, 1000; # cat hoodie.properties in
> table's base path, it include partition.fields key with an empty value
> hoodie.table.partition.fields=
> # check spark.sql.sources.schema.partCol.0 that stored in table TABLE_PARAMS
> of the HiveMetaStore .
> | 50 | spark.sql.sources.schema.partCol.0 |it has a value "".
> *Expected behavior*
> this is no hoodie.table.partition.fields in hoodie.properties and
> spark.sql.sources.schema.partCol.0 in HiveMetastore
> *Environment Description*
> * Hudi version : 0.10.0
> * Spark version : 3.2.1
> * Hive version : 3.1.2
> * Hadoop version : 3.3.1
> * Storage (HDFS/S3/GCS..) : HDFS
> * Running on Docker? (yes/no) : no
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
