[
https://issues.apache.org/jira/browse/HUDI-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17922831#comment-17922831
]
Davis Zhang commented on HUDI-8818:
-----------------------------------
[~yc2523] the issue is when the table is created by the data source write API,
there is no database set, so in the hoodie.properties you would see the
attribute is empty
{code:java}
hoodie.database.name= {code}
This makes sense as data source API does not require a database in order to
operate on the table, this is not mandatory.
On the other hand, SQL must use a database when interacting with tables, so in
your SQL session, we connected to database yxchang_wolf, for the first insert
it tries to instantiate the catalog entries for it. On the relevant code path
it does validation between writer setting and the table config. Then it find
the current_database() does not match what the table config says. So we got the
error
xchang_nolforg.apache.hudi.exception.HoodieException: Config conflict(key
current value existing value):hoodie.database.name: yxchang_nolf
you can see "current value" is followed by nothing.
The workaround is if user intend to work on table created by data source API,
data source API has to set the right database intended for SQL. We should at
least follow up with doc update on this.
Once you set the following for the first time you write to the table (which
leads to table creation)
{code:java}
.option("hoodie.database.name", "yxchang_nolf") {code}
this issue should be gone.
> Hudi 1.0 cannot use SQL to write older versioned Hudi table
> -----------------------------------------------------------
>
> Key: HUDI-8818
> URL: https://issues.apache.org/jira/browse/HUDI-8818
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Shawn Chang
> Assignee: Davis Zhang
> Priority: Blocker
> Fix For: 1.0.1
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When using Hudi 1.0 + Spark SQL to write a table created by Hudi 0.14 + Spark
> 3.5.0 using Spark dataframe API, we noticed that the INSERT query would fail
> to database config conflict
> {code:java}
> Config conflict(key current value existing value):
> hoodie.database.name: yxchang_nolf
> org.apache.hudi.exception.HoodieException: Config conflict(key current
> value existing value):
> hoodie.database.name: yxchang_nolf
> at
> org.apache.hudi.HoodieWriterUtils$.validateTableConfig(HoodieWriterUtils.scala:256)
> at
> org.apache.hudi.HoodieSparkSqlWriterInternal.writeInternal(HoodieSparkSqlWriter.scala:245)
> at
> org.apache.hudi.HoodieSparkSqlWriterInternal.$anonfun$write$1(HoodieSparkSqlWriter.scala:190)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
