[ https://issues.apache.org/jira/browse/HUDI-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yue Zhang updated HUDI-4735: ---------------------------- Fix Version/s: 0.14.0 (was: 0.13.1) > Spark2 bundles made from master after 2022-07-23 failed to stop > --------------------------------------------------------------- > > Key: HUDI-4735 > URL: https://issues.apache.org/jira/browse/HUDI-4735 > Project: Apache Hudi > Issue Type: Bug > Components: hive, spark, spark-sql > Reporter: Vitali Obukh > Assignee: Raymond Xu > Priority: Major > Fix For: 0.14.0 > > > Daily Spark2 Structured streaming application built against the latest change > on master started failing on 2022-07-23 with the error: > {quote} > 22/07/28 10:52:50 ERROR org.apache.hudi.HoodieStreamingSink: Micro batch id=2 > threw following exception: > org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database > 'test' not found; > {quote} > Actually the database exists (created by the Hudi). Spark3 application with > the same code works fine. > Hudi options used for writing: > {code} > Map("hoodie.datasource.write.operation" -> "upsert", > "hoodie.datasource.write.table.type" -> "MERGE_ON_READ", > "hoodie.datasource.write.recordkey.field" -> s"$Meta.$Key", > "hoodie.datasource.write.precombine.field" -> s"$Meta.$Version", > "hoodie.datasource.write.payload.class" -> > "com.walmart.udps.hudi.integration.HudiTestAvroPayload", > "hoodie.compaction.payload.class" -> > "com.walmart.udps.hudi.integration.HudiTestAvroPayload", > "hoodie.datasource.write.partitionpath.field" -> TopLevelPartition, > "hoodie.datasource.write.hive_style_partitioning" -> "true", > "hoodie.datasource.hive_sync.enable" -> "true", > "hoodie.datasource.hive_sync.mode" -> "hms", > "hoodie.datasource.hive_sync.database" -> dbName(), > "hoodie.datasource.hive_sync.partition_fields" -> TopLevelPartition, > "hoodie.datasource.compaction.async.enable" -> "false" > ) > {code} > Cloud platform - Google Dataproc. > Spark version - 3.1.3. -- This message was sent by Atlassian Jira (v8.20.10#820010)