[ https://issues.apache.org/jira/browse/SPARK-24758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673931#comment-16673931 ]
Yuming Wang commented on SPARK-24758: ------------------------------------- The reason is that [SessionState.start(sessionState)|https://github.com/wangyum/spark/blob/b81e3031fd247dfb4b3e02e0a986fb4b19d00f7c/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala#L133] has created {{default}} database. The database location is {{/user/hive/warehouse}}. Then it [will not create default database|https://github.com/apache/spark/blob/f38ea00e83099a5ae8d3afdec2e896e43c2db612/sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala#L113-L118] again. > Create table wants to use /user/hive/warehouse in clean clone > ------------------------------------------------------------- > > Key: SPARK-24758 > URL: https://issues.apache.org/jira/browse/SPARK-24758 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.0 > Reporter: Bruce Robbins > Priority: Minor > > Got a clean clone of repository: > - git clone [https://github.com/apache/spark.git] spark_clean > - cd spark_clean/ > - ./build/sbt -Phive -Phive-thriftserver clean package > Ran spark-sql and tried to create a table: > - ./bin/spark-sql > - create table testit as select 1 a, 2 b; > Got error: > {noformat} > 18/07/07 13:33:20 WARN HiveMetaStore: Location: > file:/user/hive/warehouse/testit specified for non-external table:testit > 18/07/07 13:33:20 INFO FileUtils: Creating directory if it doesn't exist: > file:/user/hive/warehouse/testit > Error in query: org.apache.hadoop.hive.ql.metadata.HiveException: > MetaException(message:file:/user/hive/warehouse/testit is not a directory or > unable to create one); > {noformat} > To get things working, it seems you need to do something with dataframewriter > (after removing metastore_db, if it exists): > {noformat} > scala> spark.range(0,1).write.saveAsTable("fred") > spark.range(0,1).write.saveAsTable("fred") > 18/07/07 14:08:08 WARN ObjectStore: Version information not found in > metastore. hive.metastore.schema.verification is not enabled so recording the > schema version 1.2.0 > 18/07/07 14:08:08 WARN ObjectStore: Failed to get database default, returning > NoSuchObjectException > scala> > {noformat} > After that, create table statements work: > {noformat} > spark-sql> create table testit as select 1 a, 2 b; > create table testit as select 1 a, 2 b; > 18/07/07 14:14:40 WARN HiveMetaStore: Location: > file:<work-copy-dir>/spark-warehouse/testit specified for non-external > table:testit > 18/07/07 14:14:41 WARN ObjectStore: Failed to get database global_temp, > returning NoSuchObjectException > Time taken: 3.387 seconds > spark-sql> show tables; > show tables; > default fred false > default testit false > Time taken: 0.07 seconds, Fetched 2 row(s) > spark-sql> > {noformat} > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org