[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15698424#comment-15698424 ]
Thomas Sebastian edited comment on SPARK-15034 at 11/26/16 7:54 PM: -------------------------------------------------------------------- Hi [~yhuai] Could you suggest to resolve this: I want my local mode spark metastore point to the hdfs location: /user/hive/warehouse. Could you please suggest, what solution works here. - I am using spark 2.0 in local mode and hive-1.2.1 pointing to hdfs warehouse location - Hive is already pointing to /user/hive/warehouse in hdfs location -I do not see spark not getting pointed to /user/hive/warehouse in hdfs, instead it points to a local file location-where spark is installed. -I already tried below, still not working. SparkSession.builder().appName("eg").config("spark.sql.warehouse.dir", "hdfs://localhost:9000/user/hive/warehouse").enableHiveSupport().getOrCreate() - Error thrown is: ----------------------- 16/11/27 01:06:26 WARN metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0 16/11/27 01:06:26 WARN metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException 16/11/27 01:06:31 ERROR metastore.RetryingHMSHandler: AlreadyExistsException(message:Database default already exists) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:891) was (Author: thomastechs): Hi [~yhuai] Could you suggest to resolve this: I want my local mode spark metastore point to the hdfs location: /user/hive/warehouse. Could you please suggest, what solution works here. - I am using spark 2.0 in local mode. - Hive is already pointing to /user/hive/warehouse in hdfs location -I do not see spark not getting pointed to /user/hive/warehouse in hdfs, instead it points to a local file location-where spark is installed. -I already tried below, still not working. SparkSession.builder().appName("eg").config("spark.sql.warehouse.dir", "hdfs://localhost:9000/user/hive/warehouse").enableHiveSupport().getOrCreate() - Error thrown is: ----------------------- 16/11/27 01:06:26 WARN metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0 16/11/27 01:06:26 WARN metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException 16/11/27 01:06:31 ERROR metastore.RetryingHMSHandler: AlreadyExistsException(message:Database default already exists) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:891) > Use the value of spark.sql.warehouse.dir as the warehouse location instead of > using hive.metastore.warehouse.dir > ---------------------------------------------------------------------------------------------------------------- > > Key: SPARK-15034 > URL: https://issues.apache.org/jira/browse/SPARK-15034 > Project: Spark > Issue Type: Sub-task > Components: SQL > Reporter: Yin Huai > Assignee: Yin Huai > Labels: release_notes, releasenotes > Fix For: 2.0.0 > > > Starting from Spark 2.0, spark.sql.warehouse.dir will be the conf to set > warehouse location. We will not use hive.metastore.warehouse.dir. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org