[
https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295836#comment-15295836
]
Yi Zhou commented on SPARK-15396:
---------------------------------
Hi ,
even if setting 'spark.sql.warehouse.dir' in spark-defaults.conf , it still
can't connect the my existing hive metastore database. And i also saw it
create a 'metastore_db' folder in my current working directory. Could you
please point me out what's the potential cause ? Thanks !
spark.sql.hive.metastore.version=1.1.0
spark.sql.hive.metastore.jars=/usr/lib/hive/lib/*:/usr/lib/hadoop/client/*
spark.executor.extraClassPath=/etc/hive/conf
spark.driver.extraClassPath=/etc/hive/conf
spark.yarn.jars=local:/usr/lib/spark/jars/*
spark.sql.warehouse.dir=/usr/hive/warehouse
> [Spark] [SQL] [DOC] It can't connect hive metastore database
> ------------------------------------------------------------
>
> Key: SPARK-15396
> URL: https://issues.apache.org/jira/browse/SPARK-15396
> Project: Spark
> Issue Type: Documentation
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Yi Zhou
> Assignee: Xiao Li
> Priority: Critical
> Fix For: 2.0.0
>
>
> I am try to run Spark SQL using bin/spark-sql with Spark 2.0 master
> code(commit ba181c0c7a32b0e81bbcdbe5eed94fc97b58c83e) but ran across an issue
> that it always connect local derby database and can't connect my existing
> hive metastore database. Could you help me to check what's the root cause ?
> What's specific configuration for integration with hive metastore in Spark
> 2.0 ? BTW, this case is OK in Spark 1.6. Thanks in advance !
> Build package command:
> ./dev/make-distribution.sh --tgz -Pyarn -Phadoop-2.6
> -Dhadoop.version=2.6.0-cdh5.5.1 -Phive -Phive-thriftserver -DskipTests
> Key configurations in spark-defaults.conf:
> {code}
> spark.sql.hive.metastore.version=1.1.0
> spark.sql.hive.metastore.jars=/usr/lib/hive/lib/*:/usr/lib/hadoop/client/*
> spark.executor.extraClassPath=/etc/hive/conf
> spark.driver.extraClassPath=/etc/hive/conf
> spark.yarn.jars=local:/usr/lib/spark/jars/*
> {code}
> There is existing hive metastore database named by "test_sparksql". I always
> got error "metastore.ObjectStore: Failed to get database test_sparksql,
> returning NoSuchObjectException" after issuing 'use test_sparksql'. Please
> see below steps for details.
>
> $ /usr/lib/spark/bin/spark-sql --master yarn --deploy-mode client
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/lib/spark/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/lib/avro/avro-tools-1.7.6-cdh5.5.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 16/05/12 22:23:28 WARN conf.HiveConf: HiveConf of name
> hive.enable.spark.execution.engine does not exist
> 16/05/12 22:23:30 INFO metastore.HiveMetaStore: 0: Opening raw store with
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 16/05/12 22:23:30 INFO metastore.ObjectStore: ObjectStore, initialize called
> 16/05/12 22:23:30 WARN DataNucleus.General: Plugin (Bundle)
> "org.datanucleus.store.rdbms" is already registered. Ensure you dont have
> multiple JAR versions of the same plugin in the classpath. The URL
> "file:/usr/lib/hive/lib/datanucleus-rdbms-3.2.9.jar" is already registered,
> and you are trying to register an identical plugin located at URL
> "file:/usr/lib/spark/jars/datanucleus-rdbms-3.2.9.jar."
> 16/05/12 22:23:30 WARN DataNucleus.General: Plugin (Bundle) "org.datanucleus"
> is already registered. Ensure you dont have multiple JAR versions of the same
> plugin in the classpath. The URL
> "file:/usr/lib/hive/lib/datanucleus-core-3.2.10.jar" is already registered,
> and you are trying to register an identical plugin located at URL
> "file:/usr/lib/spark/jars/datanucleus-core-3.2.10.jar."
> 16/05/12 22:23:30 WARN DataNucleus.General: Plugin (Bundle)
> "org.datanucleus.api.jdo" is already registered. Ensure you dont have
> multiple JAR versions of the same plugin in the classpath. The URL
> "file:/usr/lib/spark/jars/datanucleus-api-jdo-3.2.6.jar" is already
> registered, and you are trying to register an identical plugin located at URL
> "file:/usr/lib/hive/lib/datanucleus-api-jdo-3.2.6.jar."
> 16/05/12 22:23:30 INFO DataNucleus.Persistence: Property
> datanucleus.cache.level2 unknown - will be ignored
> 16/05/12 22:23:30 INFO DataNucleus.Persistence: Property
> hive.metastore.integral.jdo.pushdown unknown - will be ignored
> 16/05/12 22:23:31 WARN conf.HiveConf: HiveConf of name
> hive.enable.spark.execution.engine does not exist
> 16/05/12 22:23:31 INFO metastore.ObjectStore: Setting MetaStore object pin
> classes with
> hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
> 16/05/12 22:23:32 INFO DataNucleus.Datastore: The class
> "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
> "embedded-only" so does not have its own datastore table.
> 16/05/12 22:23:32 INFO DataNucleus.Datastore: The class
> "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only"
> so does not have its own datastore table.
> 16/05/12 22:23:33 INFO DataNucleus.Datastore: The class
> "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
> "embedded-only" so does not have its own datastore table.
> 16/05/12 22:23:33 INFO DataNucleus.Datastore: The class
> "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only"
> so does not have its own datastore table.
> 16/05/12 22:23:33 INFO metastore.MetaStoreDirectSql: Using direct SQL,
> underlying DB is DERBY
> 16/05/12 22:23:33 INFO metastore.ObjectStore: Initialized ObjectStore
> 16/05/12 22:23:33 WARN metastore.ObjectStore: Version information not found
> in metastore. hive.metastore.schema.verification is not enabled so recording
> the schema version 1.2.0
> 16/05/12 22:23:33 WARN metastore.ObjectStore: Failed to get database default,
> returning NoSuchObjectException
> 16/05/12 22:23:34 INFO metastore.HiveMetaStore: Added admin role in metastore
> 16/05/12 22:23:34 INFO metastore.HiveMetaStore: Added public role in metastore
> 16/05/12 22:23:34 INFO metastore.HiveMetaStore: No user is added in admin
> role, since config is empty
> 16/05/12 22:23:34 INFO metastore.HiveMetaStore: 0: get_all_databases
> 16/05/12 22:23:34 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr
> cmd=get_all_databases
> 16/05/12 22:23:34 INFO metastore.HiveMetaStore: 0: get_functions: db=default
> pat=*
> 16/05/12 22:23:34 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr
> cmd=get_functions: db=default pat=*
> 16/05/12 22:23:34 INFO DataNucleus.Datastore: The class
> "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as
> "embedded-only" so does not have its own datastore table.
> 16/05/12 22:23:34 INFO session.SessionState: Created local directory:
> /tmp/4e7ccc40-e10b-455c-b51d-ed225be85ffe_resources
> 16/05/12 22:23:34 INFO session.SessionState: Created HDFS directory:
> /tmp/hive/root/4e7ccc40-e10b-455c-b51d-ed225be85ffe
> 16/05/12 22:23:34 INFO session.SessionState: Created local directory:
> /tmp/root/4e7ccc40-e10b-455c-b51d-ed225be85ffe
> 16/05/12 22:23:34 INFO session.SessionState: Created HDFS directory:
> /tmp/hive/root/4e7ccc40-e10b-455c-b51d-ed225be85ffe/_tmp_space.db
> 16/05/12 22:23:34 INFO spark.SparkContext: Running Spark version
> 2.0.0-SNAPSHOT
> 16/05/12 22:23:34 INFO spark.SecurityManager: Changing view acls to: root
> 16/05/12 22:23:34 INFO spark.SecurityManager: Changing modify acls to: root
> 16/05/12 22:23:34 INFO spark.SecurityManager: Changing view acls groups to:
> 16/05/12 22:23:34 INFO spark.SecurityManager: Changing modify acls groups to:
> 16/05/12 22:23:34 INFO spark.SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(root); groups
> with view permissions: Set(); users with modify permissions: Set(root);
> groups with modify permissions: Set()
> 16/05/12 22:23:35 INFO util.Utils: Successfully started service 'sparkDriver'
> on port 37223.
> 16/05/12 22:23:35 INFO spark.SparkEnv: Registering MapOutputTracker
> 16/05/12 22:23:35 INFO spark.SparkEnv: Registering BlockManagerMaster
> 16/05/12 22:23:35 INFO storage.DiskBlockManager: Created local directory at
> /tmp/blockmgr-5a30adbe-4f9a-4b34-b52f-b61671f8b06d
> 16/05/12 22:23:35 INFO memory.MemoryStore: MemoryStore started with capacity
> 511.1 MB
> 16/05/12 22:23:35 INFO spark.SparkEnv: Registering OutputCommitCoordinator
> 16/05/12 22:23:35 INFO server.Server: jetty-8.y.z-SNAPSHOT
> 16/05/12 22:23:35 INFO server.AbstractConnector: Started
> SelectChannelConnector@0.0.0.0:4040
> 16/05/12 22:23:35 INFO util.Utils: Successfully started service 'SparkUI' on
> port 4040.
> 16/05/12 22:23:35 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at
> http://192.168.3.11:4040
> 16/05/12 22:23:35 INFO client.RMProxy: Connecting to ResourceManager at
> hw-node2/192.168.3.12:8032
> 16/05/12 22:23:35 INFO yarn.Client: Requesting a new application from cluster
> with 4 NodeManagers
> 16/05/12 22:23:35 INFO yarn.Client: Verifying our application has not
> requested more than the maximum memory capability of the cluster (196608 MB
> per container)
> 16/05/12 22:23:35 INFO yarn.Client: Will allocate AM container, with 896 MB
> memory including 384 MB overhead
> 16/05/12 22:23:35 INFO yarn.Client: Setting up container launch context for
> our AM
> 16/05/12 22:23:35 INFO yarn.Client: Setting up the launch environment for our
> AM container
> 16/05/12 22:23:35 INFO yarn.Client: Preparing resources for our AM container
> 16/05/12 22:23:35 INFO yarn.Client: Uploading resource
> file:/tmp/spark-a712ffb6-a0d0-48db-99b4-ee6a41b3f132/__spark_conf__7597761027449817951.zip
> ->
> hdfs://hw-node2:8020/user/root/.sparkStaging/application_1463053929123_0006/__spark_conf__.zip
> 16/05/12 22:23:36 INFO yarn.Client: Uploading resource
> file:/tmp/spark-a712ffb6-a0d0-48db-99b4-ee6a41b3f132/__spark_conf__9093112552235548615.zip
> ->
> hdfs://hw-node2:8020/user/root/.sparkStaging/application_1463053929123_0006/__spark_conf__9093112552235548615.zip
> 16/05/12 22:23:36 INFO spark.SecurityManager: Changing view acls to: root
> 16/05/12 22:23:36 INFO spark.SecurityManager: Changing modify acls to: root
> 16/05/12 22:23:36 INFO spark.SecurityManager: Changing view acls groups to:
> 16/05/12 22:23:36 INFO spark.SecurityManager: Changing modify acls groups to:
> 16/05/12 22:23:36 INFO spark.SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(root); groups
> with view permissions: Set(); users with modify permissions: Set(root);
> groups with modify permissions: Set()
> 16/05/12 22:23:36 INFO yarn.Client: Submitting application
> application_1463053929123_0006 to ResourceManager
> 16/05/12 22:23:36 INFO impl.YarnClientImpl: Submitted application
> application_1463053929123_0006
> 16/05/12 22:23:36 INFO cluster.SchedulerExtensionServices: Starting Yarn
> extension services with app application_1463053929123_0006 and attemptId None
> 16/05/12 22:23:37 INFO yarn.Client: Application report for
> application_1463053929123_0006 (state: ACCEPTED)
> 16/05/12 22:23:37 INFO yarn.Client:
> client token: N/A
> diagnostics: N/A
> ApplicationMaster host: N/A
> ApplicationMaster RPC port: -1
> queue: root.root
> start time: 1463063016173
> final status: UNDEFINED
> tracking URL:
> http://hw-node2:8088/proxy/application_1463053929123_0006/
> user: root
> 16/05/12 22:23:38 INFO yarn.Client: Application report for
> application_1463053929123_0006 (state: ACCEPTED)
> 16/05/12 22:23:38 INFO cluster.YarnSchedulerBackend$YarnSchedulerEndpoint:
> ApplicationMaster registered as NettyRpcEndpointRef(null)
> 16/05/12 22:23:38 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter.
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS
> -> hw-node2, PROXY_URI_BASES ->
> http://hw-node2:8088/proxy/application_1463053929123_0006),
> /proxy/application_1463053929123_0006
> 16/05/12 22:23:38 INFO ui.JettyUtils: Adding filter:
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
> 16/05/12 22:23:39 INFO yarn.Client: Application report for
> application_1463053929123_0006 (state: RUNNING)
> 16/05/12 22:23:39 INFO yarn.Client:
> client token: N/A
> diagnostics: N/A
> ApplicationMaster host: 192.168.3.16
> ApplicationMaster RPC port: 0
> queue: root.root
> start time: 1463063016173
> final status: UNDEFINED
> tracking URL:
> http://hw-node2:8088/proxy/application_1463053929123_0006/
> user: root
> 16/05/12 22:23:39 INFO cluster.YarnClientSchedulerBackend: Application
> application_1463053929123_0006 has started running.
> 16/05/12 22:23:39 INFO util.Utils: Successfully started service
> 'org.apache.spark.network.netty.NettyBlockTransferService' on port 45022.
> 16/05/12 22:23:39 INFO netty.NettyBlockTransferService: Server created on
> 192.168.3.11:45022
> 16/05/12 22:23:39 INFO storage.BlockManager: external shuffle service port =
> 7337
> 16/05/12 22:23:39 INFO storage.BlockManagerMaster: Trying to register
> BlockManager
> 16/05/12 22:23:39 INFO storage.BlockManagerMasterEndpoint: Registering block
> manager 192.168.3.11:45022 with 511.1 MB RAM, BlockManagerId(driver,
> 192.168.3.11, 45022)
> 16/05/12 22:23:39 INFO storage.BlockManagerMaster: Registered BlockManager
> 16/05/12 22:23:39 INFO scheduler.EventLoggingListener: Logging events to
> hdfs://hw-node2:8020/user/spark/applicationHistory/application_1463053929123_0006
> 16/05/12 22:23:39 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend
> is ready for scheduling beginning after reached minRegisteredResourcesRatio:
> 0.8
> 16/05/12 22:23:39 INFO hive.HiveSharedState: Setting Hive metastore warehouse
> path to '/root/spark-warehouse'
> 16/05/12 22:23:39 INFO hive.HiveUtils: Initializing HiveMetastoreConnection
> version 1.1.0 using
> file:/usr/lib/hive/lib/httpcore-4.2.5.jar:file:/usr/lib/hive/lib/hive-contrib.jar:file:/usr/lib/hive/lib/oro-2.0.8.jar:file:/usr/lib/hive/lib/accumulo-start-1.6.0.jar:file:/usr/lib/hive/lib/groovy-all-2.4.4.jar:file:/usr/lib/hive/lib/hive-metastore.jar:file:/usr/lib/hive/lib/hive-beeline.jar:file:/usr/lib/hive/lib/datanucleus-core-3.2.10.jar:file:/usr/lib/hive/lib/jackson-core-2.2.2.jar:file:/usr/lib/hive/lib/velocity-1.5.jar:file:/usr/lib/hive/lib/hive-serde.jar:file:/usr/lib/hive/lib/hive-metastore-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/commons-beanutils-core-1.8.0.jar:file:/usr/lib/hive/lib/hamcrest-core-1.1.jar:file:/usr/lib/hive/lib/jta-1.1.jar:file:/usr/lib/hive/lib/hive-shims-0.23-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/antlr-2.7.7.jar:file:/usr/lib/hive/lib/hive-exec-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/geronimo-jta_1.1_spec-1.1.1.jar:file:/usr/lib/hive/lib/accumulo-fate-1.6.0.jar:file:/usr/lib/hive/lib/hive-accumulo-handler.jar:file:/usr/lib/hive/lib/snappy-java-1.0.4.1.jar:file:/usr/lib/hive/lib/tempus-fugit-1.1.jar:file:/usr/lib/hive/lib/maven-scm-provider-svn-commons-1.4.jar:file:/usr/lib/hive/lib/libfb303-0.9.2.jar:file:/usr/lib/hive/lib/datanucleus-rdbms-3.2.9.jar:file:/usr/lib/hive/lib/xz-1.0.jar:file:/usr/lib/hive/lib/hbase-common.jar:file:/usr/lib/hive/lib/activation-1.1.jar:file:/usr/lib/hive/lib/hive-ant-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/accumulo-trace-1.6.0.jar:file:/usr/lib/hive/lib/hive-serde-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/commons-compress-1.4.1.jar:file:/usr/lib/hive/lib/hbase-hadoop2-compat.jar:file:/usr/lib/hive/lib/commons-configuration-1.6.jar:file:/usr/lib/hive/lib/servlet-api-2.5.jar:file:/usr/lib/hive/lib/libthrift-0.9.2.jar:file:/usr/lib/hive/lib/stax-api-1.0.1.jar:file:/usr/lib/hive/lib/hive-testutils.jar:file:/usr/lib/hive/lib/hive-shims-scheduler-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/hive-testutils-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/junit-4.11.jar:file:/usr/lib/hive/lib/jackson-annotations-2.2.2.jar:file:/usr/lib/hive/lib/stringtemplate-3.2.1.jar:file:/usr/lib/hive/lib/super-csv-2.2.0.jar:file:/usr/lib/hive/lib/hive-hwi-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/log4j-1.2.16.jar:file:/usr/lib/hive/lib/geronimo-jaspic_1.0_spec-1.0.jar:file:/usr/lib/hive/lib/accumulo-core-1.6.0.jar:file:/usr/lib/hive/lib/hive-hbase-handler.jar:file:/usr/lib/hive/lib/high-scale-lib-1.1.1.jar:file:/usr/lib/hive/lib/hbase-protocol.jar:file:/usr/lib/hive/lib/hive-common-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/hive-jdbc.jar:file:/usr/lib/hive/lib/commons-logging-1.1.3.jar:file:/usr/lib/hive/lib/derby-10.11.1.1.jar:file:/usr/lib/hive/lib/hive-jdbc-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/hive-shims-scheduler.jar:file:/usr/lib/hive/lib/asm-commons-3.1.jar:file:/usr/lib/hive/lib/hive-jdbc-standalone.jar:file:/usr/lib/hive/lib/maven-scm-api-1.4.jar:file:/usr/lib/hive/lib/janino-2.7.6.jar:file:/usr/lib/hive/lib/hive-cli.jar:file:/usr/lib/hive/lib/maven-scm-provider-svnexe-1.4.jar:file:/usr/lib/hive/lib/bonecp-0.8.0.RELEASE.jar:file:/usr/lib/hive/lib/zookeeper.jar:file:/usr/lib/hive/lib/jline-2.12.jar:file:/usr/lib/hive/lib/asm-3.2.jar:file:/usr/lib/hive/lib/logredactor-1.0.3.jar:file:/usr/lib/hive/lib/hive-ant.jar:file:/usr/lib/hive/lib/hive-shims-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/ant-launcher-1.9.1.jar:file:/usr/lib/hive/lib/hive-cli-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/gson-2.2.4.jar:file:/usr/lib/hive/lib/avro.jar:file:/usr/lib/hive/lib/parquet-hadoop-bundle.jar:file:/usr/lib/hive/lib/commons-beanutils-1.7.0.jar:file:/usr/lib/hive/lib/commons-digester-1.8.jar:file:/usr/lib/hive/lib/apache-log4j-extras-1.2.17.jar:file:/usr/lib/hive/lib/calcite-core-1.0.0-incubating.jar:file:/usr/lib/hive/lib/metrics-json-3.0.2.jar:file:/usr/lib/hive/lib/hive-jdbc-1.1.0-cdh5.5.1-standalone.jar:file:/usr/lib/hive/lib/jackson-databind-2.2.2.jar:file:/usr/lib/hive/lib/hive-exec.jar:file:/usr/lib/hive/lib/jersey-server-1.14.jar:file:/usr/lib/hive/lib/asm-tree-3.1.jar:file:/usr/lib/hive/lib/jdo-api-3.0.1.jar:file:/usr/lib/hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar:file:/usr/lib/hive/lib/metrics-core-3.0.2.jar:file:/usr/lib/hive/lib/commons-dbcp-1.4.jar:file:/usr/lib/hive/lib/mail-1.4.1.jar:file:/usr/lib/hive/lib/metrics-jvm-3.0.2.jar:file:/usr/lib/hive/lib/paranamer-2.3.jar:file:/usr/lib/hive/lib/commons-lang-2.6.jar:file:/usr/lib/hive/lib/commons-compiler-2.7.6.jar:file:/usr/lib/hive/lib/commons-codec-1.4.jar:file:/usr/lib/hive/lib/guava-14.0.1.jar:file:/usr/lib/hive/lib/hive-service-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/jersey-servlet-1.14.jar:file:/usr/lib/hive/lib/regexp-1.3.jar:file:/usr/lib/hive/lib/jpam-1.1.jar:file:/usr/lib/hive/lib/calcite-linq4j-1.0.0-incubating.jar:file:/usr/lib/hive/lib/hive-accumulo-handler-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/hbase-server.jar:file:/usr/lib/hive/lib/eigenbase-properties-1.1.4.jar:file:/usr/lib/hive/lib/commons-pool-1.5.4.jar:file:/usr/lib/hive/lib/commons-vfs2-2.0.jar:file:/usr/lib/hive/lib/jackson-jaxrs-1.9.2.jar:file:/usr/lib/hive/lib/hive-hbase-handler-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/commons-math-2.1.jar:file:/usr/lib/hive/lib/commons-cli-1.2.jar:file:/usr/lib/hive/lib/commons-io-2.4.jar:file:/usr/lib/hive/lib/ant-1.9.1.jar:file:/usr/lib/hive/lib/ST4-4.0.4.jar:file:/usr/lib/hive/lib/hive-shims-common-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/hive-common.jar:file:/usr/lib/hive/lib/jetty-all-server-7.6.0.v20120127.jar:file:/usr/lib/hive/lib/hive-service.jar:file:/usr/lib/hive/lib/hbase-hadoop-compat.jar:file:/usr/lib/hive/lib/hive-shims-0.23.jar:file:/usr/lib/hive/lib/hive-contrib-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/curator-client-2.6.0.jar:file:/usr/lib/hive/lib/commons-httpclient-3.0.1.jar:file:/usr/lib/hive/lib/plexus-utils-1.5.6.jar:file:/usr/lib/hive/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar:file:/usr/lib/hive/lib/jetty-all-7.6.0.v20120127.jar:file:/usr/lib/hive/lib/hive-shims.jar:file:/usr/lib/hive/lib/datanucleus-api-jdo-3.2.6.jar:file:/usr/lib/hive/lib/htrace-core.jar:file:/usr/lib/hive/lib/httpclient-4.2.5.jar:file:/usr/lib/hive/lib/jcommander-1.32.jar:file:/usr/lib/hive/lib/antlr-runtime-3.4.jar:file:/usr/lib/hive/lib/opencsv-2.3.jar:file:/usr/lib/hive/lib/jsr305-3.0.0.jar:file:/usr/lib/hive/lib/jackson-xc-1.9.2.jar:file:/usr/lib/hive/lib/hive-shims-common.jar:file:/usr/lib/hive/lib/curator-framework-2.6.0.jar:file:/usr/lib/hive/lib/calcite-avatica-1.0.0-incubating.jar:file:/usr/lib/hive/lib/hive-beeline-1.1.0-cdh5.5.1.jar:file:/usr/lib/hive/lib/hive-hwi.jar:file:/usr/lib/hive/lib/hbase-client.jar:file:/usr/lib/hadoop/client/httpcore-4.2.5.jar:file:/usr/lib/hadoop/client/hadoop-hdfs.jar:file:/usr/lib/hadoop/client/apacheds-i18n-2.0.0-M15.jar:file:/usr/lib/hadoop/client/apacheds-kerberos-codec.jar:file:/usr/lib/hadoop/client/slf4j-api-1.7.5.jar:file:/usr/lib/hadoop/client/commons-net.jar:file:/usr/lib/hadoop/client/commons-beanutils-core-1.8.0.jar:file:/usr/lib/hadoop/client/jackson-annotations-2.2.3.jar:file:/usr/lib/hadoop/client/commons-logging.jar:file:/usr/lib/hadoop/client/curator-recipes-2.7.1.jar:file:/usr/lib/hadoop/client/hadoop-aws-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/snappy-java.jar:file:/usr/lib/hadoop/client/leveldbjni-all.jar:file:/usr/lib/hadoop/client/jackson-databind.jar:file:/usr/lib/hadoop/client/commons-lang.jar:file:/usr/lib/hadoop/client/xmlenc-0.52.jar:file:/usr/lib/hadoop/client/snappy-java-1.0.4.1.jar:file:/usr/lib/hadoop/client/commons-httpclient.jar:file:/usr/lib/hadoop/client/hadoop-yarn-server-common.jar:file:/usr/lib/hadoop/client/jackson-databind-2.2.3.jar:file:/usr/lib/hadoop/client/guava-11.0.2.jar:file:/usr/lib/hadoop/client/xz-1.0.jar:file:/usr/lib/hadoop/client/hadoop-yarn-api-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/hadoop-mapreduce-client-jobclient.jar:file:/usr/lib/hadoop/client/hadoop-mapreduce-client-app.jar:file:/usr/lib/hadoop/client/activation-1.1.jar:file:/usr/lib/hadoop/client/jaxb-api.jar:file:/usr/lib/hadoop/client/commons-compress-1.4.1.jar:file:/usr/lib/hadoop/client/commons-configuration-1.6.jar:file:/usr/lib/hadoop/client/jackson-xc.jar:file:/usr/lib/hadoop/client/servlet-api-2.5.jar:file:/usr/lib/hadoop/client/xmlenc.jar:file:/usr/lib/hadoop/client/jackson-jaxrs.jar:file:/usr/lib/hadoop/client/jackson-xc-1.8.8.jar:file:/usr/lib/hadoop/client/hadoop-common-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/apacheds-kerberos-codec-2.0.0-M15.jar:file:/usr/lib/hadoop/client/commons-cli.jar:file:/usr/lib/hadoop/client/hadoop-mapreduce-client-app-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/aws-java-sdk-1.7.4.jar:file:/usr/lib/hadoop/client/netty.jar:file:/usr/lib/hadoop/client/protobuf-java.jar:file:/usr/lib/hadoop/client/jaxb-api-2.2.2.jar:file:/usr/lib/hadoop/client/commons-logging-1.1.3.jar:file:/usr/lib/hadoop/client/commons-net-3.1.jar:file:/usr/lib/hadoop/client/hadoop-annotations.jar:file:/usr/lib/hadoop/client/hadoop-hdfs-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/jersey-core.jar:file:/usr/lib/hadoop/client/hadoop-yarn-client-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/hadoop-auth-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/zookeeper.jar:file:/usr/lib/hadoop/client/commons-collections-3.2.2.jar:file:/usr/lib/hadoop/client/servlet-api.jar:file:/usr/lib/hadoop/client/guava.jar:file:/usr/lib/hadoop/client/hadoop-yarn-api.jar:file:/usr/lib/hadoop/client/commons-math3.jar:file:/usr/lib/hadoop/client/slf4j-api.jar:file:/usr/lib/hadoop/client/stax-api.jar:file:/usr/lib/hadoop/client/hadoop-auth.jar:file:/usr/lib/hadoop/client/commons-io.jar:file:/usr/lib/hadoop/client/commons-digester.jar:file:/usr/lib/hadoop/client/hadoop-annotations-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/gson-2.2.4.jar:file:/usr/lib/hadoop/client/hadoop-mapreduce-client-core-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/avro.jar:file:/usr/lib/hadoop/client/activation.jar:file:/usr/lib/hadoop/client/apacheds-i18n.jar:file:/usr/lib/hadoop/client/hadoop-yarn-common.jar:file:/usr/lib/hadoop/client/commons-beanutils-1.7.0.jar:file:/usr/lib/hadoop/client/hadoop-mapreduce-client-common.jar:file:/usr/lib/hadoop/client/commons-digester-1.8.jar:file:/usr/lib/hadoop/client/jetty-util.jar:file:/usr/lib/hadoop/client/jackson-core-asl-1.8.8.jar:file:/usr/lib/hadoop/client/jetty-util-6.1.26.cloudera.4.jar:file:/usr/lib/hadoop/client/httpcore.jar:file:/usr/lib/hadoop/client/curator-client.jar:file:/usr/lib/hadoop/client/netty-3.6.2.Final.jar:file:/usr/lib/hadoop/client/jackson-mapper-asl.jar:file:/usr/lib/hadoop/client/commons-beanutils-core.jar:file:/usr/lib/hadoop/client/jackson-jaxrs-1.8.8.jar:file:/usr/lib/hadoop/client/xz.jar:file:/usr/lib/hadoop/client/paranamer-2.3.jar:file:/usr/lib/hadoop/client/commons-lang-2.6.jar:file:/usr/lib/hadoop/client/jackson-annotations.jar:file:/usr/lib/hadoop/client/commons-codec-1.4.jar:file:/usr/lib/hadoop/client/jersey-core-1.9.jar:file:/usr/lib/hadoop/client/api-asn1-api-1.0.0-M20.jar:file:/usr/lib/hadoop/client/commons-collections.jar:file:/usr/lib/hadoop/client/hadoop-mapreduce-client-jobclient-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/api-util.jar:file:/usr/lib/hadoop/client/jsr305.jar:file:/usr/lib/hadoop/client/httpclient.jar:file:/usr/lib/hadoop/client/xml-apis-1.3.04.jar:file:/usr/lib/hadoop/client/hadoop-mapreduce-client-shuffle.jar:file:/usr/lib/hadoop/client/hadoop-mapreduce-client-core.jar:file:/usr/lib/hadoop/client/curator-client-2.7.1.jar:file:/usr/lib/hadoop/client/commons-httpclient-3.1.jar:file:/usr/lib/hadoop/client/hadoop-yarn-common-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/commons-cli-1.2.jar:file:/usr/lib/hadoop/client/commons-io-2.4.jar:file:/usr/lib/hadoop/client/curator-framework.jar:file:/usr/lib/hadoop/client/stax-api-1.0-2.jar:file:/usr/lib/hadoop/client/htrace-core4.jar:file:/usr/lib/hadoop/client/jackson-core-2.2.3.jar:file:/usr/lib/hadoop/client/jackson-core-asl.jar:file:/usr/lib/hadoop/client/commons-configuration.jar:file:/usr/lib/hadoop/client/commons-compress.jar:file:/usr/lib/hadoop/client/hadoop-mapreduce-client-common-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/xercesImpl-2.9.1.jar:file:/usr/lib/hadoop/client/jersey-client-1.9.jar:file:/usr/lib/hadoop/client/log4j.jar:file:/usr/lib/hadoop/client/jackson-mapper-asl-1.8.8.jar:file:/usr/lib/hadoop/client/leveldbjni-all-1.8.jar:file:/usr/lib/hadoop/client/api-util-1.0.0-M20.jar:file:/usr/lib/hadoop/client/curator-framework-2.7.1.jar:file:/usr/lib/hadoop/client/commons-codec.jar:file:/usr/lib/hadoop/client/xml-apis.jar:file:/usr/lib/hadoop/client/jersey-client.jar:file:/usr/lib/hadoop/client/hadoop-yarn-client.jar:file:/usr/lib/hadoop/client/aws-java-sdk.jar:file:/usr/lib/hadoop/client/paranamer.jar:file:/usr/lib/hadoop/client/hadoop-aws.jar:file:/usr/lib/hadoop/client/hadoop-yarn-server-common-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/commons-math3-3.1.1.jar:file:/usr/lib/hadoop/client/hadoop-mapreduce-client-shuffle-2.6.0-cdh5.5.1.jar:file:/usr/lib/hadoop/client/httpclient-4.2.5.jar:file:/usr/lib/hadoop/client/commons-beanutils.jar:file:/usr/lib/hadoop/client/hadoop-common.jar:file:/usr/lib/hadoop/client/api-asn1-api.jar:file:/usr/lib/hadoop/client/htrace-core4-4.0.1-incubating.jar:file:/usr/lib/hadoop/client/log4j-1.2.17.jar:file:/usr/lib/hadoop/client/jsr305-3.0.0.jar:file:/usr/lib/hadoop/client/curator-recipes.jar:file:/usr/lib/hadoop/client/slf4j-log4j12.jar:file:/usr/lib/hadoop/client/jackson-core.jar:file:/usr/lib/hadoop/client/protobuf-java-2.5.0.jar:file:/usr/lib/hadoop/client/xercesImpl.jar:file:/usr/lib/hadoop/client/gson.jar
> 16/05/12 22:23:40 INFO metastore.HiveMetaStore: 0: Opening raw store with
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 16/05/12 22:23:40 INFO metastore.ObjectStore: ObjectStore, initialize called
> 16/05/12 22:23:40 INFO DataNucleus.Persistence: Property
> datanucleus.cache.level2 unknown - will be ignored
> 16/05/12 22:23:40 INFO DataNucleus.Persistence: Property
> hive.metastore.integral.jdo.pushdown unknown - will be ignored
> 16/05/12 22:23:41 INFO metastore.ObjectStore: Setting MetaStore object pin
> classes with
> hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
> 16/05/12 22:23:42 INFO DataNucleus.Datastore: The class
> "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
> "embedded-only" so does not have its own datastore table.
> 16/05/12 22:23:42 INFO DataNucleus.Datastore: The class
> "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only"
> so does not have its own datastore table.
> 16/05/12 22:23:42 INFO DataNucleus.Datastore: The class
> "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
> "embedded-only" so does not have its own datastore table.
> 16/05/12 22:23:42 INFO DataNucleus.Datastore: The class
> "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only"
> so does not have its own datastore table.
> 16/05/12 22:23:42 INFO DataNucleus.Query: Reading in results for query
> "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is
> closing
> 16/05/12 22:23:42 INFO metastore.MetaStoreDirectSql: Using direct SQL,
> underlying DB is DERBY
> 16/05/12 22:23:42 INFO metastore.ObjectStore: Initialized ObjectStore
> 16/05/12 22:23:42 INFO metastore.HiveMetaStore: Added admin role in metastore
> 16/05/12 22:23:42 INFO metastore.HiveMetaStore: Added public role in metastore
> 16/05/12 22:23:42 INFO metastore.HiveMetaStore: No user is added in admin
> role, since config is empty
> 16/05/12 22:23:42 INFO log.PerfLogger: <PERFLOG method=get_all_functions
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 16/05/12 22:23:42 INFO metastore.HiveMetaStore: 0: get_all_functions
> 16/05/12 22:23:42 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr
> cmd=get_all_functions
> 16/05/12 22:23:42 INFO DataNucleus.Datastore: The class
> "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as
> "embedded-only" so does not have its own datastore table.
> 16/05/12 22:23:42 INFO log.PerfLogger: </PERFLOG method=get_all_functions
> start=1463063022896 end=1463063022941 duration=45
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0
> retryCount=0 error=false>
> 16/05/12 22:23:42 INFO session.SessionState: Created local directory:
> /tmp/f1ff20d6-3eac-4df0-adbd-64f7e73f35e8_resources
> 16/05/12 22:23:42 INFO session.SessionState: Created HDFS directory:
> /tmp/hive/root/f1ff20d6-3eac-4df0-adbd-64f7e73f35e8
> 16/05/12 22:23:42 INFO session.SessionState: Created local directory:
> /tmp/root/f1ff20d6-3eac-4df0-adbd-64f7e73f35e8
> 16/05/12 22:23:42 INFO session.SessionState: Created HDFS directory:
> /tmp/hive/root/f1ff20d6-3eac-4df0-adbd-64f7e73f35e8/_tmp_space.db
> 16/05/12 22:23:42 INFO session.SessionState: No Tez session required at this
> point. hive.execution.engine=mr.
> 16/05/12 22:23:42 INFO client.HiveClientImpl: Warehouse location for Hive
> client (version 1.1.0) is /root/spark-warehouse
> 16/05/12 22:23:43 INFO session.SessionState: Created local directory:
> /tmp/4f466b18-e85b-4fa5-9b3a-2a1a67118851_resources
> 16/05/12 22:23:43 INFO session.SessionState: Created HDFS directory:
> /tmp/hive/root/4f466b18-e85b-4fa5-9b3a-2a1a67118851
> 16/05/12 22:23:43 INFO session.SessionState: Created local directory:
> /tmp/root/4f466b18-e85b-4fa5-9b3a-2a1a67118851
> 16/05/12 22:23:43 INFO session.SessionState: Created HDFS directory:
> /tmp/hive/root/4f466b18-e85b-4fa5-9b3a-2a1a67118851/_tmp_space.db
> 16/05/12 22:23:43 INFO session.SessionState: No Tez session required at this
> point. hive.execution.engine=mr.
> 16/05/12 22:23:43 INFO client.HiveClientImpl: Warehouse location for Hive
> client (version 1.1.0) is /root/spark-warehouse
> spark-sql> use test_sparksql;
> 16/05/12 22:25:06 INFO execution.SparkSqlParser: Parsing command: use
> test_sparksql
> 16/05/12 22:25:06 INFO log.PerfLogger: <PERFLOG method=create_database
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 16/05/12 22:25:06 INFO metastore.HiveMetaStore: 0: create_database:
> Database(name:default, description:default database,
> locationUri:hdfs://hw-node2:8020/root/spark-warehouse, parameters:{})
> 16/05/12 22:25:06 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr
> cmd=create_database: Database(name:default, description:default database,
> locationUri:hdfs://hw-node2:8020/root/spark-warehouse, parameters:{})
> 16/05/12 22:25:06 ERROR metastore.RetryingHMSHandler:
> AlreadyExistsException(message:Database default already exists)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:898)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:133)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
> at com.sun.proxy.$Proxy34.create_database(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createDatabase(HiveMetaStoreClient.java:645)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:91)
> at com.sun.proxy.$Proxy35.createDatabase(Unknown Source)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createDatabase(Hive.java:341)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply$mcV$sp(HiveClientImpl.scala:292)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply(HiveClientImpl.scala:292)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply(HiveClientImpl.scala:292)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:263)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:210)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:209)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:252)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.createDatabase(HiveClientImpl.scala:291)
> at
> org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply$mcV$sp(HiveExternalCatalog.scala:94)
> at
> org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply(HiveExternalCatalog.scala:94)
> at
> org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply(HiveExternalCatalog.scala:94)
> at
> org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:68)
> at
> org.apache.spark.sql.hive.HiveExternalCatalog.createDatabase(HiveExternalCatalog.scala:93)
> at
> org.apache.spark.sql.catalyst.catalog.SessionCatalog.createDatabase(SessionCatalog.scala:142)
> at
> org.apache.spark.sql.catalyst.catalog.SessionCatalog.<init>(SessionCatalog.scala:84)
> at
> org.apache.spark.sql.hive.HiveSessionCatalog.<init>(HiveSessionCatalog.scala:50)
> at
> org.apache.spark.sql.hive.HiveSessionState.catalog$lzycompute(HiveSessionState.scala:49)
> at
> org.apache.spark.sql.hive.HiveSessionState.catalog(HiveSessionState.scala:48)
> at
> org.apache.spark.sql.hive.HiveSessionState$$anon$1.<init>(HiveSessionState.scala:63)
> at
> org.apache.spark.sql.hive.HiveSessionState.analyzer$lzycompute(HiveSessionState.scala:63)
> at
> org.apache.spark.sql.hive.HiveSessionState.analyzer(HiveSessionState.scala:62)
> at
> org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:48)
> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:61)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:541)
> at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:671)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run(SparkSQLDriver.scala:62)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:325)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:240)
> at
> org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:724)
> at
> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> 16/05/12 22:25:06 INFO log.PerfLogger: </PERFLOG method=create_database
> start=1463063106660 end=1463063106665 duration=5
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0
> retryCount=-1 error=true>
> 16/05/12 22:25:06 INFO log.PerfLogger: <PERFLOG method=get_database
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler>
> 16/05/12 22:25:06 INFO metastore.HiveMetaStore: 0: get_database: test_sparksql
> 16/05/12 22:25:06 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr
> cmd=get_database: test_sparksql
> 16/05/12 22:25:06 WARN metastore.ObjectStore: Failed to get database
> test_sparksql, returning NoSuchObjectException
> 16/05/12 22:25:06 INFO log.PerfLogger: </PERFLOG method=get_database
> start=1463063106947 end=1463063106950 duration=3
> from=org.apache.hadoop.hive.metastore.RetryingHMSHandler threadId=0
> retryCount=-1 error=true>
> Error in query: Database 'test_sparksql' not found;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
