[
https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101597#comment-16101597
]
Quincy HSIEH edited comment on SPARK-9776 at 7/26/17 12:08 PM:
---------------------------------------------------------------
Hi,
In case that someone who has the same issue and comes to this page : here is
the workaround I use to resolve the problem :
1. Configure Spark with Yarn client mode.
2. Use FairScheduler in Hadoop config yarn-site.xml to enable multi-queques so
that you can spark-submit job to different queue name with --queue
<QUEUE_NAME_YOU_PREFER>
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>
3. Switch derby to memory mode by modifying the connection string in Spark
config hive-site.xml
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:memory:<YOUR_STORAGE_DIR>/metastore_db;create=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
was (Author: quincy.tw):
Hi,
In case that someone who has the same issue and comes to this page : here is
the workaround I use to resolve the problem :
1. Configure Spark with Yarn client mode.
2. Use FairScheduler in Hadoop config yarn-site.xml to enable multi-queques so
that you can spark-submit job to different queue name with --queue
<QUEUE_NAME_YOU_PREFER>
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>
3. Switch derby to memory mode by modifying the connection string in Spark
config hive-site.xml
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:memory:<YOUR_STORAGE_DIR>/metastore_db;create=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
> Another instance of Derby may have already booted the database
> ---------------------------------------------------------------
>
> Key: SPARK-9776
> URL: https://issues.apache.org/jira/browse/SPARK-9776
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.5.0
> Environment: Mac Yosemite, spark-1.5.0
> Reporter: Sudhakar Thota
> Attachments: SPARK-9776-FL1.rtf
>
>
> val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc) results in
> error. Though the same works for spark-1.4.1.
> Caused by: ERROR XSDB6: Another instance of Derby may have already booted the
> database
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
