hive.metastore.metadb.dir not working programmatically

Experts need your help, I'm using spark 1.4.1 and when set this hive.metastore.metadb.dir programmatically for a hivecontext i.e for local metastore i.e the default metastore_db for derby, the metastore_db is still getting creating in the same path as user.dir. Can you guys provide some insights

Re: How to keep RDDs in memory between two different batch jobs?

I was about say whatever the previous post said,so +1 to the previous post,from my understanding (gut feeling) of your requirement it very easy to do this with spark-job-server. -- View this message in context:

Re: SparkR sqlContext or sc not found in RStudio

I'm sorry, I have no idea why it is failing on your side.I have been using this for a while now and it works fine.All I can say is use version 1.4.0 but I don't think so it is going to make a big difference.This is the one which I use,a/b are my directories.

Re: SparkR sqlContext or sc not found in RStudio

Yep,I saw that in your previous post and I thought it was a typing mistake that you did while posting,I never imagined that it was done on R studio.Glad it worked. -- View this message in context:

Re: Share RDD from SparkR and another application

A small correction when I typed it is not RDDBackend it is RBackend,sorry. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Share-RDD-from-SparkR-and-another-application-tp23795p23828.html Sent from the Apache Spark User List mailing list archive at

Re: Share RDD from SparkR and another application

I appreciate your reply. Yes,you are right by putting in a parquet etc and reading from another app,I would rather use spark-jobserver or IBM kernel to achieve the same if it is not SparkR as it gives more flexibility/scalabilty. Anyway,I have found a way to run R for my poc from my existing app

Share RDD from SparkR and another application

Hello, I would like to share RDD between an application and sparkR. I understand we have job-server and IBM kernel for sharing the context for different applications but not sure how we can use it with sparkR as it is some sort of front end (R shell) with spark. Any insights appreciated. Hari

spark-sql with join terribly slow.

Guys, I'm trying to join 2-3 schemaRDD's for approx 30,000 rows and it is terribly slow.No doubt I get the results but it takes 8s to do the join and get the results. I'm running on a standalone spark in my m/c having 8 cores and 12gb RAM with 4 workers. Not sure why it is consuming time,any