There's an annoying small usability issue in HiveContext.
By default, it creates a local metastore which forbids other processes
using HiveContext to be launched from the same directory.
How can I make the metastore local to each HiveContext? Is there an
in-memory metastore configuration?
I have never tried this yet, but maybe you can use an in-memory Derby
database as metastore
https://db.apache.org/derby/docs/10.7/devguide/cdevdvlpinmemdb.html
I'll investigate this when free, guess we can use this for Spark SQL
Hive support testing.
On 10/27/14 4:38 PM, Jianshi Huang
Please see
https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin#AdminManualMetastoreAdmin-EmbeddedMetastore
Cheers
On Oct 27, 2014, at 6:20 AM, Cheng Lian lian.cs@gmail.com wrote:
I have never tried this yet, but maybe you can use an in-memory Derby
database as
Thanks Ted, this is exactly what Spark SQL LocalHiveContext does. To
make an embedded metastore local to a single HiveContext, we must
allocate different Derby database directories for each HiveContext, and
Jianshi is also trying to avoid that.
On 10/27/14 9:44 PM, Ted Yu wrote:
Please see
Thanks Ted and Cheng for the in memory derby solution. I'll check it out. :)
And to me, using in-mem by default makes sense, if user wants a shared
metastore, it needs to be specified. An 'embedded' local metastore in the
working directory barely has a use case.
Jianshi
On Mon, Oct 27, 2014
