I was looking at : *hive**.metastore.max.server.threads *but reading more into it tells me its a config for the thrift server and not the metastore.
Most of our applications accessing the metastore are Spark Sql applications which do INSERT operations on multiple partitions on a hourly basis. This basically implies that most of these queries dont use the thrift server but directly connect to the metastore from the spark application. Can you give me more information on how we can have multiple metastores behind a load balancer? I read about providing multiple urls in " *hive.metastore.uris*" but saw that it would always pick the first URI and choose from the rest only in case of a failure. Thanks again for the replies. On Wed, Mar 30, 2016 at 2:30 PM, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Are you talking about increase in number of threads from Hive server2 > connection to your database (MySQL)? > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > > On 30 March 2016 at 22:28, Udit Mehta <ume...@groupon.com> wrote: > >> Hi all, >> >> We are currently running Hive in production and staging with the >> metastore connecting to a MySql database in the backend. The traffic in >> production accessing the metastore is more than staging which is expected. >> We have had a sudden increase in traffic which has led to the metastore >> operation taking a lot longer than before. The same query on staging takes >> a lot less due to the lesser traffic on the staging cluster. >> >> We tried increasing the heap space for the metastore process as well as >> bumped up the memory for the mysql database. Both these changes did not >> seem to help much and we still see delays. Is there any other config we can >> increase to counter this increased traffic? I am looking at config for max >> threads as well but im not sure if this is the right path ahead. >> >> Im wondering if the metastore is a bottleneck here or im missing >> something. >> >> Looking forward to your reply, >> Udit >> > >
