Re: Spark Thrift Server performance

2016-07-13 Thread Mich Talebzadeh
Thanks guys Any idea on this What is the limit on the number of users accessing the thrift server concurrently. Say using Yarn, will Yarn control apps accessing the thrift server or users each armed with beeline connect to thrift server. Say my STS has this conf below

Re: Spark Thrift Server performance

2016-07-13 Thread ayan guha
Not really, that is not the primary intention. Our main goal is poor man's high availability (as STS does not provide HA mechanism like HS2) :). Additionally, we have made STS part of Ambari AUTO_START group, so Ambari brings up STS if it goes down for some intermittent reason. On Thu, Jul 14,

Re: Spark Thrift Server performance

2016-07-13 Thread Michael Segel
Hey, silly question? If you’re running a load balancer, are you trying to reuse the RDDs between jobs? TIA -Mike > On Jul 13, 2016, at 9:08 AM, ayan guha > wrote: > > My 2 cents: > > Yes, we are running multiple STS (we are running on

Re: Spark Thrift Server performance

2016-07-13 Thread ayan guha
My 2 cents: Yes, we are running multiple STS (we are running on different nodes, but you can run on same node, different ports). Using Ambari, it is really convenient to manage. We have set up a nginx load balancer as well pointing to both services and all our external BI tools connect to the