The back-end execution engine is Tez, and I use YARN for resource management.
I completely agree with your deduction that the impact on the run time will be dependent on the nature of the queries. I would like to conduct some experiments (for a given workload, cluster configuration) to quantify the impact. For this, I need to be able to run queries simultaneously and measure the running times. What I glean from other threads is that, it should be good enough to fire up 2 CLI's and issue the queries. Raajay On Tue, Aug 25, 2015 at 4:17 PM, Ryan Harris <[email protected]> wrote: > You need to be a bit more clear with your environment and objective > here.... > > What is your back-end execution engine? MapReduce, Spark, or Tez? > > What are you using for resource management? YARN or MapReduce? > > > > The running time of one query in the presence of other queries will > entirely depend on the cost/complexity of the queries. If each query is > able to fully utilize your allocated resources on the cluster then they > will be slower when run at the same time. However, many times in hive, > depending on the query and depending on the cluster resources, a single > hive query will only utilize a fraction of the cluster resources, in this > case multiple queries could be run at the same time with no detrimental > impact to performance as long as those queries aren't updating the same > hive table. > > > > *From:* Raajay [mailto:[email protected]] > *Sent:* Tuesday, August 25, 2015 7:21 AM > *To:* [email protected] > *Subject:* Run multiple queries simultaneously > > > > Hello, > > I want to compare the running time of an query when run alone against the > run time in presence of other queries. > > What is the ideal setup required to run this experiment ? Should I have > two Hive CLI's open and issue queries simultaneously ? How to script such > experiment in Hive ? > > Raajay > ------------------------------ > THIS ELECTRONIC MESSAGE, INCLUDING ANY ACCOMPANYING DOCUMENTS, IS > CONFIDENTIAL and may contain information that is privileged and exempt from > disclosure under applicable law. If you are neither the intended recipient > nor responsible for delivering the message to the intended recipient, > please note that any dissemination, distribution, copying or the taking of > any action in reliance upon the message is strictly prohibited. If you have > received this communication in error, please notify the sender immediately. > Thank you. >
