Re: Hive query starts own session for LLAP
> Now we need an explanation of "map" -- can you supply it? The "map" mode runs all tasks with a TableScan operator inside LLAP instances and all other tasks in Tez YARN containers. This is the LLAP + Tez hybrid mode, which introduces some complexity in debugging a single query. The "only" mode is so far the best option since, the LlapDecider runs very late in the optimizer order the earlier optimizers need to hedge their bets on whether LLAP will finally be used for a vertex or not. The "only" mode sort of short-cuts that by assuring all optimizers that it is "LLAP or Bust!". Cheers, Gopal
Re: Hive query starts own session for LLAP
Thanks for the explanations of "all" and "only" Sergey. I've added them to the wiki, with minor edits: hive.llap.execution.mode <https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.llap.execution.mode> . Now we need an explanation of "map" -- can you supply it? -- Lefty On Mon, Sep 25, 2017 at 11:33 AM, Sergey Shelukhin <ser...@hortonworks.com> wrote: > Hello. > Hive would create a new Tez AM to coordinate the query (or use an existing > one if HS2 session pool is used). However, the YARN app for Tez should > only have a single container. Is this not the case? > If it’s running additional containers, what is hive.llap.execution.mode > set to? It should be set to all or only by default (“all” means run > everything in LLAP if at all possible; “only” is the same with fallback to > containers disabled - so the query would fail if it cannot run in LLAP). > > From: Rajesh Narayanan <rajesh.r.naraya...@oracle.com> on behalf of > Rajesh Narayanan <rajesh.r.naraya...@oracle.com> > Reply-To: "user@hive.apache.org" <user@hive.apache.org> > Date: Friday, September 22, 2017 at 11:59 > To: "user@hive.apache.org" <user@hive.apache.org> > Subject: Hive query starts own session for LLAP > > > HI All, > When I execute the hive query , that starts its own session and creates > new yarn jobs rather than using the llap enabled job > Can you please provide some suggestion? > > Thanks > Rajesh > >
Re: Hive query starts own session for LLAP
Hello. Hive would create a new Tez AM to coordinate the query (or use an existing one if HS2 session pool is used). However, the YARN app for Tez should only have a single container. Is this not the case? If it’s running additional containers, what is hive.llap.execution.mode set to? It should be set to all or only by default (“all” means run everything in LLAP if at all possible; “only” is the same with fallback to containers disabled - so the query would fail if it cannot run in LLAP). From: Rajesh Narayanan <rajesh.r.naraya...@oracle.com> on behalf of Rajesh Narayanan <rajesh.r.naraya...@oracle.com> Reply-To: "user@hive.apache.org" <user@hive.apache.org> Date: Friday, September 22, 2017 at 11:59 To: "user@hive.apache.org" <user@hive.apache.org> Subject: Hive query starts own session for LLAP HI All, When I execute the hive query , that starts its own session and creates new yarn jobs rather than using the llap enabled job Can you please provide some suggestion? Thanks Rajesh
Hive query starts own session for LLAP
HI All, When I execute the hive query , that starts its own session and creates new yarn jobs rather than using the llap enabled job Can you please provide some suggestion? Thanks Rajesh
