Hi Koji, This is exactly what I was looking for. Thank you so much for the pointer!
[image: --] Tariq, Mohammad [image: https://]about.me/mti <https://about.me/mti?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=chrome_ext> [image: http://] Tariq, Mohammad about.me/mti [image: http://] <http://about.me/mti> On Mon, Mar 20, 2017 at 8:34 PM, Koji Noguchi < knogu...@yahoo-inc.com.invalid> wrote: > > It's a valid ask and I'm afraid we currently don't have that feature (in > neither mapreduce nor tez).https://issues.apache.org/jira/browse/PIG-4424 > > This is blocked by https://issues.apache.org/jira/browse/PIG-2597 and > assigned to me. > > I need to find time to get to this. Sorry! > Koji > > > > On Sunday, March 19, 2017, 10:35:55 AM EDT, Mohammad Tariq < > donta...@gmail.com> wrote:Yeah, I'm aware of that. I think my question > wasn't clear. I intend to > assign resources to individual MR jobs which get created during the course > of execution of a pig script. Say, I have a script which performs a JOIN > followed by a GROUP BY. Now I want to provide resources to my script based > on the magnitude of computation required by these 2 separate MR jobs. For > example, separate values for memory for JOIN and GROUP BY. > > As per its default behaviour Pig will use same resource values for all the > MR jobs spawned from a single script. > > Hope I was able to clarify it a bit. > > > [image: --] > > Tariq, Mohammad > [image: https://]about.me/mti > <https://about.me/mti?promo=email_sig&utm_source=product&; > utm_medium=email_sig&utm_campaign=chrome_ext> > > > > > [image: http://] > > Tariq, Mohammad > about.me/mti > [image: http://] > <http://about.me/mti> > > > On Sun, Mar 19, 2017 at 11:38 AM, Jianfeng (Jeff) Zhang < > jzh...@hortonworks.com> wrote: > > > > > In that case, you can use keyword parallel to control the number of > > reducer task of each mr job. The number of mapper task is determined by > > the InputFormat. > > > > See. > > http://pig.apache.org/docs/r0.16.0/basic.html > > > > > > Best Regard, > > Jeff Zhang > > > > > > > > > > > > On 3/18/17, 6:03 PM, "Mohammad Tariq" <donta...@gmail.com> wrote: > > > > >Hi Jeff, > > > > > >Thank you for the prompt response. However, I can't use Tez because of > > >some > > >reasons. > > > > > > > > >[image: --] > > > > > >Tariq, Mohammad > > >[image: https://]about.me/mti > > ><https://about.me/mti?promo=email_sig&utm_source=product&; > > utm_medium=email_ > > >sig&utm_campaign=chrome_ext> > > > > > > > > > > > > > > >[image: http://] > > > > > >Tariq, Mohammad > > >about.me/mti > > >[image: http://] > > ><http://about.me/mti> > > > > > > > > >On Fri, Mar 17, 2017 at 2:24 PM, Jianfeng (Jeff) Zhang < > > >jzh...@hortonworks.com> wrote: > > > > > >> > > >> I would suggest you to use tez which just launch one yarn app for one > > >>pig > > >> script. > > >> > > >> http://pig.apache.org/docs/r0.16.0/perf.html#enable-tez > > >> > > >> > > >> > > >> > > >> Best Regard, > > >> Jeff Zhang > > >> > > >> > > >> > > >> > > >> > > >> On 3/17/17, 3:24 AM, "Mohammad Tariq" <donta...@gmail.com> wrote: > > >> > > >> >Hi group, > > >> > > > >> >In any real world pig script we end up with multiple MR jobs(well, > > >>most of > > >> >the times). I was wondering if it's possible to allocate resources to > > >> >individual MR jobs rather than assigning them at the script level > > >>itself. > > >> >Tried looking at multiple places. Would really appreciate some > pointers > > >> >regarding the same. > > >> > > > >> >Thank you so much for your valuable time! > > >> > > > >> > > > >> >[image: http://] > > >> > > > >> >Tariq, Mohammad > > >> >about.me/mti > > >> >[image: http://] > > >> ><http://about.me/mti> > > >> > > > >> > > > >> > > > >> >[image: --] > > >> > > > >> >Tariq, Mohammad > > >> >[image: https://]about.me/mti > > >> ><https://about.me/mti?promo=email_sig&utm_source=product&; > > >> utm_medium=email_ > > >> >sig&utm_campaign=chrome_ext> > > >> > > >> > > > > >
