It's a valid ask and I'm afraid we currently don't have that feature (in neither mapreduce nor tez).https://issues.apache.org/jira/browse/PIG-4424
This is blocked by https://issues.apache.org/jira/browse/PIG-2597 and assigned to me. I need to find time to get to this. Sorry! Koji On Sunday, March 19, 2017, 10:35:55 AM EDT, Mohammad Tariq <donta...@gmail.com> wrote:Yeah, I'm aware of that. I think my question wasn't clear. I intend to assign resources to individual MR jobs which get created during the course of execution of a pig script. Say, I have a script which performs a JOIN followed by a GROUP BY. Now I want to provide resources to my script based on the magnitude of computation required by these 2 separate MR jobs. For example, separate values for memory for JOIN and GROUP BY. As per its default behaviour Pig will use same resource values for all the MR jobs spawned from a single script. Hope I was able to clarify it a bit. [image: --] Tariq, Mohammad [image: https://]about.me/mti <https://about.me/mti?promo=email_sig&utm_source=product&utm_medium=email_sig&utm_campaign=chrome_ext> [image: http://] Tariq, Mohammad about.me/mti [image: http://] <http://about.me/mti> On Sun, Mar 19, 2017 at 11:38 AM, Jianfeng (Jeff) Zhang < jzh...@hortonworks.com> wrote: > > In that case, you can use keyword parallel to control the number of > reducer task of each mr job. The number of mapper task is determined by > the InputFormat. > > See. > http://pig.apache.org/docs/r0.16.0/basic.html > > > Best Regard, > Jeff Zhang > > > > > > On 3/18/17, 6:03 PM, "Mohammad Tariq" <donta...@gmail.com> wrote: > > >Hi Jeff, > > > >Thank you for the prompt response. However, I can't use Tez because of > >some > >reasons. > > > > > >[image: --] > > > >Tariq, Mohammad > >[image: https://]about.me/mti > ><https://about.me/mti?promo=email_sig&utm_source=product&; > utm_medium=email_ > >sig&utm_campaign=chrome_ext> > > > > > > > > > >[image: http://] > > > >Tariq, Mohammad > >about.me/mti > >[image: http://] > ><http://about.me/mti> > > > > > >On Fri, Mar 17, 2017 at 2:24 PM, Jianfeng (Jeff) Zhang < > >jzh...@hortonworks.com> wrote: > > > >> > >> I would suggest you to use tez which just launch one yarn app for one > >>pig > >> script. > >> > >> http://pig.apache.org/docs/r0.16.0/perf.html#enable-tez > >> > >> > >> > >> > >> Best Regard, > >> Jeff Zhang > >> > >> > >> > >> > >> > >> On 3/17/17, 3:24 AM, "Mohammad Tariq" <donta...@gmail.com> wrote: > >> > >> >Hi group, > >> > > >> >In any real world pig script we end up with multiple MR jobs(well, > >>most of > >> >the times). I was wondering if it's possible to allocate resources to > >> >individual MR jobs rather than assigning them at the script level > >>itself. > >> >Tried looking at multiple places. Would really appreciate some pointers > >> >regarding the same. > >> > > >> >Thank you so much for your valuable time! > >> > > >> > > >> >[image: http://] > >> > > >> >Tariq, Mohammad > >> >about.me/mti > >> >[image: http://] > >> ><http://about.me/mti> > >> > > >> > > >> > > >> >[image: --] > >> > > >> >Tariq, Mohammad > >> >[image: https://]about.me/mti > >> ><https://about.me/mti?promo=email_sig&utm_source=product&; > >> utm_medium=email_ > >> >sig&utm_campaign=chrome_ext> > >> > >> > >
