Re: Controlling on which node a reducer will be executed
Thank you for your reply. I really need this feature, because this will boost up execution of my use case a lot. Could you give me a hint where to look to get a good starting point for implementation? > Hi Eduard, > > This isn't impossible, just unavailable at the moment. See > https://issues.apache.org/jira/browse/MAPREDUCE-199 if you want to > take a shot at implementing this. HBase would love to have this, I > think. > > On Mon, Aug 27, 2012 at 10:41 PM, Eduard Skaley wrote: >> Hi, >> >> i have a question concerning the execution of reducers. >> To use effectively the data locality of blocks in my use case i want to >> control on which node a reducer will be executed. >> >> In my scenario i have a chain of map-reduce jobs where each job will be >> executed by exactly N reducers. >> I want to achieve that for each job reducer 1 is executed on node 1, >> reducer 2 on node 2 and so on. >> >> Is it possible with a customization of the BlockPlacementPolicy in MRv2 ? >> Or is there some other way, maybe some indirect ? Or is it impossible ? >> >> Thanks for your help. >> Eduard > >
Re: Controlling on which node a reducer will be executed
Thank you for your reply Harsh J. I really need this feature, because this will boost up execution of my use case a lot. Could you give me a hint where to look in the sources to get a good starting point for implementation? Which classes are involved? > Hi Eduard, > > This isn't impossible, just unavailable at the moment. See > https://issues.apache.org/jira/browse/MAPREDUCE-199 if you want to > take a shot at implementing this. HBase would love to have this, I > think. > > On Mon, Aug 27, 2012 at 10:41 PM, Eduard Skaley wrote: >> Hi, >> >> i have a question concerning the execution of reducers. >> To use effectively the data locality of blocks in my use case i want to >> control on which node a reducer will be executed. >> >> In my scenario i have a chain of map-reduce jobs where each job will be >> executed by exactly N reducers. >> I want to achieve that for each job reducer 1 is executed on node 1, >> reducer 2 on node 2 and so on. >> >> Is it possible with a customization of the BlockPlacementPolicy in MRv2 ? >> Or is there some other way, maybe some indirect ? Or is it impossible ? >> >> Thanks for your help. >> Eduard > >
Re: Controlling on which node a reducer will be executed
Thank you for your reply Harsh J. I really need this feature, because this will boost up execution of my use case a lot. Could you give me a hint where to look in the sources to get a good starting point for implementation? Which classes are involved? BTW. I have problems to submit messages to this user@hadoop.apache.org mailing list. I'm getting this delivery failure mentioned in this post, but my messages don't come through. https://issues.apache.org/jira/browse/HADOOP-8062 > Hi Eduard, > > This isn't impossible, just unavailable at the moment. See > https://issues.apache.org/jira/browse/MAPREDUCE-199 if you want to > take a shot at implementing this. HBase would love to have this, I > think. > > On Mon, Aug 27, 2012 at 10:41 PM, Eduard Skaley wrote: >> Hi, >> >> i have a question concerning the execution of reducers. >> To use effectively the data locality of blocks in my use case i want to >> control on which node a reducer will be executed. >> >> In my scenario i have a chain of map-reduce jobs where each job will be >> executed by exactly N reducers. >> I want to achieve that for each job reducer 1 is executed on node 1, >> reducer 2 on node 2 and so on. >> >> Is it possible with a customization of the BlockPlacementPolicy in MRv2 ? >> Or is there some other way, maybe some indirect ? Or is it impossible ? >> >> Thanks for your help. >> Eduard > >
Re: Controlling on which node a reducer will be executed
Thank you for your reply Harsh J. I really need this feature, because this will boost up execution of my use case a lot. Could you give me a hint where to look in the sources to get a good starting point for implementation? Which classes are involved? btw. I have problems to submit messages to this mailing list. I'm getting this delivery failure mentioned in this post, but my messages don't come through. https://issues.apache.org/jira/browse/HADOOP-8062 > Hi Eduard, > > This isn't impossible, just unavailable at the moment. See > https://issues.apache.org/jira/browse/MAPREDUCE-199 if you want to > take a shot at implementing this. HBase would love to have this, I > think. > > On Mon, Aug 27, 2012 at 10:41 PM, Eduard Skaley wrote: >> Hi, >> >> i have a question concerning the execution of reducers. >> To use effectively the data locality of blocks in my use case i want to >> control on which node a reducer will be executed. >> >> In my scenario i have a chain of map-reduce jobs where each job will be >> executed by exactly N reducers. >> I want to achieve that for each job reducer 1 is executed on node 1, >> reducer 2 on node 2 and so on. >> >> Is it possible with a customization of the BlockPlacementPolicy in MRv2 ? >> Or is there some other way, maybe some indirect ? Or is it impossible ? >> >> Thanks for your help. >> Eduard > >
Re: Controlling on which node a reducer will be executed
Thank you for your reply Harsh J. I really need this feature, because this will boost up execution of my use case a lot. Could you give me a hint where to look in the sources to get a good starting point for implementation? Which classes are involved? > Hi Eduard, > > This isn't impossible, just unavailable at the moment. See > https://issues.apache.org/jira/browse/MAPREDUCE-199 if you want to > take a shot at implementing this. HBase would love to have this, I > think. > > On Mon, Aug 27, 2012 at 10:41 PM, Eduard Skaley wrote: >> Hi, >> >> i have a question concerning the execution of reducers. >> To use effectively the data locality of blocks in my use case i want to >> control on which node a reducer will be executed. >> >> In my scenario i have a chain of map-reduce jobs where each job will be >> executed by exactly N reducers. >> I want to achieve that for each job reducer 1 is executed on node 1, >> reducer 2 on node 2 and so on. >> >> Is it possible with a customization of the BlockPlacementPolicy in MRv2 ? >> Or is there some other way, maybe some indirect ? Or is it impossible ? >> >> Thanks for your help. >> Eduard > >
Re: Controlling on which node a reducer will be executed
Thank you for your reply Harsh J. I really need this feature, because this will boost up execution of my use case a lot. Could you give me a hint where to look in the sources to get a good starting point for implementation? Which classes are involved? > Hi Eduard, > > This isn't impossible, just unavailable at the moment. See > https://issues.apache.org/jira/browse/MAPREDUCE-199 if you want to > take a shot at implementing this. HBase would love to have this, I > think. > > On Mon, Aug 27, 2012 at 10:41 PM, Eduard Skaley wrote: >> Hi, >> >> i have a question concerning the execution of reducers. >> To use effectively the data locality of blocks in my use case i want to >> control on which node a reducer will be executed. >> >> In my scenario i have a chain of map-reduce jobs where each job will be >> executed by exactly N reducers. >> I want to achieve that for each job reducer 1 is executed on node 1, >> reducer 2 on node 2 and so on. >> >> Is it possible with a customization of the BlockPlacementPolicy in MRv2 ? >> Or is there some other way, maybe some indirect ? Or is it impossible ? >> >> Thanks for your help. >> Eduard > >
Re: Controlling on which node a reducer will be executed
Thank you for your reply Harsh J. I really need this feature, because this will boost up execution of my use case a lot. Could you give me a hint where to look in the sources to get a good starting point for implementation? Which classes are involved? > Hi Eduard, > > This isn't impossible, just unavailable at the moment. See > https://issues.apache.org/jira/browse/MAPREDUCE-199 if you want to > take a shot at implementing this. HBase would love to have this, I > think. > > On Mon, Aug 27, 2012 at 10:41 PM, Eduard Skaley wrote: >> Hi, >> >> i have a question concerning the execution of reducers. >> To use effectively the data locality of blocks in my use case i want to >> control on which node a reducer will be executed. >> >> In my scenario i have a chain of map-reduce jobs where each job will be >> executed by exactly N reducers. >> I want to achieve that for each job reducer 1 is executed on node 1, >> reducer 2 on node 2 and so on. >> >> Is it possible with a customization of the BlockPlacementPolicy in MRv2 ? >> Or is there some other way, maybe some indirect ? Or is it impossible ? >> >> Thanks for your help. >> Eduard > >
Re: Controlling on which node a reducer will be executed
Thank you for your reply Harsh J. I really need this feature, because this will boost up execution of my use case a lot. Could you give me a hint where to look in the sources to get a good starting point for implementation? Which classes are involved? > Hi Eduard, > > This isn't impossible, just unavailable at the moment. See > https://issues.apache.org/jira/browse/MAPREDUCE-199 if you want to > take a shot at implementing this. HBase would love to have this, I > think. > > On Mon, Aug 27, 2012 at 10:41 PM, Eduard Skaley wrote: >> Hi, >> >> i have a question concerning the execution of reducers. >> To use effectively the data locality of blocks in my use case i want to >> control on which node a reducer will be executed. >> >> In my scenario i have a chain of map-reduce jobs where each job will be >> executed by exactly N reducers. >> I want to achieve that for each job reducer 1 is executed on node 1, >> reducer 2 on node 2 and so on. >> >> Is it possible with a customization of the BlockPlacementPolicy in MRv2 ? >> Or is there some other way, maybe some indirect ? Or is it impossible ? >> >> Thanks for your help. >> Eduard > >
Re: Controlling on which node a reducer will be executed
Hi Eduard, This isn't impossible, just unavailable at the moment. See https://issues.apache.org/jira/browse/MAPREDUCE-199 if you want to take a shot at implementing this. HBase would love to have this, I think. On Mon, Aug 27, 2012 at 10:41 PM, Eduard Skaley wrote: > Hi, > > i have a question concerning the execution of reducers. > To use effectively the data locality of blocks in my use case i want to > control on which node a reducer will be executed. > > In my scenario i have a chain of map-reduce jobs where each job will be > executed by exactly N reducers. > I want to achieve that for each job reducer 1 is executed on node 1, > reducer 2 on node 2 and so on. > > Is it possible with a customization of the BlockPlacementPolicy in MRv2 ? > Or is there some other way, maybe some indirect ? Or is it impossible ? > > Thanks for your help. > Eduard -- Harsh J
