Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-17 Thread Emmanuel Bernard
the case for Map/Reduce v2, but I haven't found a > definitive reference in the docs yet. YARN is architected to split resource > management and job scheduling/monitoring into different pieces, but I think > task execution is the same as MRv1. > > Thanks, > Alan > >

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-17 Thread Gustavo Fernandes
> Thanks, > Alan > > -- > > *From: *"Emmanuel Bernard" > *To: *"infinispan -Dev List" > *Sent: *Monday, March 17, 2014 11:31:34 AM > *Subject: *Re: [infinispan-dev] Infinispan - Hadoop integration > > > G

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-17 Thread Alan Field
ard" > To: "infinispan -Dev List" > Sent: Monday, March 17, 2014 11:31:34 AM > Subject: Re: [infinispan-dev] Infinispan - Hadoop integration > Got it now. > That being said, if Alan is correct (one JVM per M/R task run per node), we > will need to implement C/S lo

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-17 Thread Emmanuel Bernard
Got it now. That being said, if Alan is correct (one JVM per M/R task run per node), we will need to implement C/S local key and keyset lookup. Emmanuel On 14 Mar 2014, at 12:34, Sanne Grinovero wrote: > On 14 March 2014 09:06, Emmanuel Bernard wrote: >> >> >>> On 13 mars 2014, at 23:39, Sa

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-14 Thread Alan Field
migrated to Spark. It's certainly being billed as the replacement, or at least the future of Map/Reduce. Thanks, Alan [1] https://spark.apache.org/ - Original Message - > From: "Mircea Markus" > To: "infinispan -Dev List" > Sent: Friday, March 14,

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-14 Thread Mircea Markus
On Mar 14, 2014, at 9:06, Emmanuel Bernard wrote: > > >> On 13 mars 2014, at 23:39, Sanne Grinovero wrote: >> >>> On 13 March 2014 22:19, Mircea Markus wrote: >>> On Mar 13, 2014, at 22:17, Sanne Grinovero wrote: > On 13 March 2014 22:05, Mircea Markus wrote: > >

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-14 Thread Sanne Grinovero
On 14 March 2014 09:06, Emmanuel Bernard wrote: > > >> On 13 mars 2014, at 23:39, Sanne Grinovero wrote: >> >>> On 13 March 2014 22:19, Mircea Markus wrote: >>> On Mar 13, 2014, at 22:17, Sanne Grinovero wrote: > On 13 March 2014 22:05, Mircea Markus wrote: > > On Mar 13,

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-14 Thread Emmanuel Bernard
> On 13 mars 2014, at 23:39, Sanne Grinovero wrote: > >> On 13 March 2014 22:19, Mircea Markus wrote: >> >>> On Mar 13, 2014, at 22:17, Sanne Grinovero wrote: >>> On 13 March 2014 22:05, Mircea Markus wrote: On Mar 13, 2014, at 20:59, Ales Justin wrote: >> - also

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-13 Thread Sanne Grinovero
On 13 March 2014 22:19, Mircea Markus wrote: > > On Mar 13, 2014, at 22:17, Sanne Grinovero wrote: > >> On 13 March 2014 22:05, Mircea Markus wrote: >>> >>> On Mar 13, 2014, at 20:59, Ales Justin wrote: >>> > - also important to notice that we will have both an Hadoop and an > Infinisp

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-13 Thread Mircea Markus
On Mar 13, 2014, at 22:17, Sanne Grinovero wrote: > On 13 March 2014 22:05, Mircea Markus wrote: >> >> On Mar 13, 2014, at 20:59, Ales Justin wrote: >> - also important to notice that we will have both an Hadoop and an Infinispan cluster running in parallel: the user will interact

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-13 Thread Sanne Grinovero
On 13 March 2014 22:05, Mircea Markus wrote: > > On Mar 13, 2014, at 20:59, Ales Justin wrote: > >>> - also important to notice that we will have both an Hadoop and an >>> Infinispan cluster running in parallel: the user will interact with the >>> former in order to run M/R tasks. Hadoop will u

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-13 Thread Mircea Markus
On Mar 13, 2014, at 20:59, Ales Justin wrote: >> - also important to notice that we will have both an Hadoop and an >> Infinispan cluster running in parallel: the user will interact with the >> former in order to run M/R tasks. Hadoop will use Infinispan (integration >> achieved through Input

Re: [infinispan-dev] Infinispan - Hadoop integration

2014-03-13 Thread Ales Justin
> - also important to notice that we will have both an Hadoop and an Infinispan > cluster running in parallel: the user will interact with the former in order > to run M/R tasks. Hadoop will use Infinispan (integration achieved through > InputFormat and OutputFormat ) in order to get the data to

[infinispan-dev] Infinispan - Hadoop integration

2014-03-13 Thread Mircea Markus
Hi, I had a very good conversation with Jonathan Halliday, Sanne and Emmanuel around the integration between Infinispan and Hadoop. Just to recap, the goal is to be able to run Hadoop M/R tasks on data that is stored in Infinispan in order to gain speed. (once we have a prototype in place, one

[infinispan-dev] Infinispan - Hadoop integration

2014-03-13 Thread Mircea Markus
Hi, I had a very good conversation with Jonathan Halliday, Sanne and Emmanuel around the integration between Infinispan and Hadoop. Just to recap, the goal is to be able to run Hadoop M/R tasks on data that is stored in Infinispan in order to gain speed. (once we have a prototype in place, one