hough a combiner in the final m/r pass would really speed
> up the hadoop shuffle.
>
> Lance
>
> From: "Ted Dunning"
> To: user@hadoop.apache.org
> Sent: Wednesday, October 10, 2012 11:13:36 PM
> Subject: Re: Hadoop/Lucene + Solr architecture suggesti
. It's a disk bandwidth v.s. network bandwidth
problem. Although a combiner in the final m/r pass would really speed
up the hadoop shuffle.
Lance
From: "Ted Dunning"
To: user@hadoop.apache.org
Sent: Wednesday, October 10, 2012 11:13:36 PM
Subject: Re: Hadoo
months ago at least...
>>
>> Thanks,
>> --tim
>>
>> > - Original Message -
>> > | From: "Ted Dunning" < tdunn...@maprtech.com>
>> > | To: user@hadoop.apache.org
>> > | Cc: "Hadoop User" < user@hadoop.apache.org>
o work with a decent-sized (e.g. >
> 250 node) cluster? Even a trivial cluster seems to crush SolrCloud
> from a few months ago at least...
>
> Thanks,
> --tim
>
> > - Original Message -
> > | From: "Ted Dunning"
> > | To: user@hadoop.apache
: user@hadoop.apache.org
> > | Cc: "Hadoop User"
> > | Sent: Wednesday, October 10, 2012 7:58:57 AM
> > | Subject: Re: Hadoop/Lucene + Solr architecture suggestions?
> > |
> > | I prefer to create indexes in the reducer personally.
> > |
> > | Al
Sent: Wednesday, October 10, 2012 7:58:57 AM
> | Subject: Re: Hadoop/Lucene + Solr architecture suggestions?
> |
> | I prefer to create indexes in the reducer personally.
> |
> | Also you can avoid the copies if you use an advanced hadoop-derived
> | distro. Email me off list for de
- Original Message -
> | From: "Ted Dunning"
> | To: user@hadoop.apache.org
> | Cc: "Hadoop User"
> | Sent: Wednesday, October 10, 2012 7:58:57 AM
> | Subject: Re: Hadoop/Lucene + Solr architecture suggestions?
> |
> | I prefer to create indexes i
ednesday, October 10, 2012 7:58:57 AM
| Subject: Re: Hadoop/Lucene + Solr architecture suggestions?
|
| I prefer to create indexes in the reducer personally.
|
| Also you can avoid the copies if you use an advanced hadoop-derived
| distro. Email me off list for details.
|
| Sent from my iPhone
|
I prefer to create indexes in the reducer personally.
Also you can avoid the copies if you use an advanced hadoop-derived distro.
Email me off list for details.
Sent from my iPhone
On Oct 9, 2012, at 7:47 PM, Mark Kerzner wrote:
> Hi,
>
> if I create a Lucene index in each mapper, locally,
Hi Mark,
I don't know Lucene/Solr very well but your question made me remember the
lily project: http://www.lilyproject.org/lily/index.html. They use
hadoop/hbase and solr to provide a searchable data management platform.
Maybe you will find ideas in their documentation.
BR,
Ivan
2012/10/10 Mar
Hi,
if I create a Lucene index in each mapper, locally, then copy them to under
/jobid/mapid1, /jodid/mapid2, and then in the reducers copy them to some
Solr machine (perhaps even merging), does such architecture makes sense, to
create a searchable index with Hadoop?
Are there links for similar a
11 matches
Mail list logo
