RE: hadoop configuarions for SOLR-1301 patch
Hi, Patch(SOLR-1301) provides distributed indexing (using Hadoop). Now I have Hadoop cluster with 1 master and 2 slaves. Also I have applied above path to solr and build solr. So how I integrate above solr executables with Hadoop cluster? Can u please tell what are the steps for this. Shall I just have copy solr war to Hadoop cluster or what else ? (Note: I have two setup : 1. Hadoop setup 2. Solr setup) So to run distributed indexing how to bridge these two setup? Thanks -Pravin -Original Message- From: Jason Rutherglen [mailto:jason.rutherg...@gmail.com] Sent: Friday, October 16, 2009 7:45 AM To: solr-user@lucene.apache.org Subject: Re: hadoop configuarions for SOLR-1301 patch Hi Pravin, You'll need to setup a Hadoop cluster which is independent of SOLR-1301. 1301 is for building Solr indexes only, so there isn't a master and slave. After building the indexes one needs to provision the indexes to Solr servers. In my case I only have slaves because I'm not incrementally indexing on the Hadoop generated shards. 1301 does need a Hadoop specific unit test, which I got started and need to complete, that could help a little in understanding. -J On Wed, Oct 14, 2009 at 5:45 AM, Pravin Karne pravin_ka...@persistent.co.in wrote: Hi, I am using SOLR-1301 path. I have build the solr with given patch. But I am not able to configure Hadoop for above war. I want to run solr(create index) with 3 nodes (1+2) cluster. How to do the Hadoop configurations for above patch? How to set master and slave? Thanks -Pravin DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
Re: hadoop configuarions for SOLR-1301 patch
Pravin, Hadoop allows jaring libraries into a jar under /lib. So I have my code, then put apache-solr-core-1.4-dev.jar, and all the other Solr related jars into it using ant clean jar jar uvf target/myhadoopsolr.jar lib/*. Then on your Hadoop cluster run something like: /app/hadoop/bin/hadoop jar myhadoopsolr.jar com.company.HadoopSolrIndexer As far as a more comprehensive installation instructions, like say creating a Hadoop cluster from scratch and running a this with, it will be best to start a wiki page, which I'll do shortly. -J On Thu, Oct 15, 2009 at 11:01 PM, Pravin Karne pravin_ka...@persistent.co.in wrote: Hi, Patch(SOLR-1301) provides distributed indexing (using Hadoop). Now I have Hadoop cluster with 1 master and 2 slaves. Also I have applied above path to solr and build solr. So how I integrate above solr executables with Hadoop cluster? Can u please tell what are the steps for this. Shall I just have copy solr war to Hadoop cluster or what else ? (Note: I have two setup : 1. Hadoop setup 2. Solr setup) So to run distributed indexing how to bridge these two setup? Thanks -Pravin -Original Message- From: Jason Rutherglen [mailto:jason.rutherg...@gmail.com] Sent: Friday, October 16, 2009 7:45 AM To: solr-user@lucene.apache.org Subject: Re: hadoop configuarions for SOLR-1301 patch Hi Pravin, You'll need to setup a Hadoop cluster which is independent of SOLR-1301. 1301 is for building Solr indexes only, so there isn't a master and slave. After building the indexes one needs to provision the indexes to Solr servers. In my case I only have slaves because I'm not incrementally indexing on the Hadoop generated shards. 1301 does need a Hadoop specific unit test, which I got started and need to complete, that could help a little in understanding. -J On Wed, Oct 14, 2009 at 5:45 AM, Pravin Karne pravin_ka...@persistent.co.in wrote: Hi, I am using SOLR-1301 path. I have build the solr with given patch. But I am not able to configure Hadoop for above war. I want to run solr(create index) with 3 nodes (1+2) cluster. How to do the Hadoop configurations for above patch? How to set master and slave? Thanks -Pravin DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
Re: hadoop configuarions for SOLR-1301 patch
Hi Pravin, You'll need to setup a Hadoop cluster which is independent of SOLR-1301. 1301 is for building Solr indexes only, so there isn't a master and slave. After building the indexes one needs to provision the indexes to Solr servers. In my case I only have slaves because I'm not incrementally indexing on the Hadoop generated shards. 1301 does need a Hadoop specific unit test, which I got started and need to complete, that could help a little in understanding. -J On Wed, Oct 14, 2009 at 5:45 AM, Pravin Karne pravin_ka...@persistent.co.in wrote: Hi, I am using SOLR-1301 path. I have build the solr with given patch. But I am not able to configure Hadoop for above war. I want to run solr(create index) with 3 nodes (1+2) cluster. How to do the Hadoop configurations for above patch? How to set master and slave? Thanks -Pravin DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
hadoop configuarions for SOLR-1301 patch
Hi, I am using SOLR-1301 path. I have build the solr with given patch. But I am not able to configure Hadoop for above war. I want to run solr(create index) with 3 nodes (1+2) cluster. How to do the Hadoop configurations for above patch? How to set master and slave? Thanks -Pravin DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
Re: hadoop configuarions for SOLR-1301 patch
On Wed, Oct 14, 2009 at 6:15 PM, Pravin Karne pravin_ka...@persistent.co.in wrote: Hi, I am using SOLR-1301 path. I have build the solr with given patch. But I am not able to configure Hadoop for above war. I want to run solr(create index) with 3 nodes (1+2) cluster. How to do the Hadoop configurations for above patch? How to set master and slave? Pravin, questions on specific patches are best asked on the Jira issue. -- Regards, Shalin Shekhar Mangar.
