I never tried ES with Nutch 2.3 but it should be similar to setup as for 1.x:
- enable the plugin "indexer-elastic" in plugin.includes
(upgrade and rename to "indexer-elastic2" in 2.4)
- expects ES 1.4.1
- available/required options are found in the log file (hadoop.log):
ElasticIndexWriter
elastic.cluster : elastic prefix cluster
elastic.host : hostname
elastic.port : port (default 9300)
elastic.index : elastic index command
elastic.max.bulk.docs : elastic bulk index doc counts. (default 250)
elastic.max.bulk.size : elastic bulk index length. (default 2500500
~2.5MB)
Sebastian
On 02/28/2018 01:26 PM, Yash Thenuan Thenuan wrote:
> Yeah
> I was also thinking that
> Can somebody help me with nutch 2.3?
>
> On 28 Feb 2018 17:53, "Yossi Tamari" <yossi.tam...@pipl.com> wrote:
>
>> Sorry, I just realized that you're using Nutch 2.x and I'm answering for
>> Nutch 1.x. I'm afraid I can't help you.
>>
>>> -----Original Message-----
>>> From: Yash Thenuan Thenuan [mailto:rit2014...@iiita.ac.in]
>>> Sent: 28 February 2018 14:20
>>> To: user@nutch.apache.org
>>> Subject: RE: Regarding Indexing to elasticsearch
>>>
>>> IndexingJob (<batchId> | -all |-reindex) [-crawlId <id>] This is the
>> output of
>>> nutch index i have already configured the nutch-site.xml.
>>>
>>> On 28 Feb 2018 17:41, "Yossi Tamari" <yossi.tam...@pipl.com> wrote:
>>>
>>>> I suggest you run "nutch index", take a look at the returned help
>>>> message, and continue from there.
>>>> Broadly, first of all you need to configure your elasticsearch
>>>> environment in nutch-site.xml, and then you need to run nutch index
>>>> with the location of your CrawlDB and either the segment you want to
>>>> index or the directory that contains all the segments you want to
>> index.
>>>>
>>>>> -----Original Message-----
>>>>> From: Yash Thenuan Thenuan [mailto:rit2014...@iiita.ac.in]
>>>>> Sent: 28 February 2018 14:06
>>>>> To: user@nutch.apache.org
>>>>> Subject: RE: Regarding Indexing to elasticsearch
>>>>>
>>>>> All I want is to index my parsed data to elasticsearch.
>>>>>
>>>>>
>>>>> On 28 Feb 2018 17:34, "Yossi Tamari" <yossi.tam...@pipl.com> wrote:
>>>>>
>>>>> Hi Yash,
>>>>>
>>>>> The nutch index command does not have a -all flag, so I'm not sure
>>>>> what
>>>> you're
>>>>> trying to achieve here.
>>>>>
>>>>> Yossi.
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Yash Thenuan Thenuan [mailto:rit2014...@iiita.ac.in]
>>>>>> Sent: 28 February 2018 13:55
>>>>>> To: user@nutch.apache.org
>>>>>> Subject: Regarding Indexing to elasticsearch
>>>>>>
>>>>>> Can somebody please tell me what happens when we hit the bin/nutc
>>>>>> index
>>>>> -all
>>>>>> command.
>>>>>> Because I can't figure out why the write function inside the
>>>>> elastic-indexer is not
>>>>>> getting executed.
>>>>
>>>>
>>
>>
>
