Hi Nima

Thanks for reminding me about this JIRA issue, it hasn't been commented on
for some time and I'd forgotten about it. Judging by the discussion on
NUTCH-978 <https://issues.apache.org/jira/browse/NUTCH-978> things got
stuck when Emmanuel tried to get in touch with Emir (who in the meantime
seems to have stopped using Nutch - see
http://www.atlantbh.com/book-review-web-crawling-and-data-mining-with-apache-nutch/
).

It would be a good thing to get in touch with him indeed, alternatively
Albin's plugin could be a good starting point. There clearly is a need for
such a functionality and quite a few people keen to make it happen.

Thanks

Julien


On 25 September 2014 18:19, Nima Falaki <nfal...@popsugar.com> wrote:

> And the reason why I think this is because of this ticket (Look at the
> conversation at the bottom between Emmanuel and Lewis John)
>
> https://issues.apache.org/jira/browse/NUTCH-978
>
> On Thu, Sep 25, 2014 at 8:44 AM, Nima Falaki <nfal...@popsugar.com> wrote:
>
>> Hi Julien:
>>
>> I was under the impression that the nutch community was going to use a
>> generic xls parser? This one.
>> http://www.atlantbh.com/precise-data-extraction-with-apache-nutch/ Is
>> the nutch community going to use this?
>>
>>
>>
>> On Thu, Sep 25, 2014 at 5:49 AM, Julien Nioche <
>> lists.digitalpeb...@gmail.com> wrote:
>>
>>> Hi Albin,
>>>
>>> You don't have to have a separate plugin for each html structure you
>>> want to parse. You can have a single plugin with multiple HTMLParseFilters.
>>>
>>> Having a generic extractor with the extraction logic configured in an
>>> external file is definitely a good idea and would make a great contribution
>>> to the project. In a nutshell, you haven't missed anything and that wheel
>>> definitely needs inventing ;-)
>>>
>>> Best
>>>
>>> Julien
>>>
>>>
>>> On 25 September 2014 09:24, Albin Vigier <albinsc...@gmail.com> wrote:
>>>
>>>> Hello everybody,
>>>>
>>>> I'm just wondering if it is possible to fetch specific metadata with
>>>> an existing nutch plugin.
>>>>
>>>> Let's take an example.
>>>> I want to extract some metadata from "div" or "td" tags from html
>>>> pages that have specific ids and name them the way I like (this is
>>>> done at parser time).
>>>> Then, at indexer time, I would use index-metadata (a very good plugin)
>>>> to add my custom metadata.
>>>>
>>>> Currently from what I've seen on the wiki and by quickly analyzing
>>>> plugins I suppose I have to code my own plugin each time I've got a
>>>> new site (with a new html structure). I've already done that by using
>>>> a node walker in a custom htmlParseFilter but the extraction can be a
>>>> little bit boring :)
>>>>
>>>> So on my side i've coded a little plugin that enables me to specify
>>>> xpaths in an xml file. But before diving into more functionalities I'm
>>>> just wondering if I did not missed something.
>>>> This work allowed me to explore some nutch aspects but I don't want to
>>>> reinvent the wheel or miss something.
>>>>
>>>> Albin
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> Open Source Solutions for Text Engineering
>>>
>>> http://digitalpebble.blogspot.com/
>>> http://www.digitalpebble.com
>>> http://twitter.com/digitalpebble
>>>
>>
>>
>>
>> --
>>
>>
>>
>> Nima Falaki
>> Software Engineer
>> nfal...@popsugar.com
>>
>>
>
>
> --
>
>
>
> Nima Falaki
> Software Engineer
> nfal...@popsugar.com
>
>


-- 

Open Source Solutions for Text Engineering

http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
http://twitter.com/digitalpebble

Reply via email to