[ https://issues.apache.org/jira/browse/NUTCH-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17452112#comment-17452112 ]
Lewis John McGibbney edited comment on NUTCH-2910 at 12/2/21, 1:49 AM: ----------------------------------------------------------------------- Hi [~snagel] thanks for reviewing this and chiming in so quickly. I remembered looking at this before and your explanation is where I eventually ended up... it had just slipped my mind. The good thing is that nothing is wrong here so ultimately I will resolve this issue as not a problem however I think making use HADOOP-15005 tags would help a lot. It's just time consuming to go through nutch-site.xml and tag everything. I can maybe have some interns divide and conquer this work. I made some trivial updates to the wiki page you referenced above. I totally forgot that even existed as well... doh. was (Author: lewismc): Hi [~snagel] thanks for reviewing this and chiming in so quickly. I remembered looking at this before and your explanation is where I eventually ended up... it had just slipped my mind. The good thing is that nothing is wrong here so ultimately I will resolve this issue as not a problem however I will open a ticket to make use of HADOOP-15005 tags. i think this will help a lot. It's just time consuming to do though and do that. I can maybe have some interns divide and conquer this work. I made some trivial updates to the wiki page you referenced above. I totally forgot that even existed as well... doh. > FetchItemQueues overloaded constructor also interprets fetcher timeout as -1 > e.g. no-timeout. > --------------------------------------------------------------------------------------------- > > Key: NUTCH-2910 > URL: https://issues.apache.org/jira/browse/NUTCH-2910 > Project: Nutch > Issue Type: Bug > Components: fetcher > Affects Versions: 1.18 > Reporter: Lewis John McGibbney > Assignee: Lewis John McGibbney > Priority: Major > Fix For: 1.19 > > > The FetchItemQueues overloaded constructor [attempts to obtain the > *NON-EXISTENT* _fetcher.timelimit_ configuration > property|https://github.com/apache/nutch/blob/master/src/java/org/apache/nutch/fetcher/FetchItemQueues.java#L84]. > {code:java} > this.timelimit = conf.getLong("fetcher.timelimit", -1); > {code} > As you can see a default value of -1 is provided. The first parameter is > however wrong. It should instead reference the following configuration > property. > {code:xml} > <property> > <name>fetcher.timelimit.mins</name> > <value>-1</value> > <description>This is the number of minutes allocated to the fetching. > Once this value is reached, any remaining entry from the input URL list is > skipped > and all active queues are emptied. The default value of -1 deactivates the > time limit. > </description> > </property> > {code} > Note, *_fetcher.timelimit.mins_* > I think that this essentially means the Fetcher has no time limit which is > ofcourse not desired. -- This message was sent by Atlassian Jira (v8.20.1#820001)