Abidari wrote:
>
> Ian
>
> Can you please help with this? I have upgraded to Nutch 0.9. I am able to
> run Nutch in a standalone mode, ie without hadoop. But with hadoop I get
> the
> error "Generator: 0 records selected for fetching, exiting ...".
> I have performed this step - bin/hadoop dfs -put urls urls. And upon
> running bin/hadoop dfs -ls, I see that urls is there in the dfs
>
> Output of Crawl.
>
> crawl started in: crawl
> rootUrlDir = urls
> threads = 10
> depth = 3
> topN = 50
> Injector: starting
> Injector: crawlDb: crawl/crawldb
> Injector: urlDir: urls
> Injector: Converting injected urls to crawl db entries.
> Injector: Merging injected urls into crawl db.
> Injector: done
> Generator: Selecting best-scoring urls due for fetch.
> Generator: starting
> Generator: segment: crawl/segments/20070419134155
> Generator: filtering: false
> Generator: topN: 50
> Generator: 0 records selected for fetching, exiting ...
> Stopping at depth=0 - no more URLs to fetch.
> No URLs to fetch - check your seed list and URL filters.
> crawl finished: crawl
>
>
Hi Abidari,
I ran into this problem as well.
I'm not sure if it is related, but when I examine the stderr of the mapper
job I see:
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /opt/nutch/search/logs (Is a directory)
at java.io.FileOutputStream.openAppend(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:289)
at
org.apache.log4j.FileAppender.activateOptions(FileAppender.java:163)
at
org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:215)
at
org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:256)
at
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:132)
at
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
at
org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:654)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:612)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:509)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:415)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:441)
at
org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:468)
at org.apache.log4j.LogManager.<clinit>(LogManager.java:122)
at org.apache.log4j.Logger.getLogger(Logger.java:104)
at
org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
at
org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
at
org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529)
at
org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:370)
at
org.apache.hadoop.mapred.TaskTracker.<clinit>(TaskTracker.java:82)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1423)
log4j:ERROR Either File or DatePattern options are not set for appender
[DRFA].
which points to log4j being mis configured.
abidari, did you get any further with this? Andrei any hints???
--
View this message in context:
http://www.nabble.com/Nutch-0.9---Generator%3A-0-records-selected-for-fetching%2C-exiting-tf3609078.html#a10757841
Sent from the Nutch - User mailing list archive at Nabble.com.
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
