Hi Guys,
I have a cluster of 2 machine : Linux; Java 1.6
I started a crawl on a list of few website only. I used the command
bin/nutch crawl urls/site1 -dir crawld -depth 10 -topN 100000 -threads 30
I had an error on my 6th depth.
CrawlDb update: starting
CrawlDb update: db: crawld/crawldb
CrawlDb update: segments: [crawld/segments/20070627053531]
CrawlDb update: additions allowed: true
CrawlDb update: URL normalizing: true
CrawlDb update: URL filtering: true
CrawlDb update: Merging segment data into db.
task_0035_m_000005_0: log4j:ERROR setFile(null,true) call failed.
task_0035_m_000005_0: java.io.FileNotFoundException:
/data/sengine/search/bin/../logs (Is a directory)
task_0035_m_000005_0: at java.io.FileOutputStream.openAppend(Native
Method)
task_0035_m_000005_0: at java.io.FileOutputStream.<init>(
FileOutputStream.java:177)
task_0035_m_000005_0: at java.io.FileOutputStream.<init>(
FileOutputStream.java:102)
task_0035_m_000005_0: at org.apache.log4j.FileAppender.setFile(
FileAppender.java:289)
task_0035_m_000005_0: at org.apache.log4j.FileAppender.activateOptions(
FileAppender.java:163)
task_0035_m_000005_0: at
org.apache.log4j.DailyRollingFileAppender.activateOptions(
DailyRollingFileAppender.java:215)
task_0035_m_000005_0: at org.apache.log4j.config.PropertySetter.activate(
PropertySetter.java:256)
task_0035_m_000005_0: at
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java
:132)
task_0035_m_000005_0: at
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:96)
task_0035_m_000005_0: at
org.apache.log4j.PropertyConfigurator.parseAppender(
PropertyConfigurator.java:654)
task_0035_m_000005_0: at
org.apache.log4j.PropertyConfigurator.parseCategory(
PropertyConfigurator.java:612)
task_0035_m_000005_0: at
org.apache.log4j.PropertyConfigurator.configureRootCategory(
PropertyConfigurator.java:509)
task_0035_m_000005_0: at org.apache.log4j.PropertyConfigurator.doConfigure
(PropertyConfigurator.java:415)
task_0035_m_000005_0: at org.apache.log4j.PropertyConfigurator.doConfigure
(PropertyConfigurator.java:441)
task_0035_m_000005_0: at
org.apache.log4j.helpers.OptionConverter.selectAndConfigure(
OptionConverter.java:468)
task_0035_m_000005_0: at org.apache.log4j.LogManager.<clinit>(
LogManager.java:122)
task_0035_m_000005_0: at org.apache.log4j.Logger.getLogger(Logger.java
:104)
task_0035_m_000005_0: at
org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
task_0035_m_000005_0: at org.apache.commons.logging.impl.Log4JLogger
.<init>(Log4JLogger.java:65)
task_0035_m_000005_0: at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
task_0035_m_000005_0: at
sun.reflect.NativeConstructorAccessorImpl.newInstance(
NativeConstructorAccessorImpl.java:39)
task_0035_m_000005_0: at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
DelegatingConstructorAccessorImpl.java:27)
task_0035_m_000005_0: at java.lang.reflect.Constructor.newInstance(
Constructor.java:513)
task_0035_m_000005_0: at
org.apache.commons.logging.impl.LogFactoryImpl.newInstance(
LogFactoryImpl.java:529)
task_0035_m_000005_0: at
org.apache.commons.logging.impl.LogFactoryImpl.getInstance(
LogFactoryImpl.java:235)
task_0035_m_000005_0: at org.apache.commons.logging.LogFactory.getLog(
LogFactory.java:370)
task_0035_m_000005_0: at org.apache.hadoop.mapred.TaskTracker.<clinit>(
TaskTracker.java:82)
task_0035_m_000005_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(
TaskTracker.java:1423)
task_0035_m_000005_0: log4j:ERROR Either File or DatePattern options are not
set for appender [DRFA].
task_0035_m_000004_0: log4j:ERROR setFile(null,true) call failed.
I notice that every time I had a nutch error when i use my cluster it give
me this exception regardings the log4j. I don't understand why. Any idea ?
Besides my crawl stopped and I don't why too. How can i get any details ?
cheers
E
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
