Did another test and got this error:
2009-06-25 21:19:44,663 ERROR mapred.EagerTaskInitializationListener - Job
initialization failed:
java.lang.IllegalArgumentException: Pathname
/d:/Bii/nutch/logs/history/user/_logs/history/localhost_1245956549829_job_200906252102_0001_pc-xxx%xxx_inject+urls
fro
2009/6/24 Andrzej Bialecki
> MilleBii wrote:
>
>> What's also i have discovered
>> + hadoop (script) works with unix like paths and works fine on windows
>> + nutch (script) works with Windows paths
>>
>
> bin/nutch works with Windows paths? I think this could happen only by
> accident - both scr
MilleBii wrote:
What's also i have discovered
+ hadoop (script) works with unix like paths and works fine on windows
+ nutch (script) works with Windows paths
bin/nutch works with Windows paths? I think this could happen only by
accident - both scripts work with Cygwin paths. On the other hand
What's also i have discovered
+ hadoop (script) works with unix like paths and works fine on windows
+ nutch (script) works with Windows paths
Could it be that there is some incompatibility because one works unix like
paths and not the other ???
2009/6/24 MilleBii
> Actually tried and it fail
Actually tried and it fails but this is what I found :
bin/hadoop-config.sh does the conversion from relative to absolute path
this="$0"
while [ -h "$this" ]; do
ls=`ls -ld "$this"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '.*/.*' > /dev/null; then
this="$link"
else
th
Yes I'm using both relative path & cygwin under windows. so /d: is not
introduced by me, but either nutch or hadoop.
Regarding the cygwin path you are righ... actually where I lost quite some
time.
OK will try absolute paths and let you know.
-MilleBii-
2009/6/24 Andrzej Bialecki
> MilleBii
MilleBii wrote:
HLPPP !!!
Stuck for 3 days on not able to start any nutch job.
hdfs works fine, ie I can put & look at files.
When i start nutch crawl, I get the following error
Job initialization failed:
java.lang.IllegalArgumentException: Pathname
/d:/Bii/nutch/logs/history/user/_log
HLPPP !!!
Stuck for 3 days on not able to start any nutch job.
hdfs works fine, ie I can put & look at files.
When i start nutch crawl, I get the following error
Job initialization failed:
java.lang.IllegalArgumentException: Pathname
/d:/Bii/nutch/logs/history/user/_logs/history/localhos
Looks like I just needed to transfer from the local filesystem to hdfs:
Is it safe to transfer a crawl directory (and subs) from the local file
system to hdfs and start crawling again ?
1. hadoop fs -put crawl crawl
2. nutch generate crawl/crawldb crawl/segments -topN 500 (where now it
should use
I have newly installed hadoop in a distributed single node configuration.
When I run nutch commands it is looking for files my user home directory
and not at the nutch directory ?
How can I change this ?
--
-MilleBii-
10 matches
Mail list logo
