Hi, I think HDFS now works but I have another problem...
I execute the following command and I get the following error...
bin/hadoop jar hadoop-*-examples.jar wordcount
/tmp/hadoop-a/dfs/tmp/tmp/LICENSE.txt /tmp/hadoop-a/dfs/1.txt
org.apache.hadoop.mapred.InvalidInputException: Input path doesnt exist :
/tmp/hadoop-a/dfs/tmp/tmp/LICENSE.txt
at
org.apache.hadoop.mapred.InputFormatBase.validateInput(InputFormatBase.java:138)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:315)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:398)
at org.apache.hadoop.examples.WordCount.main(WordCount.java:143)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
I dont know why but the file is in that directory...
Thank you
Richard Yang <[EMAIL PROTECTED]> έγραψε:
ok. this means HDFS is not functioning normally. it could be a number of
problems. we need to examine files in 'logs' folder.
1. reboot your machine.
2. go into the 'logs' folder and delete all log files (rm -rf *)
3. start Hadoop by start-all.sh
Hadoop would populate the 'log' folder again.
in particular, could you post the contents of
hadoop-***-namenode-***.log
hadoop-***-datanode-***.log
we should see the reason why HDFS does not start correctly in either, or both
files.
-----Original Message-----
From: Andreas Papadakis [mailto:[EMAIL PROTECTED]
Sent: Sat 3/24/2007 10:50 PM
To: [email protected]
Subject: Re: RE: RE: Re: Hadoop for perl
I execute the command
bin/hadoop dfs -ls myfs
and I get
Bad connection to FS. command aborted.
How can I fix this problem...?
Richard Yang ??????: hello,
when you do
bin/hadoop dfs -ls myfs
it should say
1 item(s) found
LICENSE.txt
if you have no problem moving files/folder to and away from HDFS, this should
indicate HDFS dfs is functioning normally. then,
how about
bin/hadoop jar hadoop-*-examples.jar wordcount myfs/LICENSE.txt out
and the following comman
bin/hadoop jar hadoop-*-examples.jar wordcount ./myfs/LICENSE.txt ./myFS/out
I have already executed ./bin/start-all.sh and I have formatted ./myFS/
Richard Yang ??????:
By the error message, it looks like 'workingdirectory' is out of touch. Is
HDFS dfs portion functioning normally??
Best Regards
Richard Yang
[EMAIL PROTECTED]
[EMAIL PROTECTED]
-----Original Message-----
From: Andreas Papadakis [mailto:[EMAIL PROTECTED]
Sent: Friday, March 23, 2007 1:18 PM
To: [email protected]
Subject: Re: Re: Hadoop for perl
Is there any other way to use hadoop with perl?
I read that I can use it with python(using jython)... I tried the example
but jython cannot find the module org.apache.hadoop.fs...(I dont know much
about java). Anyone knows where can I find that module and where should I
place it??
Also, I tried the java example and I get too many errors....
I used the command
bin/hadoop jar hadoop-*-examples.jar wordcount mydfs/LICENSE.txt mydfs/a
but I get the following errors:
java.lang.RuntimeException: java.net.SocketTimeoutException: timed out
waiting for rpc response
at
org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:247)
at org.apache.hadoop.mapred.JobConf.setInputPath(JobConf.java:150)
at org.apache.hadoop.examples.WordCount.main(WordCount.java:137)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver
.java:71)
at
org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143)
at
org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
response
at org.apache.hadoop.ipc.Client.call(Client.java:469)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164)
at org.apache.hadoop.dfs.$Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:248)
at org.apache.hadoop.dfs.DFSClient.(DFSClient.java:106)
at
org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSystem
.java:65)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:160)
at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:119)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:91)
at
org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:243)
Anyone can help me???
Thank you
Erik Hetzner ??????: At Fri, 23 Mar 2007 20:58:38
+0100,
Maximilian Sch??fmann wrote:
>
> > I want to ask if there is an implementation of Hadoop for perl...or
> > something similar to Hadoop for python
>
> Well, there is one for Ruby - at least the MapReduce part: Starfish
>
> http://rufy.com/starfish/doc/
Starfish isn???t (or wasn???t last time I
looked) really much like
MapReduce at all.
best,
Erik Hetzner
---------------------------------
?????????????? Yahoo!
?????????? ?? ?????????? ???? ???? (spam); ?? Yahoo! Mail ???????? ???
???????? ?????? ????????? ???? ??? ??????????? ?????????
http://login.yahoo.com/config/mail?.intl=gr
---------------------------------
?????????????? Yahoo!
?????????? ?? ?????????? ???? ???? (spam); ?? Yahoo! Mail ???????? ???
???????? ?????? ????????? ???? ??? ??????????? ?????????
http://login.yahoo.com/config/mail?.intl=gr
Richard Yang ??????: hello,
when you do
bin/hadoop dfs -ls myfs
it should say
1 item(s) found
LICENSE.txt
if you have no problem moving files/folder to and away from HDFS, this should
indicate HDFS dfs is functioning normally. then,
how about
bin/hadoop jar hadoop-*-examples.jar wordcount myfs/LICENSE.txt out
and the following comman
bin/hadoop jar hadoop-*-examples.jar wordcount ./myfs/LICENSE.txt ./myFS/out
I have already executed ./bin/start-all.sh and I have formatted ./myFS/
Richard Yang ??????:
By the error message, it looks like 'workingdirectory' is out of touch. Is
HDFS dfs portion functioning normally??
Best Regards
Richard Yang
[EMAIL PROTECTED]
[EMAIL PROTECTED]
-----Original Message-----
From: Andreas Papadakis [mailto:[EMAIL PROTECTED]
Sent: Friday, March 23, 2007 1:18 PM
To: [email protected]
Subject: Re: Re: Hadoop for perl
Is there any other way to use hadoop with perl?
I read that I can use it with python(using jython)... I tried the example
but jython cannot find the module org.apache.hadoop.fs...(I dont know much
about java). Anyone knows where can I find that module and where should I
place it??
Also, I tried the java example and I get too many errors....
I used the command
bin/hadoop jar hadoop-*-examples.jar wordcount mydfs/LICENSE.txt mydfs/a
but I get the following errors:
java.lang.RuntimeException: java.net.SocketTimeoutException: timed out
waiting for rpc response
at
org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:247)
at org.apache.hadoop.mapred.JobConf.setInputPath(JobConf.java:150)
at org.apache.hadoop.examples.WordCount.main(WordCount.java:137)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver
.java:71)
at
org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143)
at
org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
response
at org.apache.hadoop.ipc.Client.call(Client.java:469)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164)
at org.apache.hadoop.dfs.$Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:248)
at org.apache.hadoop.dfs.DFSClient.(DFSClient.java:106)
at
org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSystem
.java:65)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:160)
at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:119)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:91)
at
org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:243)
Anyone can help me???
Thank you
Erik Hetzner ??????: At Fri, 23 Mar 2007 20:58:38
+0100,
Maximilian Sch??fmann wrote:
>
> > I want to ask if there is an implementation of Hadoop for perl...or
> > something similar to Hadoop for python
>
> Well, there is one for Ruby - at least the MapReduce part: Starfish
>
> http://rufy.com/starfish/doc/
Starfish isn???t (or wasn???t last time I
looked) really much like
MapReduce at all.
best,
Erik Hetzner
---------------------------------
?????????????? Yahoo!
?????????? ?? ?????????? ???? ???? (spam); ?? Yahoo! Mail ???????? ???
???????? ?????? ????????? ???? ??? ??????????? ?????????
http://login.yahoo.com/config/mail?.intl=gr
---------------------------------
?????????????? Yahoo!
?????????? ?? ?????????? ???? ???? (spam); ?? Yahoo! Mail ???????? ???
???????? ?????? ????????? ???? ??? ??????????? ?????????
http://login.yahoo.com/config/mail?.intl=gr
---------------------------------
?????????????? Yahoo!
?????????? ?? ?????????? ???? ???? (spam); ?? Yahoo! Mail ???????? ???
???????? ?????? ????????? ???? ??? ??????????? ?????????
http://login.yahoo.com/config/mail?.intl=gr
---------------------------------
Χρησιμοποιείτε Yahoo!
Βαρεθήκατε τα ενοχλητικά μηνύ ματα (spam); Το Yahoo! Mail διαθέτει την
καλύτερη δυνατή προστασία κατά των ενοχλητικών μηνυμάτων
http://login.yahoo.com/config/mail?.intl=gr
