I'm guessing this is http://issues.apache.org/jira/browse/HADOOP-3139
which is fixed in Hadoop 0.16.3 (currently being voted on for release
by the developer community).
Nige
On Apr 14, 2008, at 3:07 AM, Andreas Kostyrka wrote:
As another item, the submitting Java process hangs in a futex call:
[EMAIL PROTECTED]:~# strace -p 3810
Process 3810 attached - interrupt to quit
futex(0xb7d6ebd8, FUTEX_WAIT, 3832, NULL
and hangs, hangs, hangs, ...
Andreas
Am Montag, den 14.04.2008, 11:46 +0200 schrieb Andreas Kostyrka:
Ok, a short grep in the sources suggests that the exceptions
happen just
in the closeAll method of FileSystem. So no indication what hadoop is
working on :(
Am Montag, den 14.04.2008, 07:26 +0200 schrieb Andreas Kostyrka:
Hi!
I'm getting the following hang, when trying to run a streaming
command:
[EMAIL PROTECTED]:~/hadoop-0.16.2$ time bin/hadoop jar
contrib/streaming/hadoop-0.16.2-streaming.jar -mapper '/home/
hadoop/bin/llfp -f [EMAIL PROTECTED] -t [EMAIL PROTECTED] -s
heaven.kostyrka.org -d gen_dailysites -d fb_memberfind ' -reducer
NONE -input run-0003-input/* -output run-0003-output
additionalConfSpec_:null
null=@@@userJobConfProps_.get(stream.shipped.hadoopstreaming
packageJobJar: [/tmp/hadoop-hadoop/hadoop-unjar60050/] [] /tmp/
streamjob60051.jar tmpDir=null
08/04/14 05:22:10 INFO fs.FileSystem: FileSystem.closeAll() threw
an exception:
org.apache.hadoop.io.MultipleIOException: 2 exceptions
[java.io.IOException: S3FileSystem
([EMAIL PROTECTED]) and Key
([EMAIL PROTECTED]://lokgad) do not match., java.io.IOException:
LocalFileSystem([EMAIL PROTECTED]) and
Key([EMAIL PROTECTED]://null) do not match.]
The exception happens only when I press Control-C after many
minutes.
The cluster uses an HDFS/S3 filesystem, and concurrently to this I'm
issueing many many "relative small" hadoop fs -put requests,
copying in
data to a different folder.
The above hang happened multiple times, so I wonder what could it
be?
Andreas