Hadoop 0.21.0 streaming giving no status information
Hi, I'm trying to upgrade an application previously written for Hadoop 0.20.0 for 0.21.0. I'm running into an issue with the status output missing which is making it difficult to get the jobid/success status: hadoop/bin/hadoop jar hadoop/mapred/contrib/streaming/hadoop-0.21.0-streaming.jar -D mapreduce.job.reduces=0 -input file:///dev/null -mapper ./cmd.sh -file ./cmd.sh -output '/users/foo/tmp/job-1234' -verbose > OUTPUT 2>&1 This gives me a bunch of settings output such as: STREAM: net.topology.script.number.args=100 STREAM: s3.blocksize=67108864 STREAM: s3.bytes-per-checksum=512 STREAM: s3.client-write-packet-size=65536 STREAM: s3.replication=3 STREAM: s3.stream-buffer-size=4096 finally ending with: STREAM: webinterface.private.actions=false STREAM: STREAM: submitting to jobconf:machine.hostname.domain:8023 After that, I get no further status information. The job does complete successfully. I would expect to get this type status information: 11/04/23 01:03:24 INFO streaming.StreamJob: getLocalDirs(): [/home/hadoop/hadoop/tmp/dir/hadoop-hadoop/mapred/local] 11/04/23 01:03:24 INFO streaming.StreamJob: Running job: job_201104222325_0021 11/04/23 01:03:24 INFO streaming.StreamJob: To kill this job, run: 11/04/23 01:03:24 INFO streaming.StreamJob: /home/hadoop/hadoop/bin/../bin/hadoop job -Dmapred.job.tracker=localhost:54311 -kill job_201104222325_0021 11/04/23 01:03:24 INFO streaming.StreamJob: Tracking URL: http://localhost:50030/jobdetails.jsp?jobid=job_201104222325_0021 11/04/23 01:03:25 INFO streaming.StreamJob: map 0% reduce 0% 11/04/23 01:03:31 INFO streaming.StreamJob: map 50% reduce 0% 11/04/23 01:03:41 INFO streaming.StreamJob: map 50% reduce 17% 11/04/23 01:03:56 INFO streaming.StreamJob: map 100% reduce 100% I've tried playing with various switches including: -Dhadoop.root.logger=INFO,console -Dhadoop.log.file=hadoop.log -Dhadoop.log.dir=$PWD but none of these make a difference. Any help would be greatly appreciated! -- - Patrick Donnelly
Caching in HDFS C API Client
Hi List, I need to explain an higher than expected throughput (bandwidth) for a HDFS C API Client. Specifically, the client is getting bandwidth higher than its link rate :). The client is first writing a 512 MB file followed by reading the entire file back. The file read is what's getting the higher than link rate bandwidth. I assume this is a consequence of caching? Is this done by HDFS or by Linux? Thanks for any help, -- - Patrick Donnelly
Hadoop, C API, and fork
Hi, I have a distributed file server front end to Hadoop that uses the libhdfs C API to talk to Hadoop. Normally the file server will fork on a new client connection but this does not work with the libhdfs shared library (it is loaded using dlopen). If the server is in single process mode (no forking and can handle only one client at a time) then everything works fine. I have tried changing it so the server disconnects the Hadoop connection before forking and having both processes re-connect post fork. Essentially in the server: hdfsDisconnect(...); pid = fork(); hdfsConnect(...); if (pid == 0) ... else ... This causes a hang in the child process on Connect with the following backtrace: (gdb) bt #0 0x0034d160ad09 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x2ace492559f7 in os::PlatformEvent::park () from /afs/nd.edu/user37/ccl/software/external/java/jdk/jre/lib/amd64/server/libjvm.so #2 0x2ace4930a5da in ObjectMonitor::wait () from /afs/nd.edu/user37/ccl/software/external/java/jdk/jre/lib/amd64/server/libjvm.so #3 0x2ace49307b13 in ObjectSynchronizer::wait () from /afs/nd.edu/user37/ccl/software/external/java/jdk/jre/lib/amd64/server/libjvm.so #4 0x2ace490cf5fb in JVM_MonitorWait () from /afs/nd.edu/user37/ccl/software/external/java/jdk/jre/lib/amd64/server/libjvm.so #5 0x2ace49c87f50 in ?? () #6 0x0001 in ?? () #7 0x2ace4cd84d10 in ?? () #8 0x3f80 in ?? () #9 0x2ace49c8841d in ?? () #10 0x7fff0b4d04c0 in ?? () #11 0x in ?? () Leaving the connection open in the server: pid = fork(); if (pid == 0) ... else ... Also produces a hang in the child: (gdb) bt #0 0x0034d160ad09 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x2b3d7193d9f7 in os::PlatformEvent::park () from /afs/nd.edu/user37/ccl/software/external/java/jdk/jre/lib/amd64/server/libjvm.so #2 0x2b3d719f25da in ObjectMonitor::wait () from /afs/nd.edu/user37/ccl/software/external/java/jdk/jre/lib/amd64/server/libjvm.so #3 0x2b3d719efb13 in ObjectSynchronizer::wait () from /afs/nd.edu/user37/ccl/software/external/java/jdk/jre/lib/amd64/server/libjvm.so #4 0x2b3d717b75fb in JVM_MonitorWait () from /afs/nd.edu/user37/ccl/software/external/java/jdk/jre/lib/amd64/server/libjvm.so #5 0x2b3d7236ff50 in ?? () #6 0x in ?? () Does anyone have a suggestion on debugging/fixing this? Thanks for any help, -- - Patrick Donnelly
