Looks like I was pointing to incorrect ports. After correcting the port
numbers,
conf.set("fs.defaultFS", "hdfs://<server_address>:8020");
conf.set("mapred.job.tracker", "<server_address>:8021");
I am now getting the following exception:
2880 [Thread-15] INFO
org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob -
java.lang.IllegalArgumentException: java.net.UnknownHostException: bdatadev
at
org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:414)
at
org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:164)
at
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:129)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:389)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:356)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:124)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2218)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2252)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2234)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194)
at
org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:103)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:902)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:896)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:896)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:531)
at
org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob.submit(CrunchControlledJob.java:305)
at
org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.startReadyJobs(CrunchJobControl.java:180)
at
org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.pollJobStatusAndStartNewOnes(CrunchJobControl.java:209)
at
org.apache.crunch.impl.mr.exec.MRExecutor.monitorLoop(MRExecutor.java:100)
at org.apache.crunch.impl.mr.exec.MRExecutor.access$000(MRExecutor.java:51)
at org.apache.crunch.impl.mr.exec.MRExecutor$1.run(MRExecutor.java:75)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.net.UnknownHostException: bdatadev
... 27 more
However nowhere in my code a host named "bdatadev" is mentioned, and I
cannot ping this host.
Thanks for the help.
On Fri, Aug 30, 2013 at 3:04 PM, Narlin M <hpn...@gmail.com> wrote:
> I am getting following exception while trying to submit a crunch pipeline
> job to a remote hadoop cluster:
>
> Exception in thread "main" java.lang.RuntimeException: Cannot create job
> output directory /tmp/crunch-324987940
> at
> org.apache.crunch.impl.mr.MRPipeline.createTempDirectory(MRPipeline.java:344)
> at org.apache.crunch.impl.mr.MRPipeline.<init>(MRPipeline.java:125)
> at test.CrunchTest.setup(CrunchTest.java:98)
> at test.CrunchTest.main(CrunchTest.java:367)
> Caused by: java.io.IOException: Failed on local exception:
> com.google.protobuf.InvalidProtocolBufferException: Protocol message
> end-group tag did not match expected tag.; Host Details : local host is:
> "NARLIN/127.0.0.1"; destination host is: "<server_address>":50070;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:759)
> at org.apache.hadoop.ipc.Client.call(Client.java:1164)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
> at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
> at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:425)
> at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1943)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:523)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1799)
> at
> org.apache.crunch.impl.mr.MRPipeline.createTempDirectory(MRPipeline.java:342)
> ... 3 more
> Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol
> message end-group tag did not match expected tag.
> at
> com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:73)
> at
> com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java:124)
> at
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:213)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:746)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:238)
> at
> com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:282)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:760)
> at
> com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:288)
> at
> com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:752)
> at
> org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcPayloadHeaderProtos.java:985)
> at
> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:882)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:813)
> 0 [Thread-3] WARN org.apache.hadoop.util.ShutdownHookManager -
> ShutdownHook 'ClientFinalizer' failed, java.lang.NoSuchMethodError:
> com.google.common.collect.LinkedListMultimap.values()Ljava/util/List;
> java.lang.NoSuchMethodError:
> com.google.common.collect.LinkedListMultimap.values()Ljava/util/List;
> at org.apache.hadoop.hdfs.SocketCache.clear(SocketCache.java:135)
> at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:672)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:539)
> at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2308)
> at
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2324)
> at
> org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
>
> Google search on this error yielded solutions that asked to confirm that
> /etc/hosts file contained the entry for NARLIN which it does in my case.
>
> Here's the code that I am using to set up the MRPipeline:
>
> Configuration conf = HBaseConfiguration.create();
>
> conf.set("fs.defaultFS", "hdfs://<server_address>:50070");
> conf.set("mapred.job.tracker", "<server_address>:50030");
>
> System.out.println("Hadoop configuration created.");
> System.out.println("Initializing crunch pipeline ...");
>
> conf.set("mapred.jar", "<path_to_jar_file>");
>
> pipeline = new MRPipeline(getClass(), "crunchjobtest", conf);
>
> Has anyone faced this issue before and knows how to resolve it/point out
> if I am missing anything?
>
> Thanks for the help.
>
