Hi , I am trying to integrate Pig with Hadoop for processing of jobs.
I am able to run Pig in local mode and Hadoop with streaming api perfectly. But when I try to run Pig with Hadoop I get follwong Error: Pig Stack Trace --------------- ERROR 2116: Unexpected error. Could not validate the output specification for: file:///Users/rahulmalviya/Documents/Pig/dev/main_merged_hdp_out org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop at org.apache.pig.impl.plan.PlanValidator.validate(PlanValidator.java:56) at org.apache.pig.impl.logicalLayer.validators.InputOutputFileValidator.validate(InputOutputFileValidator.java:49) at org.apache.pig.impl.logicalLayer.validators.InputOutputFileValidator.validate(InputOutputFileValidator.java:37) at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:89) at org.apache.pig.PigServer.validate(PigServer.java:930) at org.apache.pig.PigServer.compileLp(PigServer.java:910) at org.apache.pig.PigServer.compileLp(PigServer.java:871) at org.apache.pig.PigServer.compileLp(PigServer.java:852) at org.apache.pig.PigServer.execute(PigServer.java:816) at org.apache.pig.PigServer.access$100(PigServer.java:105) at org.apache.pig.PigServer$Graph.execute(PigServer.java:1080) at org.apache.pig.PigServer.executeBatch(PigServer.java:288) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:109) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89) at org.apache.pig.Main.main(Main.java:391) Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 2116: Unexpected error. Could not validate the output specification for: file:///Users/rahulmalviya/Documents/Pig/dev/main_merged_hdp_out at org.apache.pig.impl.logicalLayer.validators.InputOutputFileVisitor.visit(InputOutputFileVisitor.java:93) at org.apache.pig.impl.logicalLayer.LOStore.visit(LOStore.java:140) at org.apache.pig.impl.logicalLayer.LOStore.visit(LOStore.java:37) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:67) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69) at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69) at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:50) at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51) at org.apache.pig.impl.plan.PlanValidator.validate(PlanValidator.java:50) ... 16 more Caused by: java.io.IOException: Call to localhost/127.0.0.1:9001 failed on local exception: java.io.EOFException at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) at org.apache.hadoop.ipc.Client.call(Client.java:743) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) at org.apache.hadoop.mapred.$Proxy0.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:429) at org.apache.hadoop.mapred.JobClient.init(JobClient.java:423) at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:410) at org.apache.hadoop.mapreduce.Job.<init>(Job.java:50) at org.apache.pig.impl.logicalLayer.validators.InputOutputFileVisitor.visit(InputOutputFileVisitor.java:89) ... 24 more Caused by: java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:375) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446) ================================================================================ Did anyone got the same error. I think it related to connection between pig and hadoop. Can someone tell me how to connect Pig and hadoop. Thanks.