Hello all sometimes my pig job fails to run successfully (stack trace below) In fact the last 2 days it has been the same stack trace Googling for it gave this http://lucene.472066.n3.nabble.com/what-does-it-mean-java-io-IOException-Filesystem-closed-td1829780.html which talks about OS open file limit Has anybody else encountered similar issues ?
Any advise/suggestions that you have of what could be causing this and what I should do? 2042: Internal error. Unable to translate logical plan to physical plan ERROR 6002: Unable to obtain a temporary path. Caused by: java.io.IOException: Filesystem closed Issue is pressing, so any help is appreciated thanks and regards Anindita > Full Exception Stacktrace: > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to > store alias pop_short_csv > at org.apache.pig.PigServer.store(PigServer.java:577) > at org.apache.pig.PigServer.store(PigServer.java:533) > at > com.nokia.productscores.ScoresTrendsPigJob.run(ScoresTrendsPigJob.java:316) > at > com.nokia.productscores.ScoresTrendsPigJob.runJob(ScoresTrendsPigJob.java:137) > at > com.nokia.productscores.ProductScoresImpl.compute(ProductScoresImpl.java:113) > at > com.nokia.relevancy.scheduler.quartz.ComputerProductScoresJob.execute(ComputerProductScoresJob.java:32) > at org.quartz.core.JobRunShell.run(JobRunShell.java:199) > at > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.ja > va:549) Caused by: > org.apache.pig.backend.executionengine.ExecException: ERROR 2042: Internal > error. Unable to translate logical plan to physical plan. > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:289) > at org.apache.pig.PigServer.compilePp(PigServer.java:917) > at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:832) > at org.apache.pig.PigServer.store(PigServer.java:569) > ... 7 more > Caused by: > org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogicalToPhysicalTranslatorException: > ERROR 6002: Unable to obtain a temporary path. > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:1464) > at org.apache.pig.impl.logicalLayer.LOSplit.visit(LOSplit.java:118) > at org.apache.pig.impl.logicalLayer.LOSplit.visit(LOSplit.java:40) > at > org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69) > at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:283) > ... 10 more > Caused by: java.io.IOException: Filesystem closed > at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:226) > at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:617) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:453) > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:648) > at org.apache.pig.backend.hadoop.datastorage.HPath.exists(HPath.java:106) > at > org.apache.pig.impl.io.FileLocalizer.getTemporaryPath(FileLocalizer.java:465) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:1448) > ... 15 more
