[ https://issues.apache.org/jira/browse/CASSANDRA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-1590: ---------------------------------------- Fix Version/s: 0.6.9 Not quite. > Pig loadfunc fails with java.io.FileNotFoundException: > ...:.../job.jar!/storage-conf.xml > ---------------------------------------------------------------------------------------- > > Key: CASSANDRA-1590 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1590 > Project: Cassandra > Issue Type: Bug > Components: Contrib, Hadoop > Affects Versions: 0.6.4 > Environment: Ubuntu 10.04 with Hadoop from Cloudera CDH3b2 > Reporter: Tommi Virtanen > Fix For: 0.6.9 > > Attachments: casspig.tgz, p1590.diff > > > Trying to run the example job from contrib/pig (after fixing it to start at > all in the first place; details later) results in this: > 2010-10-06 15:43:32,117 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: > Store(hdfs://localhost/tmp/temp-1257182404/tmp1075428643:org.apache.pig.builtin.BinStorage) > - 1-60 Operator Key: 1-60) > 2010-10-06 15:43:32,164 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer > - Choosing to move algebraic foreach to combiner > 2010-10-06 15:43:32,224 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size before optimization: 3 > 2010-10-06 15:43:32,224 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size after optimization: 3 > 2010-10-06 15:43:32,302 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 > 2010-10-06 15:43:40,356 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - Setting up single store job > 2010-10-06 15:43:40,450 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 1 map-reduce job(s) waiting for submission. > 2010-10-06 15:43:40,457 [Thread-12] WARN org.apache.hadoop.mapred.JobClient > - Use GenericOptionsParser for parsing the arguments. Applications should > implement Tool for the same. > 2010-10-06 15:43:40,950 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 0% complete > 2010-10-06 15:43:41,038 [Thread-12] INFO > org.apache.cassandra.config.DatabaseDescriptor - DiskAccessMode 'auto' > determined to be mmap, indexAccessMode is mmap > 2010-10-06 15:43:41,211 [Thread-12] WARN > org.apache.cassandra.config.DatabaseDescriptor - KeysCachedFraction is > deprecated: use KeysCached instead. > 2010-10-06 15:43:41,232 [Thread-12] WARN > org.apache.cassandra.config.DatabaseDescriptor - KeysCachedFraction is > deprecated: use KeysCached instead. > 2010-10-06 15:43:42,305 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - HadoopJobId: job_201010061447_0008 > 2010-10-06 15:43:42,305 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - More information at: > http://localhost:50030/jobdetails.jsp?jobid=job_201010061447_0008 > 2010-10-06 15:44:15,025 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 33% complete > 2010-10-06 15:44:17,037 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 100% complete > 2010-10-06 15:44:17,037 [main] ERROR > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 1 map reduce job(s) failed! > 2010-10-06 15:44:17,067 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Failed! > 2010-10-06 15:44:17,199 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR > 2997: Unable to recreate exception from backed error: Error: > java.lang.RuntimeException: java.io.FileNotFoundException: > /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml > (No such file or directory) > Details at logfile: > /home/tv/casspig/cassandra/contrib/pig/pig_1286405010154.log > Contents of that pig_*.log: > Backend error message > --------------------- > Error: java.lang.RuntimeException: java.io.FileNotFoundException: > /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml > (No such file or directory) > at > org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:542) > at > org.apache.cassandra.hadoop.ConfigHelper.getThriftPort(ConfigHelper.java:188) > at > org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.<init>(ColumnFamilyRecordReader.java:118) > at > org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.<init>(ColumnFamilyRecordReader.java:104) > at > org.apache.cassandra.hadoop.ColumnFamilyRecordReader.initialize(ColumnFamilyRecordReader.java:93) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initialize(PigRecordReader.java:133) > at > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:418) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:620) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) > at org.apache.hadoop.mapred.Child.main(Child.java:170) > Caused by: java.io.FileNotFoundException: > /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml > (No such file or directory) > at java.io.FileInputStream.open(Native Method) > at java.io.FileInputStream.<init>(FileInputStream.java:106) > at java.io.FileInputStream.<init>(FileInputStream.java:66) > at > sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70) > at > sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161) > at > com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653) > at > com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186) > at > com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772) > at > com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737) > at > com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119) > at > com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235) > at > com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284) > at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:208) > at org.apache.cassandra.utils.XMLUtils.<init>(XMLUtils.java:43) > at > org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:167) > ... 9 more > Pig Stack Trace > --------------- > ERROR 2997: Unable to recreate exception from backed error: Error: > java.lang.RuntimeException: java.io.FileNotFoundException: > /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml > (No such file or directory) > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to > open iterator for alias topnames > at org.apache.pig.PigServer.openIterator(PigServer.java:607) > at > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:544) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138) > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75) > at org.apache.pig.Main.main(Main.java:380) > Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2997: > Unable to recreate exception from backed error: Error: > java.lang.RuntimeException: java.io.FileNotFoundException: > /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml > (No such file or directory) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:231) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:175) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:270) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:308) > at > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1007) > at org.apache.pig.PigServer.store(PigServer.java:697) > at org.apache.pig.PigServer.openIterator(PigServer.java:590) > ... 6 more > ================================================================================ > I'm attaching a tarball with everything needed to reproduce this, see the run > script there. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.