Ignore the above error, I somehow passed that stage. However, I am still having problem with it.
grunt> register /home/felix/pig-0.7.0/pig-0.7.1-dev.jar; register /home/felix/cassandra/lib/libthrift.jar; grunt> rows = LOAD 'cassandra://test/data' USING CassandraStorage(); grunt> cols = FOREACH rows GENERATE flatten($1); grunt> colnames = FOREACH cols GENERATE $0; grunt> limit_colnames = limit colnames 10; grunt> dump limit_colnames 2011-01-05 15:44:17,378 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId= 2011-01-05 15:44:17,460 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: Store(file:/tmp/temp-1545399343/tmp576746049:org.apache.pig.builtin.BinStorage) - 1-27 Operator Key: 1-27) 2011-01-05 15:44:17,507 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1 2011-01-05 15:44:17,507 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1 2011-01-05 15:44:17,533 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 2011-01-05 15:44:17,539 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 2011-01-05 15:44:17,539 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 2011-01-05 15:44:21,785 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job 2011-01-05 15:44:21,841 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 2011-01-05 15:44:21,842 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission. 2011-01-05 15:44:21,846 [Thread-5] WARN org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 2011-01-05 15:44:22,115 [Thread-5] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 2011-01-05 15:44:22,133 [Thread-5] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 2011-01-05 15:44:22,344 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete 2011-01-05 15:44:22,348 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2117: Unexpected error when launching map reduce job. Details at logfile: /home/felix/cassandra/contrib/pig/pig_1294263823129.log cat pig_1294263823129.log Pig Stack Trace --------------- ERROR 2117: Unexpected error when launching map reduce job. org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias limit_colnames at org.apache.pig.PigServer.openIterator(PigServer.java:521) at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:544) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75) at org.apache.pig.Main.main(Main.java:357) Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable to store alias limit_colnames at org.apache.pig.PigServer.store(PigServer.java:577) at org.apache.pig.PigServer.openIterator(PigServer.java:504) ... 6 more Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2117: Unexpected error when launching map reduce job. at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:209) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:308) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:835) at org.apache.pig.PigServer.store(PigServer.java:569) ... 7 more Caused by: java.lang.RuntimeException: Could not resolve error that occured when launching map reduce job: java.lang.ExceptionInInitializerError at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$JobControlThreadExceptionHandler.uncaughtException(MapReduceLauncher.java:510) at java.lang.Thread.dispatchUncaughtException(Thread.java:1831) ================================================================================ On Wed, Jan 5, 2011 at 12:02 PM, felix gao <gre1...@gmail.com> wrote: > I am having problem running the cassandra_loadfunc.jar on my build of > cassandra. > > PIG_CLASSPATH=:bin/../build/cassandra_loadfunc.jar::bin/../../..//lib/antlr-3.1.3.jar:bin/../../..//lib/avro-1.2.0-dev.jar:bin/../../..//lib/clhm-production.jar:bin/../../..//lib/commons-cli-1.1.jar:bin/../../..//lib/commons-codec-1.2.jar:bin/../../..//lib/commons-collections-3.2.1.jar:bin/../../..//lib/commons-lang-2.4.jar:bin/../../..//lib/google-collections-1.0.jar:bin/../../..//lib/hadoop-core-0.20.1.jar:bin/../../..//lib/high-scale-lib.jar:bin/../../..//lib/jackson-core-asl-1.4.0.jar:bin/../../..//lib/jackson-mapper-asl-1.4.0.jar:bin/../../..//lib/jline-0.9.94.jar:bin/../../..//lib/json-simple-1.1.jar:bin/../../..//lib/libthrift.jar:bin/../../..//lib/log4j-1.2.14.jar:bin/../../..//lib/slf4j-api-1.5.8.jar:bin/../../..//lib/slf4j-log4j12-1.5.8.jar:bin/../../..//lib/spymemcached-2.4.2.jar:bin/../../..//lib/zapcat-1.2.jar:bin/../../..//build/lib/jars/ant-1.6.5.jar:bin/../../..//build/lib/jars/apache-rat-0.6.jar:bin/../../..//build/lib/jars/apache-rat-core-0.6.jar:bin/../../..//build/lib/jars/apache-rat-tasks-0.6.jar:bin/../../..//build/lib/jars/asm-3.2.jar:bin/../../..//build/lib/jars/avalon-framework-4.1.3.jar:bin/../../..//build/lib/jars/commons-cli-1.1.jar:bin/../../..//build/lib/jars/commons-collections-3.2.jar:bin/../../..//build/lib/jars/commons-lang-2.1.jar:bin/../../..//build/lib/jars/commons-logging-1.1.1.jar:bin/../../..//build/lib/jars/junit-4.6.jar:bin/../../..//build/lib/jars/log4j-1.2.12.jar:bin/../../..//build/lib/jars/logkit-1.0.1.jar:bin/../../..//build/lib/jars/paranamer-ant-2.1.jar:bin/../../..//build/lib/jars/paranamer-generator-2.1.jar:bin/../../..//build/lib/jars/qdox-1.10.jar:bin/../../..//build/lib/jars/servlet-api-2.3.jar:bin/../../..//build/apache-cassandra-0.6.4.jar:bin/../../..//build/ivy-2.1.0.jar:/usr/local/pig-0.7.0/pig.jar > > In Grunt I did register again just in case it is not picked up by the > classpath > register /usr/local/pig-0.7.0/pig.jar; register > /home/felix/cassandra/lib/libthrift.jar; register > /home/felix/cassandra/contrib/pig/build/cassandra_loadfunc.jar > grunt> rows = LOAD 'cassandra://test.data' USING CassandraStorge(); > > 2011-01-05 13:50:50,071 [main] ERROR > org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve > CassandraStorge using imports: [org.apache.cassandra.hadoop.pig., , > org.apache.pig.builtin., org.apache.pig.impl.builtin.] > Details at logfile: /home/felix/cassandra/contrib/pig/pig_1294257032719.log > > > the log file contains > > Pig Stack Trace > --------------- > ERROR 1070: Could not resolve CassandraStorge using imports: > [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin., > org.apache.pig.impl.builtin.] > > java.lang.RuntimeException: Cannot instantiate:CassandraStorge > at > org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:455) > at > org.apache.pig.impl.logicalLayer.parser.QueryParser.NonEvalFuncSpec(QueryParser.java:5087) > at > org.apache.pig.impl.logicalLayer.parser.QueryParser.LoadClause(QueryParser.java:1434) > at > org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1245) > at > org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:911) > at > org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:700) > at > org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63) > at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1164) > at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1114) > at org.apache.pig.PigServer.registerQuery(PigServer.java:425) > at > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:737) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:324) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138) > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75) > at org.apache.pig.Main.main(Main.java:357) > Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR > 1070: Could not resolve CassandraStorge using imports: > [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin., > org.apache.pig.impl.builtin.] > at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:440) > at > org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:452) > ... 15 more > > Running hadoop 0.20.2 with pig0.7.0 and have to use cassandra 0.6.4. > > Thanks, > > Felix > >