Hi,
everything works fine with cassandra 0.7.5, but when I tried with 0.7.3
another errors showed up, but task finished with success whats strange.....
2011-04-20 11:45:40,674 INFO org.apache.hadoop.mapred.TaskInProgress: Error
from attempt_201104201139_0004_m_000000_3: Error: java.lang.ClassNotF
oundException: org.apache.thrift.TException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:426)
at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:456)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getLoadFunc(PigInputFormat.java:153)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:105)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:588)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
2011-04-20 11:45:43,629 INFO org.apache.hadoop.mapred.TaskInProgress: Error
from attempt_201104201139_0004_m_000001_3: org.apache.pig.backend.exe
cutionengine.ExecException: ERROR 2044: The type null cannot be collected as
a Key type
at
org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:143)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:105)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:238)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
2011-04-20 11:42:49,498 INFO org.apache.hadoop.mapred.TaskInProgress: Error
from attempt_201104201139_0001_m_000000_1: Error: java.lang.ClassNotF
oundException: org.apache.commons.lang.ArrayUtils
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at
org.apache.cassandra.utils.ByteBufferUtil.<clinit>(ByteBufferUtil.java:75)
at org.apache.cassandra.hadoop.pig.CassandraStorage.<clinit>(Unknown
Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:426)
at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:456)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getLoadFunc(PigInputFormat.java:153)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:105)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:588)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
2011/4/20 Jeremy Hanna <jeremy.hanna1...@gmail.com>
> Just as an example:
>
> <property>
> <name>cassandra.thrift.address</name>
> <value>10.12.34.56</value>
> </property>
> <property>
> <name>cassandra.thrift.port</name>
> <value>9160</value>
> </property>
> <property>
> <name>cassandra.partitioner.class</name>
> <value>org.apache.cassandra.dht.RandomPartitioner</value>
> </property>
>
>
> On Apr 19, 2011, at 10:28 PM, Jeremy Hanna wrote:
>
> > oh yeah - that's what's going on. what I do is on the machine that I run
> the pig script from, I set the PIG_CONF variable to my HADOOP_HOME/conf
> directory and in my mapred-site.xml file found there, I set the three
> variables.
> >
> > I don't use environment variables when I run against a cluster.
> >
> > On Apr 19, 2011, at 9:54 PM, Jeffrey Wang wrote:
> >
> >> Did you set PIG_RPC_PORT in your hadoop-env.sh? I was seeing this error
> for a while before I added that.
> >>
> >> -Jeffrey
> >>
> >> From: pob [mailto:peterob...@gmail.com]
> >> Sent: Tuesday, April 19, 2011 6:42 PM
> >> To: user@cassandra.apache.org
> >> Subject: Re: pig + hadoop
> >>
> >> Hey Aaron,
> >>
> >> I read it, and all of 3 env variables was exported. The results are
> same.
> >>
> >> Best,
> >> P
> >>
> >> 2011/4/20 aaron morton <aa...@thelastpickle.com>
> >> Am guessing but here goes. Looks like the cassandra RPC port is not set,
> did you follow these steps in contrib/pig/README.txt
> >>
> >> Finally, set the following as environment variables (uppercase,
> >> underscored), or as Hadoop configuration variables (lowercase, dotted):
> >> * PIG_RPC_PORT or cassandra.thrift.port : the port thrift is listening
> on
> >> * PIG_INITIAL_ADDRESS or cassandra.thrift.address : initial address to
> connect to
> >> * PIG_PARTITIONER or cassandra.partitioner.class : cluster partitioner
> >>
> >> Hope that helps.
> >> Aaron
> >>
> >>
> >> On 20 Apr 2011, at 11:28, pob wrote:
> >>
> >>
> >> Hello,
> >>
> >> I did cluster configuration by
> http://wiki.apache.org/cassandra/HadoopSupport. When I run pig
> example-script.pig
> >> -x local, everything is fine and i get correct results.
> >>
> >> Problem is occurring with -x mapreduce
> >>
> >> Im getting those errors :>
> >>
> >>
> >> 2011-04-20 01:24:21,791 [main] ERROR
> org.apache.pig.tools.pigstats.PigStats - ERROR:
> java.lang.NumberFormatException: null
> >> 2011-04-20 01:24:21,792 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> >> 2011-04-20 01:24:21,793 [main] INFO
> org.apache.pig.tools.pigstats.PigStats - Script Statistics:
> >>
> >> Input(s):
> >> Failed to read data from "cassandra://Keyspace1/Standard1"
> >>
> >> Output(s):
> >> Failed to produce result in
> "hdfs://ip:54310/tmp/temp-1383865669/tmp-1895601791"
> >>
> >> Counters:
> >> Total records written : 0
> >> Total bytes written : 0
> >> Spillable Memory Manager spill count : 0
> >> Total bags proactively spilled: 0
> >> Total records proactively spilled: 0
> >>
> >> Job DAG:
> >> job_201104200056_0005 -> null,
> >> null -> null,
> >> null
> >>
> >>
> >> 2011-04-20 01:24:21,793 [main] INFO
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Failed!
> >> 2011-04-20 01:24:21,803 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1066: Unable to open iterator for alias topnames. Backend error :
> java.lang.NumberFormatException: null
> >>
> >>
> >>
> >> ====
> >> thats from jobtasks web management - error from task directly:
> >>
> >> java.lang.RuntimeException: java.lang.NumberFormatException: null
> >> at
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader.initialize(ColumnFamilyRecordReader.java:123)
> >> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initialize(PigRecordReader.java:176)
> >> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:418)
> >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:620)
> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> >> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> >> Caused by: java.lang.NumberFormatException: null
> >> at java.lang.Integer.parseInt(Integer.java:417)
> >> at java.lang.Integer.parseInt(Integer.java:499)
> >> at
> org.apache.cassandra.hadoop.ConfigHelper.getRpcPort(ConfigHelper.java:233)
> >> at
> org.apache.cassandra.hadoop.ColumnFamilyRecordReader.initialize(ColumnFamilyRecordReader.java:105)
> >> ... 5 more
> >>
> >>
> >>
> >> Any suggestions where should be problem?
> >>
> >> Thanks,
> >>
> >>
> >>
> >
>
>
