See http://people.apache.org/~edwardyoon/MyTest.txt
On Tue, Feb 25, 2014 at 8:25 PM, <[email protected]> wrote: > > I really can't find a solution, can you please send me a simple custom > message type of yours that is working so that I can clear some bug > possibilities out? > > Thanks. > > > > Quoting [email protected]: > >> Ok I will check again. I just found it strange to throw that kind of error >> without changing anything. >> >> >> >> Quoting "Edward J. Yoon" <[email protected]>: >> >>> Please see >>> http://svn.apache.org/repos/asf/hama/trunk/examples/src/main/java/org/apache/hama/examples/PageRank.java >>> and check your program whether it is implemented correctly or not. >>> >>> I guess, it's not a problem of aggregator. It's a bug in your program. >>> >>> On Mon, Feb 24, 2014 at 6:50 PM, <[email protected]> wrote: >>>> >>>> This is the code. What changed and suddenly it throws the error? >>>> >>>> >>>> >>>> package corever2; >>>> import java.io.DataInput; >>>> import java.io.DataOutput; >>>> import java.io.IOException; >>>> import org.apache.hadoop.io.DoubleWritable; >>>> import org.apache.hadoop.io.IntWritable; >>>> import org.apache.hadoop.io.Text; >>>> import org.apache.hadoop.io.Writable; >>>> >>>> public class MessageCore implements Writable { >>>> >>>> private DoubleWritable type; >>>> private Text nodeID; >>>> private IntWritable degree; >>>> private IntWritable maxDegree; >>>> private IntWritable nodeCounter; >>>> >>>> public MessageCore() { >>>> type = new DoubleWritable(); >>>> nodeID = new Text(); >>>> degree = new IntWritable(); >>>> maxDegree = new IntWritable(); >>>> nodeCounter = new IntWritable(); >>>> } >>>> >>>> public MessageCore(DoubleWritable type, Text nodeID, IntWritable >>>> degree, >>>> IntWritable maxDegree, IntWritable nodeCounter) { >>>> this.type = type; >>>> this.nodeID = nodeID; >>>> this.degree = degree; >>>> this.maxDegree = maxDegree; >>>> this.nodeCounter = nodeCounter; >>>> } >>>> >>>> public MessageCore(DoubleWritable type, Text nodeID) { >>>> this.type = type; >>>> this.nodeID = nodeID; >>>> degree = new IntWritable(); >>>> maxDegree = new IntWritable(); >>>> nodeCounter = new IntWritable(); >>>> } >>>> >>>> public void setNodeCounter(IntWritable nodeCounter) { >>>> this.nodeCounter = nodeCounter; >>>> } >>>> >>>> public void setMaxDegree(IntWritable maxDegree) { >>>> this.maxDegree = maxDegree; >>>> } >>>> >>>> public void setType(DoubleWritable type) { >>>> this.type = type; >>>> } >>>> >>>> public void setNodeID(Text nodeID) { >>>> this.nodeID = nodeID; >>>> } >>>> >>>> public DoubleWritable getType() { >>>> return type; >>>> } >>>> >>>> public Text getNodeID() { >>>> return nodeID; >>>> } >>>> >>>> public IntWritable getDegree() { >>>> return degree; >>>> } >>>> >>>> public IntWritable getMaxDegree() { >>>> return maxDegree; >>>> } >>>> >>>> public IntWritable getNodeCounter() { >>>> return nodeCounter; >>>> } >>>> >>>> public void setDegree(IntWritable degree) { >>>> this.degree = degree; >>>> } >>>> >>>> @Override >>>> public String toString() { >>>> return nodeID + "\t" + degree; >>>> } >>>> >>>> @Override >>>> public void write(DataOutput d) throws IOException { >>>> type.write(d); >>>> nodeID.write(d); >>>> degree.write(d); >>>> maxDegree.write(d); >>>> nodeCounter.write(d); >>>> } >>>> >>>> @Override >>>> public void readFields(DataInput di) throws IOException { >>>> type.readFields(di); >>>> nodeID.readFields(di); >>>> degree.readFields(di); >>>> maxDegree.readFields(di); >>>> nodeCounter.readFields(di); >>>> >>>> } >>>> } >>>> >>>> >>>> >>>> Quoting "Edward J. Yoon" <[email protected]>: >>>> >>>>> Please attach your MessageCore. >>>>> >>>>> On Sun, Feb 23, 2014 at 11:37 PM, <[email protected]> wrote: >>>>>> >>>>>> >>>>>> Hello there, >>>>>> >>>>>> i downloaded hama-trunk a few days ago and till now I am trying to >>>>>> resolve >>>>>> the following problem that came up: >>>>>> >>>>>> My program runs without errors without aggregators. When I set the >>>>>> aggregator class to be used, I have the following error (which is like >>>>>> "cant >>>>>> find the message class" when the java file exists in the package!): >>>>>> >>>>>> corever2 is the package and MessageCore is the name of the class. >>>>>> >>>>>> java.io.IOException: can't find class: corever2.MessageCore because >>>>>> corever2.MessageCore >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hadoop.io.AbstractMapWritable.readFields(AbstractMapWritable.java:204) >>>>>> at >>>>>> org.apache.hadoop.io.MapWritable.readFields(MapWritable.java:146) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:129) >>>>>> at >>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:108) >>>>>> at >>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:78) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:356) >>>>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381) >>>>>> at >>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133) >>>>>> at >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212) >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>>>>> at >>>>>> >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>>>>> at >>>>>> >>>>>> >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) >>>>>> at >>>>>> >>>>>> >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) >>>>>> at java.lang.Thread.run(Thread.java:662) >>>>>> java.io.EOFException >>>>>> at java.io.DataInputStream.readFully(DataInputStream.java:180) >>>>>> at org.apache.hadoop.io.Text.readFields(Text.java:265) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:136) >>>>>> at >>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:108) >>>>>> at >>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:78) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:356) >>>>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381) >>>>>> at >>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133) >>>>>> at >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212) >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>>>>> at >>>>>> >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>>>>> at >>>>>> >>>>>> >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) >>>>>> at >>>>>> >>>>>> >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) >>>>>> at java.lang.Thread.run(Thread.java:662) >>>>>> 14/02/23 18:30:23 ERROR bsp.LocalBSPRunner: Exception during BSP >>>>>> execution! >>>>>> java.lang.UnsupportedOperationException: Unknown message type: >>>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null] >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.graph.GraphJobRunner.parseMessages(GraphJobRunner.java:590) >>>>>> at >>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:136) >>>>>> at >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212) >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>>>>> at >>>>>> >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>>>>> at >>>>>> >>>>>> >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) >>>>>> at >>>>>> >>>>>> >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) >>>>>> at java.lang.Thread.run(Thread.java:662) >>>>>> 14/02/23 18:30:23 ERROR bsp.LocalBSPRunner: Exception during BSP >>>>>> execution! >>>>>> java.util.concurrent.ExecutionException: >>>>>> java.lang.UnsupportedOperationException: Unknown message type: >>>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null] >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) >>>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:315) >>>>>> at java.lang.Thread.run(Thread.java:662) >>>>>> Caused by: java.lang.UnsupportedOperationException: Unknown message >>>>>> type: >>>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null] >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.graph.GraphJobRunner.parseMessages(GraphJobRunner.java:590) >>>>>> at >>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:136) >>>>>> at >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212) >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>>>>> at >>>>>> >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>>>>> at >>>>>> >>>>>> >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) >>>>>> at >>>>>> >>>>>> >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) >>>>>> ... 1 more >>>>>> 14/02/23 18:30:25 INFO bsp.BSPJobClient: Current supersteps number: 1 >>>>>> 14/02/23 18:30:25 INFO bsp.BSPJobClient: Job failed. >>>>>> >>>>>> >>>>>> The only thing I managed to figure is that this comes up when the >>>>>> aggregators are enabled. >>>>>> >>>>>> Thank you. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Edward J. Yoon (@eddieyoon) >>>>> Chief Executive Officer >>>>> DataSayer, Inc. >>>> >>>> >>>> >>>> >>> >>> >>> >>> -- >>> Edward J. Yoon (@eddieyoon) >>> Chief Executive Officer >>> DataSayer, Inc. > > > > -- Edward J. Yoon (@eddieyoon) Chief Executive Officer DataSayer, Inc.
