Hi all, I have managed to build a production giraph application off the hadoop2 profile (but only mapreduce as yarn has few unresolved bugs). I think I have now come across a bug which prevents me from using mutations to do de-duplicate edges in the first superstep of the computation.
Basically whether I use requestRemoveEdges(..) or vertex.removeEdges() the program falls over with this exception: java.lang.IllegalStateException: next: Failed on pos 16 edge (targetVertexId = , value = dataxu) at org.apache.giraph.edge.ByteArrayEdges$ByteArrayEdgeIterator.next( ByteArrayEdges.java:170)at org.apache.giraph.edge.ByteArrayEdges.remove( ByteArrayEdges.java:111)at org.apache.giraph.graph.DefaultVertex.removeEdges(DefaultVertex.java:232)at net.imagini.giraph.EdgeDeduplicationJob.compute(EdgeDeduplicationJob.java:34)at org.apache.giraph.graph.ComputeCallable.computePartition( ComputeCallable.java:247)at org.apache.giraph.graph.ComputeCallable.call( ComputeCallable.java:168)at org.apache.giraph.graph.ComputeCallable.call( ComputeCallable.java:1)at org.apache.giraph.utils.LogStacktraceCallable.call( LogStacktraceCallable.java:51)at java.util.concurrent.FutureTask.run( FutureTask.java:262)at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1145)at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:615)at java.lang.Thread.run(Thread.java:744) I have been using my own class for VertexID but even after reverting to simple Text to see if problem is with my writable implementation I still get what seems a corrupt state of the serialised byte array edges after requesting removal. There is no mention of this problem in jira and googling yields nothing. Anybody tried using removal of edges as part of superstep mutation ? Anybody seen something similar? I am seeing the problem with trunk or release-1.1, haven't tried older. -- Michal Haris Technical Architect direct line: +44 (0) 207 749 0229 www.visualdna.com | t: +44 (0) 207 734 7033,
