Hi Chadi, That does seem like a serialization issue. Which OutEdges class are you using, is it something you implemented?
Regards, Maja From: chadi jaber <chadijaber...@hotmail.com<mailto:chadijaber...@hotmail.com>> Reply-To: "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@giraph.apache.org<mailto:user@giraph.apache.org>> Date: Thursday, January 9, 2014 2:08 AM To: Lukas Nalezenec <lukas.naleze...@firma.seznam.cz<mailto:lukas.naleze...@firma.seznam.cz>>, "user@giraph.apache.org<mailto:user@giraph.apache.org>" <user@giraph.apache.org<mailto:user@giraph.apache.org>> Subject: RE: Problem with Giraph (please help me) Hello Lukas I have enclosed in my previous emails the exception. It seems to be a serialization issue (This occurs only when workers > 1) ... 2013-12-31 16:27:33,494 INFO org.apache.giraph.comm.netty.NettyClient: connectAllAddresses: Successfully added 4 connections, (4 total connected) 0 failed, 0 failures total. 2013-12-31 16:27:33,501 INFO org.apache.giraph.worker.BspServiceWorker: loadInputSplits: Using 1 thread(s), originally 1 threads(s) for 1 total splits. 2013-12-31 16:27:33,508 INFO org.apache.giraph.comm.SendPartitionCache: SendPartitionCache: maxVerticesPerTransfer = 10000 2013-12-31 16:27:33,508 INFO org.apache.giraph.comm.SendPartitionCache: SendPartitionCache: maxEdgesPerTransfer = 80000 2013-12-31 16:27:33,524 INFO org.apache.giraph.worker.InputSplitsCallable: call: Loaded 0 input splits in 0.020270009 secs, (v=0, e=0) 0.0 vertices/sec, 0.0 edges/sec 2013-12-31 16:27:33,527 INFO org.apache.giraph.comm.netty.NettyClient: waitAllRequests: Finished all requests. MBytes/sec sent = 0, MBytes/sec received = 0, MBytesSent = 0, MBytesReceived = 0, ave sent req MBytes = 0, ave received req MBytes = 0, secs waited = 0.656 2013-12-31 16:27:33,527 INFO org.apache.giraph.worker.BspServiceWorker: setup: Finally loaded a total of (v=0, e=0) 2013-12-31 16:27:33,598 INFO org.apache.giraph.comm.netty.handler.RequestDecoder: decode: Server window metrics MBytes/sec sent = 0, MBytes/sec received = 0, MBytesSent = 0, MBytesReceived = 0, ave sent req MBytes = 0, ave received req MBytes = 0, secs waited = 0.816 2013-12-31 16:27:33,605 WARN org.apache.giraph.comm.netty.handler.RequestServerHandler: exceptionCaught: Channel failed with remote address /172.16.45.53:59257 java.io.EOFException at org.jboss.netty.buffer.ChannelBufferInputStream.checkAvailable(ChannelBufferInputStream.java:231) at org.jboss.netty.buffer.ChannelBufferInputStream.readInt(ChannelBufferInputStream.java:174) at org.apache.giraph.edge.ByteArrayEdges.readFields(ByteArrayEdges.java:172) at org.apache.giraph.utils.WritableUtils.reinitializeVertexFromDataInput(WritableUtils.java:480) at org.apache.giraph.utils.WritableUtils.readVertexFromDataInput(WritableUtils.java:511) at org.apache.giraph.partition.SimplePartition.readFields(SimplePartition.java:126) at org.apache.giraph.comm.requests.SendVertexRequest.readFieldsRequest(SendVertexRequest.java:66) at org.apache.giraph.comm.requests.WritableRequest.readFields(WritableRequest.java:120) at org.apache.giraph.comm.netty.handler.RequestDecoder.decode(RequestDecoder.java:92) at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:72) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:69) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) the code for my vertex compute function : public class MergeVertex extends Vertex<LongWritable,DoubleWritable, DoubleWritable, NodeMessage> { ... /*** * Convert a Vertex Id from its LongWritable format to Point format (2 Element Array Format) * @param lng LongWritable Format of the VertexId * @return Alignment point Array */ public static int[] cvtLongToPoint(LongWritable lng){ int[] point={0,0}; point[0]=(int) (lng.get()/1000); point[1]=(int) (lng.get()% 1000); return point; } @Override public void compute(Iterable<NodeMessage> messages) throws IOException { int currentId[]= cvtLongToPoint(getId()); if (getSuperstep()==0) { //NodeValue nv=new NodeValue(); setValue(new DoubleWritable(0d)); } _signallength=getContext().getConfiguration().getInt("SignalLength",0); if((getSuperstep() < _signallength && getId().get()!=0L) || (getSuperstep()== 0 && getId().get()==0L)){ LongWritable dstId=new LongWritable(); //Nodes which are on Graph "Spine" //Remaining Edges Construction if(currentId[0]== currentId[1]){ //right Side for (int i=currentId[1]+1;i<_signallength;i++){ dstId=cvtPointToLong(currentId[0]+1,i); addVertexRequest(dstId,new DoubleWritable(Double.MAX_VALUE)); addEdgeRequest(getId(),EdgeFactory.create(dstId, new DoubleWritable(computeCost(getId(),dstId)))); } //Left Side for (int i=currentId[0]+2;i<_signallength;i++){ dstId=cvtPointToLong(i,currentId[1]+1); addVertexRequest(dstId,new DoubleWritable(Double.MAX_VALUE)); addEdgeRequest(getId(),EdgeFactory.create(dstId, new DoubleWritable(computeCost(getId(),dstId)))); } //Nodes which are not on Graph "Spine" //Remaining Edges Construction }else{ //right Side if(currentId[0]+1<_signallength){ for (int i=currentId[1]+1;i<_signallength;i++){ dstId=cvtPointToLong(currentId[0]+1,i); addEdgeRequest(getId(),EdgeFactory.create(dstId, new DoubleWritable(computeCost(getId(),dstId)))); } } //Left Side if(currentId[1]+1<_signallength){ for (int i=currentId[0]+2;i<_signallength;i++){ dstId=cvtPointToLong(i,currentId[1]+1); addEdgeRequest(getId(),EdgeFactory.create(dstId, new DoubleWritable(computeCost(getId(),dstId)))); } } } //No need to other vertex than source to be active if(getId().get() != 0L){ voteToHalt(); } }else if (getSuperstep() >= _signallength && getSuperstep() < 2*_signallength){ double minDist; long minSource=0L; if(getId().get() == 0L){ minDist=0; }else{ minDist=Double.MAX_VALUE; } for(NodeMessage message : messages){ if(minDist > message.get()){ minDist=message.get(); minSource=message.getSourceID(); } } if (minDist < getValue().get()){ setValue(new DoubleWritable(minDist)); for (Edge<LongWritable, DoubleWritable> edge : getEdges()) { double distance = minDist + edge.getValue().get(); sendMessage(edge.getTargetVertexId(), new NodeMessage(distance,getId().get())); } } //Only last Node is active if(currentId[0] != _signallength-1 || currentId[1] != _signallength-1){ voteToHalt(); } }else if(getSuperstep() >= 2*_signallength){ voteToHalt(); } } If you need more details please don't hesitate. Thanks in advance, Chadi ________________________________ Date: Thu, 9 Jan 2014 10:49:54 +0100 From: lukas.naleze...@firma.seznam.cz<mailto:lukas.naleze...@firma.seznam.cz> To: chadijaber...@hotmail.com<mailto:chadijaber...@hotmail.com> Subject: Re: Problem with Giraph (please help me) Hi, Find the mapper running on the remote address and check what happened. Maybe there will be exception. Lukas On 9.1.2014 09:38, chadi jaber wrote: exceptionCaught: Channel failed with remote address /172.16.45.53:59257
