Anyone ? -----Message d'origine----- De : VARENE Olivier DTSI/DSI Envoyé : jeudi 4 septembre 2014 15:31 À : dev@giraph.apache.org Objet : BUG ? SequenceFileVertexInputFormat with custom Writable : can not iterate over vertices
Hi, first of all many thanks to the community for the great work you are doing. with giraph 1.1-SNAPSHOT built for hadoop 2.4.0 I am stalled on a bug using custom SequenceFile Vertex Input format. I have opened a project on github to allow you to run/play/view the bug https://github.com/ovarene/giraph_SequenceFileVertexInput you will find everything needed :D I try to load/create my vertices from a SequenceFile<LongWritable,MyWritable> "LongWritable" being the vertexId "MyWritable" being a custom value writable used to generate VertexValue and its edges Vertices are created correctly and loaded (as seen on the log) 2014-09-04 10:44:44,683 DEBUG [load-0] MyReader (SequenceFileLongMyVertexInputFormat.java:getCurrentVertex(108)) - Create Vertex from "1.0 0.0 0:" to : Vertex(id=1,value=1.0,#edges=0) 2014-09-04 10:44:44,694 DEBUG [load-0] MyReader (SequenceFileLongMyVertexInputFormat.java:getCurrentVertex(108)) - Create Vertex from "2.0 0.0 2:1|6|" to : Vertex(id=2,value=2.0,#edges=2) 2014-09-04 10:44:44,695 DEBUG [load-0] MyReader (SequenceFileLongMyVertexInputFormat.java:getCurrentVertex(108)) - Create Vertex from "5.0 0.0 3:4|1|6|" to : Vertex(id=5,value=5.0,#edges=3) 2014-09-04 10:44:44,697 INFO [load-0] worker.InputSplitsCallable (InputSplitsCallable.java:call(235)) - call: Loaded 1 input splits in 0.12198963 secs, (v=3, e=5) 24.592255 vertices/sec, 40.98709 edges/sec But alas, when trying to iterate over them during a compute session, I get the following error ERROR : But when reading in PrintVertex (BasicComputation): 2014-09-04 10:44:44,701 ERROR [load-0] utils.LogStacktraceCallable (LogStacktraceCallable.java:call(57)) - Execution of callable failed java.lang.IllegalStateException: next: IOException at org.apache.giraph.utils.VertexIterator.next(VertexIterator.java:101) at org.apache.giraph.partition.BasicPartition.addPartitionVertices(BasicPartition.java:99) at org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:110) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:482) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:428) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:241) at org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60) at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: ensureRemaining: Only 5 bytes remaining, trying to read 8 at org.apache.giraph.utils.UnsafeByteArrayInputStream.ensureRemaining(UnsafeByteArrayInputStream.java:114) at org.apache.giraph.utils.UnsafeByteArrayInputStream.readLong(UnsafeByteArrayInputStream.java:197) at org.apache.hadoop.io.LongWritable.readFields(LongWritable.java:47) at org.apache.giraph.utils.WritableUtils.reinitializeVertexFromDataInput(WritableUtils.java:522) at org.apache.giraph.utils.VertexIterator.next(VertexIterator.java:98) ... 11 more Any idea ? Thanks a lot Olivier _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
