> On Aug. 26, 2013, 12:34 p.m., Claudio Martella wrote: > > giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java, > > line 1093 > > <https://reviews.apache.org/r/13756/diff/3/?file=344713#file344713line1093> > > > > can you count these in the for-loop above instead of using > > getNumEdges()? As you're already iterating them, it costs nothing, and we > > avoid recounting them for those implementations of OutEdges that actually > > count edges every time at that call (such as the byte-serialized ones). > > Armando wrote: > I don't understand what you mean: since getNumEdges is a method of > Vertex, I call it once per vertex not more, am I wrong?
I'm saying that for some implementations of OutEdges the cost of this method might not be O(1) (Vertex calls the method on the OutEdges). As you're already iterating over the edges, you might piggyback that count earlier. - Claudio ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13756/#review25549 ----------------------------------------------------------- On Aug. 26, 2013, 12:17 p.m., Armando wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/13756/ > ----------------------------------------------------------- > > (Updated Aug. 26, 2013, 12:17 p.m.) > > > Review request for giraph, Armando, Claudio Martella, and Nitay Joffe. > > > Bugs: GIRAPH-732 > https://issues.apache.org/jira/browse/GIRAPH-732 > > > Repository: giraph-git > > > Description > ------- > > This patch is a possible solution to provide an EdgeOutputFormat API in > Giraph. > The idea is to add the same logic as the one for the vertices for edges. For > this reason I add saveEdges after saveVertices. > This requires the user to have to different paths for the resulting files. > For this reason I have implemented TextEdgeOutputFormat > and modified TextVertexOutputFormat so that a subdirectory can be specified. > The code I provide should be compliant with previous versions of hadoop as > well as YARN (I was yet not able to test it on YARN). > It is also retro-compatible since when not specified, TextVertexOutputFormat > behaves as before. > I am also providing an actual usable implementation with the associated tests > (SrcIdDstIdEdgeValueOutputFormat). > Also very interesting is the fact that this implementation is totally > transparent to the classes implementing TextVertexOutputFormat. > > passed "maven verify" > > > Diffs > ----- > > giraph-core/src/main/java/org/apache/giraph/GiraphRunner.java 1bd79b5 > giraph-core/src/main/java/org/apache/giraph/conf/GiraphClasses.java 71fe885 > giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java > 23bcd32 > giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java > c276c2a > > giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java > 49a2ebc > giraph-core/src/main/java/org/apache/giraph/io/EdgeOutputFormat.java > PRE-CREATION > giraph-core/src/main/java/org/apache/giraph/io/EdgeWriter.java PRE-CREATION > > giraph-core/src/main/java/org/apache/giraph/io/formats/GiraphTextOutputFormat.java > PRE-CREATION > > giraph-core/src/main/java/org/apache/giraph/io/formats/IdWithValueTextOutputFormat.java > bd69586 > > giraph-core/src/main/java/org/apache/giraph/io/formats/SrcIdDstIdEdgeValueTextOutputFormat.java > PRE-CREATION > > giraph-core/src/main/java/org/apache/giraph/io/formats/TextEdgeOutputFormat.java > PRE-CREATION > > giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexOutputFormat.java > c91d543 > > giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeOutputFormat.java > PRE-CREATION > giraph-core/src/main/java/org/apache/giraph/utils/ConfigurationUtils.java > 745764b > giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java > da1e7fb > > giraph-core/src/test/java/org/apache/giraph/io/TestSrcIdDstIdEdgeValueTextOutputFormat.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/13756/diff/ > > > Testing > ------- > > > Thanks, > > Armando > >
