> On Aug. 25, 2013, 8:30 p.m., Nitay Joffe wrote:
> > giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java,
> > line 1099
> > <https://reviews.apache.org/r/13756/diff/2/?file=344112#file344112line1099>
> >
> > Should we track and add something in here about number of edges saved
> > instead of just vertices, seeing as this is all about edge writing?
something like this is acceptable?
LoggerUtils.setStatusAndLog(getContext(), LOG, Level.INFO,
"saveEdges: Saved " + edges +
" edges out of " + partition.getEdgeCount() +
" partition edges, on partition " + partitionIndex +
" out of " + numPartitions);
- Armando
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13756/#review25523
-----------------------------------------------------------
On Aug. 23, 2013, 1:34 a.m., Armando wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13756/
> -----------------------------------------------------------
>
> (Updated Aug. 23, 2013, 1:34 a.m.)
>
>
> Review request for giraph, Armando, Claudio Martella, and Nitay Joffe.
>
>
> Bugs: GIRAPH-732
> https://issues.apache.org/jira/browse/GIRAPH-732
>
>
> Repository: giraph-git
>
>
> Description
> -------
>
> This patch is a possible solution to provide an EdgeOutputFormat API in
> Giraph.
> The idea is to add the same logic as the one for the vertices for edges. For
> this reason I add saveEdges after saveVertices.
> This requires the user to have to different paths for the resulting files.
> For this reason I have implemented TextEdgeOutputFormat
> and modified TextVertexOutputFormat so that a subdirectory can be specified.
> The code I provide should be compliant with previous versions of hadoop as
> well as YARN (I was yet not able to test it on YARN).
> It is also retro-compatible since when not specified, TextVertexOutputFormat
> behaves as before.
> I am also providing an actual usable implementation with the associated tests
> (SrcIdDstIdEdgeValueOutputFormat).
> Also very interesting is the fact that this implementation is totally
> transparent to the classes implementing TextVertexOutputFormat.
>
> passed "maven verify"
>
>
> Diffs
> -----
>
> giraph-core/src/main/java/org/apache/giraph/GiraphRunner.java 1bd79b5
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphClasses.java 71fe885
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java
> 23bcd32
> giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java
> c276c2a
>
> giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java
> 49a2ebc
> giraph-core/src/main/java/org/apache/giraph/io/EdgeOutputFormat.java
> PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/io/EdgeWriter.java PRE-CREATION
>
> giraph-core/src/main/java/org/apache/giraph/io/formats/GiraphTextOutputFormat.java
> PRE-CREATION
>
> giraph-core/src/main/java/org/apache/giraph/io/formats/SrcIdDstIdEdgeValueTextOutputFormat.java
> PRE-CREATION
>
> giraph-core/src/main/java/org/apache/giraph/io/formats/TextEdgeOutputFormat.java
> PRE-CREATION
>
> giraph-core/src/main/java/org/apache/giraph/io/formats/TextVertexOutputFormat.java
> c91d543
>
> giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeOutputFormat.java
> PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/utils/ConfigurationUtils.java
> 745764b
> giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java
> da1e7fb
>
> giraph-core/src/test/java/org/apache/giraph/io/TestSrcIdDstIdEdgeValueTextOutputFormat.java
> PRE-CREATION
>
> Diff: https://reviews.apache.org/r/13756/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Armando
>
>
