[
https://issues.apache.org/jira/browse/GIRAPH-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021265#comment-14021265
]
Hudson commented on GIRAPH-908:
-------------------------------
ABORTED: Integrated in Giraph-trunk-Commit #1445 (See
[https://builds.apache.org/job/Giraph-trunk-Commit/1445/])
GIRAPH-908: support for partitioned input in giraph (pavanka) (pavanka:
http://git-wip-us.apache.org/repos/asf?p=giraph.git&a=commit&h=4a133f5766c09362917e0416af503c0a00b24e87)
* giraph-core/src/main/java/org/apache/giraph/mapping/DefaultLongByteOps.java
* giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java
* giraph-core/src/main/java/org/apache/giraph/io/MappingReader.java
* giraph-core/src/main/java/org/apache/giraph/mapping/package-info.java
* giraph-hive/src/main/java/org/apache/giraph/hive/common/HiveUtils.java
*
giraph-core/src/main/java/org/apache/giraph/partition/GraphPartitionerFactory.java
*
giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedMappingInputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/partition/LongMappingStorePartitionerFactory.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/mapping/HiveMappingInputFormat.java
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphClasses.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/common/GiraphHiveConstants.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/mapping/package-info.java
*
giraph-core/src/main/java/org/apache/giraph/partition/SimpleLongRangePartitionerFactory.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/mapping/examples/LongInt2ByteHiveToMapping.java
*
giraph-core/src/main/java/org/apache/giraph/partition/HashRangePartitionerFactory.java
*
giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallable.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/mapping/examples/LongByteHiveToMapping.java
* giraph-core/src/main/java/org/apache/giraph/master/MasterThread.java
*
giraph-core/src/main/java/org/apache/giraph/mapping/DefaultEmbeddedLongByteOps.java
*
giraph-core/src/main/java/org/apache/giraph/partition/SimpleWorkerPartitioner.java
*
giraph-core/src/main/java/org/apache/giraph/partition/HashPartitionerFactory.java
* giraph-core/src/main/java/org/apache/giraph/mapping/LongByteMappingStore.java
* giraph-core/src/main/java/org/apache/giraph/mapping/MappingStore.java
* giraph-core/src/main/java/org/apache/giraph/io/MappingInputFormat.java
*
giraph-core/src/main/java/org/apache/giraph/mapping/translate/TranslateEdge.java
*
giraph-core/src/main/java/org/apache/giraph/worker/MappingInputSplitsCallable.java
* giraph-core/src/main/java/org/apache/giraph/mapping/MappingStoreOps.java
*
giraph-core/src/main/java/org/apache/giraph/mapping/translate/package-info.java
* giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java
*
giraph-core/src/main/java/org/apache/giraph/conf/ImmutableClassesGiraphConfiguration.java
*
giraph-core/src/main/java/org/apache/giraph/worker/MappingInputSplitsCallableFactory.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/mapping/AbstractHiveToMapping.java
* CHANGELOG
*
giraph-core/src/test/java/org/apache/giraph/partition/SimpleRangePartitionFactoryTest.java
* giraph-core/src/main/java/org/apache/giraph/worker/FullInputSplitCallable.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/mapping/SimpleHiveToMapping.java
*
giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedMappingReader.java
* giraph-hive/src/main/java/org/apache/giraph/hive/HiveGiraphRunner.java
* giraph-core/src/main/java/org/apache/giraph/mapping/AbstractLongByteOps.java
*
giraph-core/src/main/java/org/apache/giraph/partition/SimplePartitionerFactory.java
*
giraph-core/src/main/java/org/apache/giraph/partition/SimpleIntRangePartitionerFactory.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/mapping/HiveMappingReader.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/mapping/examples/package-info.java
*
giraph-core/src/main/java/org/apache/giraph/io/iterables/MappingReaderWrapper.java
*
giraph-hive/src/main/java/org/apache/giraph/hive/input/mapping/HiveToMapping.java
* giraph-core/src/main/java/org/apache/giraph/bsp/BspService.java
* giraph-core/src/main/java/org/apache/giraph/worker/LocalData.java
*
giraph-core/src/main/java/org/apache/giraph/worker/VertexInputSplitsCallable.java
*
giraph-core/src/main/java/org/apache/giraph/mapping/translate/LongByteTranslateEdge.java
* giraph-core/src/main/java/org/apache/giraph/mapping/MappingEntry.java
* giraph-core/src/main/java/org/apache/giraph/bsp/CentralizedServiceMaster.java
> support for partitioned input in giraph
> ---------------------------------------
>
> Key: GIRAPH-908
> URL: https://issues.apache.org/jira/browse/GIRAPH-908
> Project: Giraph
> Issue Type: Sub-task
> Reporter: Pavan Kumar
> Assignee: Pavan Kumar
> Attachments: GIRAPH-908.patch
>
>
> When the graph we need to work on is already partitioned into a fixed number
> of buckets, with properties such as high edge-locality, low fan-out to other
> buckets, etc. [for instance using techniques such as
> https://people.cam.cornell.edu/~jugander/papers/wsdm13-blp.pdf ]
> we should be able to partition our graph based on such a mapping, to improve
> local-requests, etc. thus avoiding huge network communication. this diff is
> especially useful we repeatedly run algorithms on top of the same graph. In
> such case, we can compute the partitioning once & then use it to speed up
> processing, (also requiring less network bandwidth, etc.) for the rest
> applications on same/ similar graphs.
> The diff is big & partly reviewed by my colleagues. Putting it up for review
--
This message was sent by Atlassian JIRA
(v6.2#6252)
