----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23140/#review47169 -----------------------------------------------------------
Thanks, much shorter now. Should we add some tests to make sure things don't get broken again? giraph-core/src/main/java/org/apache/giraph/bsp/BspService.java <https://reviews.apache.org/r/23140/#comment82778> Why ignore superstep 0? For example there might be a lot of filtering going on during input superstep and it's cheaper to restart from checkpoint than read all the data again giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java <https://reviews.apache.org/r/23140/#comment82781> Interesting, where do we rely on this? giraph-core/src/main/java/org/apache/giraph/utils/io/ExtendedDataInputOutput.java <https://reviews.apache.org/r/23140/#comment82777> Nice bug ;-) giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java <https://reviews.apache.org/r/23140/#comment82787> This is what output threads are called, please name these differently giraph-core/src/main/java/org/apache/giraph/worker/WorkerContext.java <https://reviews.apache.org/r/23140/#comment82775> We are not using Serializable - what's transient here for? giraph-examples/src/test/java/org/apache/giraph/master/TestAggregatorsHandling.java <https://reviews.apache.org/r/23140/#comment82772> Why did you move this file? - Maja Kabiljo On July 2, 2014, 12:57 a.m., Sergey Edunov wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/23140/ > ----------------------------------------------------------- > > (Updated July 2, 2014, 12:57 a.m.) > > > Review request for giraph. > > > Repository: giraph-git > > > Description > ------- > > This fix merely makes checkpointing work again. > > > Diffs > ----- > > > giraph-core/src/main/java/org/apache/giraph/aggregators/AggregatorWrapper.java > 9613805 > giraph-core/src/main/java/org/apache/giraph/bsp/BspService.java 2e35373 > giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java f0ecca2 > giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java > 7d7ceb2 > giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java > ad7e045 > > giraph-core/src/main/java/org/apache/giraph/master/MasterAggregatorHandler.java > 325d91f > > giraph-core/src/main/java/org/apache/giraph/partition/BasicPartitionOwner.java > 545d1af > > giraph-core/src/main/java/org/apache/giraph/partition/HashMasterPartitioner.java > 240687e > > giraph-core/src/main/java/org/apache/giraph/partition/HashWorkerPartitioner.java > d833895 > > giraph-core/src/main/java/org/apache/giraph/partition/MasterGraphPartitioner.java > 50c750a > > giraph-core/src/main/java/org/apache/giraph/partition/PartitionBalancer.java > 3454d62 > giraph-core/src/main/java/org/apache/giraph/partition/PartitionOwner.java > 0ac74da > > giraph-core/src/main/java/org/apache/giraph/partition/SimpleMasterPartitioner.java > f128f34 > > giraph-core/src/main/java/org/apache/giraph/partition/SimpleWorkerPartitioner.java > 3c0de44 > > giraph-core/src/main/java/org/apache/giraph/partition/WorkerGraphPartitioner.java > 004ea81 > giraph-core/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java > 09dd46d > > giraph-core/src/main/java/org/apache/giraph/utils/io/ExtendedDataInputOutput.java > af45426 > giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java > 8dcf19a > giraph-core/src/main/java/org/apache/giraph/worker/WorkerContext.java > 17347db > > giraph-core/src/test/java/org/apache/giraph/partition/SimpleRangePartitionFactoryTest.java > 96bd5d7 > > giraph-examples/src/test/java/org/apache/giraph/aggregators/TestAggregatorsHandling.java > e2b611b > > giraph-examples/src/test/java/org/apache/giraph/master/TestAggregatorsHandling.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/23140/diff/ > > > Testing > ------- > > I tested it running multiple different jobs. I run page rank on 2*10^9 > vertices on 200 workers and it seems to work just fine. It only takes 2 > minutes to save checkpoint. > > > Thanks, > > Sergey Edunov > >
