[ https://issues.apache.org/jira/browse/GIRAPH-293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maja Kabiljo updated GIRAPH-293: -------------------------------- Attachment: GIRAPH-293.patch Making aggregators work correctly with checkpointing - saving the aggregator name, class, value and whether it's persistent. Apart from that, I removed the code for aggregators handling from BspServiceWorker and BspServiceMaster to separate classes, since I think it's cleaner this way, and those two classes do too much different stuff as it is. But that's the reason why the patch looks big. Later with GIRAPH-273 AggregatorHandler classes should become more independent of BspServices. I added test for aggregator serialization and manual restarting from checkpoint (that one also relies on recent GIRAPH-296 and GIRAPH-298 working). The patch passes mvn verify and tests in pseudo-distributed mode. > Should aggregators be checkpointed? > ----------------------------------- > > Key: GIRAPH-293 > URL: https://issues.apache.org/jira/browse/GIRAPH-293 > Project: Giraph > Issue Type: Bug > Reporter: Alessandro Presta > Assignee: Maja Kabiljo > Attachments: GIRAPH-293.patch > > > As I understand, we don't include aggregators in checkpoints because they are > kept in the Zookeeper. > One of our bootcampers is working on fixing TestManualCheckpoint, which > currently involves starting a new job from a checkpoint from a previous job*. > If this is a functionality we want going forward, then persistent aggregators > should be checkpointed. > [*] That test relies on the fact that either aggregators are checkpointed or > they are always reset at each superstep. None of these is happening, but the > error cancels out with the fact that we are not actually resuming from a > checkpoint, but re-running the job from scratch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira