Re: Problems with running page rank using OutOfCore setting

> > So, does this mean that there is no other solution but to increase the > physical memory? > > > Cheers, > > Darshan > > > On 16 Jul 2017, at 23:04, Hassan Eslami sn.esl...@gmail.com>> wrote: > > Hi, > > giraph.useOutOfCoreMessages is no longe

Re: Problems with running page rank using OutOfCore setting

Hi, giraph.useOutOfCoreMessages is no longer in use. The main problem here is that you are using default flow control mechanism (NoOpFlowControl), that causes a lot of outstanding/received messages. As a consequence, you fill up the memory so fast, and the job would fail for various reasons. Plea

[jira] [Created] (GIRAPH-1150) Fixing MemoryEstimatorOracle

Hassan Eslami created GIRAPH-1150: - Summary: Fixing MemoryEstimatorOracle Key: GIRAPH-1150 URL: https://issues.apache.org/jira/browse/GIRAPH-1150 Project: Giraph Issue Type: Bug

[jira] [Resolved] (GIRAPH-1125) Add memory estimation mechanism to out-of-core

[ https://issues.apache.org/jira/browse/GIRAPH-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1125. --- Resolution: Fixed > Add memory estimation mechanism to out-of-c

[jira] [Commented] (GIRAPH-1144) Out-of-core control flow sends messages after Netty shuts down

[ https://issues.apache.org/jira/browse/GIRAPH-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979192#comment-15979192 ] Hassan Eslami commented on GIRAPH-1144: --- This is the "resume signal&qu

[jira] [Created] (GIRAPH-1137) Remove channel probing from Netty worker thread for credit-based flow-control

Hassan Eslami created GIRAPH-1137: - Summary: Remove channel probing from Netty worker thread for credit-based flow-control Key: GIRAPH-1137 URL: https://issues.apache.org/jira/browse/GIRAPH-1137

[jira] [Created] (GIRAPH-1125) Add memory estimation mechanism to out-of-core

Hassan Eslami created GIRAPH-1125: - Summary: Add memory estimation mechanism to out-of-core Key: GIRAPH-1125 URL: https://issues.apache.org/jira/browse/GIRAPH-1125 Project: Giraph Issue Type

Re: [VOTE] Apache Giraph 1.2.0 RC1

most important are: > - New out of core implementation (Hassan Eslami & Dionysios Logothetis) > - FacebookConfiguration, this is a set of parameters used internally > in Facebook > to run Giraph jobs. (Maja Kabiljo) > - Multiple improvements to Blocks framework (Igor Kabil

[jira] [Updated] (GIRAPH-1088) We should be able to enable out-of-core mechanism with minimal flags/options

[ https://issues.apache.org/jira/browse/GIRAPH-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami updated GIRAPH-1088: -- Fix Version/s: (was: 1.2.0) > We should be able to enable out-of-core mechanism w

[jira] [Resolved] (GIRAPH-1089) In fixed out-of-core policy the definition of in-memory partition is not clear

[ https://issues.apache.org/jira/browse/GIRAPH-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1089. --- Resolution: Fixed Thanks Maja for reviewing and committing the diff! > In fixed out-of-c

[jira] [Updated] (GIRAPH-1074) Update website description for out-of-core

[ https://issues.apache.org/jira/browse/GIRAPH-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami updated GIRAPH-1074: -- Fix Version/s: (was: 1.2.0) > Update website description for out-of-c

[jira] [Commented] (GIRAPH-1089) In fixed out-of-core policy the definition of in-memory partition is not clear

[ https://issues.apache.org/jira/browse/GIRAPH-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373324#comment-15373324 ] Hassan Eslami commented on GIRAPH-1089: --- https://reviews.facebook.net/D60573

[jira] [Created] (GIRAPH-1089) In fixed out-of-core policy the definition of in-memory partition is not clear

Hassan Eslami created GIRAPH-1089: - Summary: In fixed out-of-core policy the definition of in-memory partition is not clear Key: GIRAPH-1089 URL: https://issues.apache.org/jira/browse/GIRAPH-1089

[jira] [Commented] (GIRAPH-1088) We should be able to enable out-of-core mechanism with minimal flags/options

[ https://issues.apache.org/jira/browse/GIRAPH-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373318#comment-15373318 ] Hassan Eslami commented on GIRAPH-1088: --- https://reviews.facebook.net/D60609

[jira] [Created] (GIRAPH-1088) We should be able to enable out-of-core mechanism with minimal flags/options

Hassan Eslami created GIRAPH-1088: - Summary: We should be able to enable out-of-core mechanism with minimal flags/options Key: GIRAPH-1088 URL: https://issues.apache.org/jira/browse/GIRAPH-1088

[jira] [Assigned] (GIRAPH-1088) We should be able to enable out-of-core mechanism with minimal flags/options

[ https://issues.apache.org/jira/browse/GIRAPH-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami reassigned GIRAPH-1088: - Assignee: Hassan Eslami > We should be able to enable out-of-core mechanism w

[jira] [Commented] (GIRAPH-1074) Update website description for out-of-core

[ https://issues.apache.org/jira/browse/GIRAPH-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373315#comment-15373315 ] Hassan Eslami commented on GIRAPH-1074: --- https://reviews.facebook.net/D6

[jira] [Resolved] (GIRAPH-1081) Fix a bug in internal out-of-core infra: multithreaded accesses to buffers

[ https://issues.apache.org/jira/browse/GIRAPH-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1081. --- Resolution: Fixed Thanks Maja for reviewing and committing the diff! > Fix a bug

[jira] [Assigned] (GIRAPH-1081) Fix a bug in internal out-of-core infra: multithreaded accesses to buffers

[ https://issues.apache.org/jira/browse/GIRAPH-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami reassigned GIRAPH-1081: - Assignee: Hassan Eslami > Fix a bug in internal out-of-core infra: multithrea

[jira] [Created] (GIRAPH-1081) Fix a bug in internal out-of-core infra: multithreaded accesses to buffers

Hassan Eslami created GIRAPH-1081: - Summary: Fix a bug in internal out-of-core infra: multithreaded accesses to buffers Key: GIRAPH-1081 URL: https://issues.apache.org/jira/browse/GIRAPH-1081 Project

[jira] [Commented] (GIRAPH-1081) Fix a bug in internal out-of-core infra: multithreaded accesses to buffers

[ https://issues.apache.org/jira/browse/GIRAPH-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354188#comment-15354188 ] Hassan Eslami commented on GIRAPH-1081: --- https://reviews.facebook.net/D6

[jira] [Updated] (GIRAPH-1074) Update website description for out-of-core

[ https://issues.apache.org/jira/browse/GIRAPH-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami updated GIRAPH-1074: -- Fix Version/s: 1.2.0 > Update website description for out-of-c

[jira] [Resolved] (GIRAPH-1073) Decouple out-of-core persistence infrastructure from out-of-core computation

[ https://issues.apache.org/jira/browse/GIRAPH-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1073. --- Resolution: Fixed Thanks Maja Kabiljo for reviewing and committing the diff. > Decouple

[jira] [Resolved] (GIRAPH-1072) Improve metrics reported in out-of-core computation

[ https://issues.apache.org/jira/browse/GIRAPH-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1072. --- Resolution: Fixed Thanks Dionysios for reviewing the code and Maja for final reviewing and

[jira] [Created] (GIRAPH-1074) Update website description for out-of-core

Hassan Eslami created GIRAPH-1074: - Summary: Update website description for out-of-core Key: GIRAPH-1074 URL: https://issues.apache.org/jira/browse/GIRAPH-1074 Project: Giraph Issue Type

[jira] [Commented] (GIRAPH-1073) Decouple out-of-core persistence infrastructure from out-of-core computation

[ https://issues.apache.org/jira/browse/GIRAPH-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332260#comment-15332260 ] Hassan Eslami commented on GIRAPH-1073: --- https://reviews.facebook.net/D5

[jira] [Created] (GIRAPH-1073) Decouple out-of-core persistence infrastructure from out-of-core computation

Hassan Eslami created GIRAPH-1073: - Summary: Decouple out-of-core persistence infrastructure from out-of-core computation Key: GIRAPH-1073 URL: https://issues.apache.org/jira/browse/GIRAPH-1073

[jira] [Commented] (GIRAPH-1072) Improve metrics reported in out-of-core computation

[ https://issues.apache.org/jira/browse/GIRAPH-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332236#comment-15332236 ] Hassan Eslami commented on GIRAPH-1072: --- https://reviews.facebook.net/D5

[jira] [Created] (GIRAPH-1072) Improve metrics reported in out-of-core computation

Hassan Eslami created GIRAPH-1072: - Summary: Improve metrics reported in out-of-core computation Key: GIRAPH-1072 URL: https://issues.apache.org/jira/browse/GIRAPH-1072 Project: Giraph Issue

[jira] [Created] (GIRAPH-1071) Make out-of-core capable of handling massive mutations

Hassan Eslami created GIRAPH-1071: - Summary: Make out-of-core capable of handling massive mutations Key: GIRAPH-1071 URL: https://issues.apache.org/jira/browse/GIRAPH-1071 Project: Giraph

[jira] [Commented] (GIRAPH-1071) Make out-of-core capable of handling massive mutations

[ https://issues.apache.org/jira/browse/GIRAPH-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332226#comment-15332226 ] Hassan Eslami commented on GIRAPH-1071: --- https://reviews.facebook.net/D5

[jira] [Resolved] (GIRAPH-1067) Out-of-core message mechanism needs clean up

[ https://issues.apache.org/jira/browse/GIRAPH-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1067. --- Resolution: Fixed Thanks to Maja Kabiljo for reviewing and committing the code. > Out

[jira] [Commented] (GIRAPH-1067) Out-of-core message mechanism needs clean up

[ https://issues.apache.org/jira/browse/GIRAPH-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308816#comment-15308816 ] Hassan Eslami commented on GIRAPH-1067: --- https://reviews.facebook.net/D5

[jira] [Created] (GIRAPH-1067) Out-of-core message mechanism needs clean up

Hassan Eslami created GIRAPH-1067: - Summary: Out-of-core message mechanism needs clean up Key: GIRAPH-1067 URL: https://issues.apache.org/jira/browse/GIRAPH-1067 Project: Giraph Issue Type

[jira] [Resolved] (GIRAPH-1026) New Out-of-core mechanism does not work

[ https://issues.apache.org/jira/browse/GIRAPH-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1026. --- Resolution: Fixed This issue has been resolved in the new out-of-core design in GIRAPH-1066

[jira] [Commented] (GIRAPH-462) Multithreading breaks out-of-core graph

[ https://issues.apache.org/jira/browse/GIRAPH-462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294416#comment-15294416 ] Hassan Eslami commented on GIRAPH-462: -- The new out-of-core feature has been pu

[jira] [Resolved] (GIRAPH-1066) Functional adaptive out-of-core mechanism

[ https://issues.apache.org/jira/browse/GIRAPH-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1066. --- Resolution: Fixed Thanks Dionysios Logothetis for reviewing the code and Sergey Edunov for

[jira] [Commented] (GIRAPH-1066) Functional adaptive out-of-core mechanism

[ https://issues.apache.org/jira/browse/GIRAPH-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290341#comment-15290341 ] Hassan Eslami commented on GIRAPH-1066: --- https://reviews.facebook.net/D5

[jira] [Created] (GIRAPH-1066) Functional adaptive out-of-core mechanism

Hassan Eslami created GIRAPH-1066: - Summary: Functional adaptive out-of-core mechanism Key: GIRAPH-1066 URL: https://issues.apache.org/jira/browse/GIRAPH-1066 Project: Giraph Issue Type: New

[jira] [Resolved] (GIRAPH-1057) Decoupling NettyClient from control flow policy

[ https://issues.apache.org/jira/browse/GIRAPH-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1057. --- Resolution: Fixed Thanks Dionysis Logothetis for reviewing the code and Sergey Edunov for

[jira] [Created] (GIRAPH-1057) Decoupling NettyClient from control flow policy

Hassan Eslami created GIRAPH-1057: - Summary: Decoupling NettyClient from control flow policy Key: GIRAPH-1057 URL: https://issues.apache.org/jira/browse/GIRAPH-1057 Project: Giraph Issue

[jira] [Commented] (GIRAPH-1057) Decoupling NettyClient from control flow policy

[ https://issues.apache.org/jira/browse/GIRAPH-1057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15258591#comment-15258591 ] Hassan Eslami commented on GIRAPH-1057: --- https://reviews.facebook.net/D5

[jira] [Resolved] (GIRAPH-1025) Flow-control mechanism improving out-of-core feature

[ https://issues.apache.org/jira/browse/GIRAPH-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1025. --- Resolution: Fixed Addressed in GIRAPH-1027 > Flow-control mechanism improving out-of-c

[jira] [Resolved] (GIRAPH-1056) Setting auto-read in Netty to false

[ https://issues.apache.org/jira/browse/GIRAPH-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1056. --- Resolution: Fixed > Setting auto-read in Netty to fa

[jira] [Commented] (GIRAPH-1056) Setting auto-read in Netty to false

[ https://issues.apache.org/jira/browse/GIRAPH-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15258565#comment-15258565 ] Hassan Eslami commented on GIRAPH-1056: --- Thanks Maja for reviewing and commit

[jira] [Commented] (GIRAPH-1056) Setting auto-read in Netty to false

[ https://issues.apache.org/jira/browse/GIRAPH-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15258564#comment-15258564 ] Hassan Eslami commented on GIRAPH-1056: --- https://reviews.facebook.net/D5

[jira] [Created] (GIRAPH-1056) Setting auto-read in Netty to false

Hassan Eslami created GIRAPH-1056: - Summary: Setting auto-read in Netty to false Key: GIRAPH-1056 URL: https://issues.apache.org/jira/browse/GIRAPH-1056 Project: Giraph Issue Type

[jira] [Assigned] (GIRAPH-1056) Setting auto-read in Netty to false

[ https://issues.apache.org/jira/browse/GIRAPH-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami reassigned GIRAPH-1056: - Assignee: Hassan Eslami > Setting auto-read in Netty to fa

[jira] [Resolved] (GIRAPH-1027) Improve flow-control on sender side

[ https://issues.apache.org/jira/browse/GIRAPH-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1027. --- Resolution: Fixed > Improve flow-control on sender s

[jira] [Commented] (GIRAPH-1027) Improve flow-control on sender side

[ https://issues.apache.org/jira/browse/GIRAPH-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225292#comment-15225292 ] Hassan Eslami commented on GIRAPH-1027: --- Thanks Dionysios and Maja for revie

[jira] [Resolved] (GIRAPH-1009) Spammy 'lost reservation' messages from ZooKeeper in workers' log at the end of the computation.

[ https://issues.apache.org/jira/browse/GIRAPH-1009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1009. --- Resolution: Fixed > Spammy 'lost reservation' messages from ZooKeeper in worker

[jira] [Resolved] (GIRAPH-1048) Redesign of out-of-core mechanism (first patch -- out-of-core mechanism keeping fixed number of partitions in memory)

[ https://issues.apache.org/jira/browse/GIRAPH-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami resolved GIRAPH-1048. --- Resolution: Fixed Thanks to Dionysis for reviewing the diff and Sergey Edunov for committing

[jira] [Commented] (GIRAPH-1048) Redesign of out-of-core mechanism (first patch -- out-of-core mechanism keeping fixed number of partitions in memory)

[ https://issues.apache.org/jira/browse/GIRAPH-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194615#comment-15194615 ] Hassan Eslami commented on GIRAPH-1048: --- https://reviews.facebook.net/D5

[jira] [Created] (GIRAPH-1048) Redesign of out-of-core mechanism (first patch -- out-of-core mechanism keeping fixed number of partitions in memory)

Hassan Eslami created GIRAPH-1048: - Summary: Redesign of out-of-core mechanism (first patch -- out-of-core mechanism keeping fixed number of partitions in memory) Key: GIRAPH-1048 URL: https://issues.apache.org

Re: OOC on Giraph 1.1 or 1.2

Yes, OOC mechanism in 1.1 may not work in certain scenarios: - If a worker goes into long GC pauses (which is likely since a job that is supposed to use OOC feature usually suffer from high memory pressure), all incoming messages will be buffered before coming to Giraph runtime, and this will caus

[jira] [Commented] (GIRAPH-1026) New Out-of-core mechanism does not work

[ https://issues.apache.org/jira/browse/GIRAPH-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734141#comment-14734141 ] Hassan Eslami commented on GIRAPH-1026: --- Thanks Vitaly for reporting the issue

[jira] [Assigned] (GIRAPH-1026) New Out-of-core mechanism does not work

[ https://issues.apache.org/jira/browse/GIRAPH-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami reassigned GIRAPH-1026: - Assignee: Hassan Eslami > New Out-of-core mechanism does not w

[jira] [Commented] (GIRAPH-1026) New Out-of-core mechanism does not work

[ https://issues.apache.org/jira/browse/GIRAPH-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662220#comment-14662220 ] Hassan Eslami commented on GIRAPH-1026: --- Thanks Vitaly for describing the is

[jira] [Commented] (GIRAPH-1027) Improve flow-control on sender side

[ https://issues.apache.org/jira/browse/GIRAPH-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662203#comment-14662203 ] Hassan Eslami commented on GIRAPH-1027: --- Code review discus

[jira] [Created] (GIRAPH-1027) Improve flow-control on sender side

Hassan Eslami created GIRAPH-1027: - Summary: Improve flow-control on sender side Key: GIRAPH-1027 URL: https://issues.apache.org/jira/browse/GIRAPH-1027 Project: Giraph Issue Type

[jira] [Commented] (GIRAPH-1026) New Out-of-core mechanism does not work

[ https://issues.apache.org/jira/browse/GIRAPH-1026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653913#comment-14653913 ] Hassan Eslami commented on GIRAPH-1026: --- Thanks Max for bringing this up. I

[jira] [Commented] (GIRAPH-1025) Flow-control mechanism improving out-of-core feature

[ https://issues.apache.org/jira/browse/GIRAPH-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650608#comment-14650608 ] Hassan Eslami commented on GIRAPH-1025: --- Code review discussion: h

[jira] [Created] (GIRAPH-1025) Flow-control mechanism improving out-of-core feature

Hassan Eslami created GIRAPH-1025: - Summary: Flow-control mechanism improving out-of-core feature Key: GIRAPH-1025 URL: https://issues.apache.org/jira/browse/GIRAPH-1025 Project: Giraph

[jira] [Updated] (GIRAPH-1023) Adding out-of-core messages to the newly added adaptive out-of-core mechanism

[ https://issues.apache.org/jira/browse/GIRAPH-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami updated GIRAPH-1023: -- Description: GIRAPH-1022 implements an adaptive mechanism for out-of-core capable of handling

[jira] [Commented] (GIRAPH-1023) Adding out-of-core messages to the newly added adaptive out-of-core mechanism

[ https://issues.apache.org/jira/browse/GIRAPH-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643596#comment-14643596 ] Hassan Eslami commented on GIRAPH-1023: --- https://reviews.facebook.net/D4

[jira] [Created] (GIRAPH-1023) Adding out-of-core messages to the newly added adaptive out-of-core mechanism

Hassan Eslami created GIRAPH-1023: - Summary: Adding out-of-core messages to the newly added adaptive out-of-core mechanism Key: GIRAPH-1023 URL: https://issues.apache.org/jira/browse/GIRAPH-1023

[jira] [Assigned] (GIRAPH-1022) Out-of-core mechanism for input superstep and graph data

[ https://issues.apache.org/jira/browse/GIRAPH-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami reassigned GIRAPH-1022: - Assignee: Hassan Eslami > Out-of-core mechanism for input superstep and graph d

[jira] [Updated] (GIRAPH-1022) Out-of-core mechanism for input superstep and graph data

[ https://issues.apache.org/jira/browse/GIRAPH-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Eslami updated GIRAPH-1022: -- Description: Currently the out-of-core mechanism has several issues: - It does not work

[jira] [Created] (GIRAPH-1022) Out-of-core mechanism for input superstep and graph data

Hassan Eslami created GIRAPH-1022: - Summary: Out-of-core mechanism for input superstep and graph data Key: GIRAPH-1022 URL: https://issues.apache.org/jira/browse/GIRAPH-1022 Project: Giraph

Re: When Giraph writes to output files

Khaled, Here is what I think: The code section you are looking at is executed per superstep for each vertex, and is intended for outputs during a superstep. The code section you actually want to look at is "cleanup" in "BspServiceWorker" where it calls "saveVertices" and "saveEdges". Best, Hassan

[jira] [Created] (GIRAPH-1021) Missing progress report for graph mutations

Hassan Eslami created GIRAPH-1021: - Summary: Missing progress report for graph mutations Key: GIRAPH-1021 URL: https://issues.apache.org/jira/browse/GIRAPH-1021 Project: Giraph Issue Type

[jira] [Created] (GIRAPH-1020) TaskInfo equality condition

Hassan Eslami created GIRAPH-1020: - Summary: TaskInfo equality condition Key: GIRAPH-1020 URL: https://issues.apache.org/jira/browse/GIRAPH-1020 Project: Giraph Issue Type: Bug

[jira] [Created] (GIRAPH-1019) Optimizing and debugging vertex mutation

Hassan Eslami created GIRAPH-1019: - Summary: Optimizing and debugging vertex mutation Key: GIRAPH-1019 URL: https://issues.apache.org/jira/browse/GIRAPH-1019 Project: Giraph Issue Type

[jira] [Created] (GIRAPH-1018) Improving PartitionStore API to better match its expected behaviour

Hassan Eslami created GIRAPH-1018: - Summary: Improving PartitionStore API to better match its expected behaviour Key: GIRAPH-1018 URL: https://issues.apache.org/jira/browse/GIRAPH-1018 Project

[jira] [Created] (GIRAPH-1009) Spammy 'lost reservation' messages from ZooKeeper in workers' log at the end of the computation.

Hassan Eslami created GIRAPH-1009: - Summary: Spammy 'lost reservation' messages from ZooKeeper in workers' log at the end of the computation. Key: GIRAPH-1009 URL: https://issues.apache.org/jir