Re: Out of core computation fails with KryoException: Buffer underflow

[Denis Dudinski]

Hi Hassan,

I found the source of problem with KryoException: BufferUnderflow.

There is a code snippet In metod
org.apache.giraph.ooc.data.DiskBackedPartitionStore#loadInMemoryPartitionData:

for (int i = 0; i < numVertices; ++i) {
readOutEdges(dataInput, partition);
  }

Variable numVertices is set to a number of vertices stored in
"vertices storage file". It implies that if a value of vertices in
"vertices storage file" is greater than a value of vertices stored in
"edges storage file" then the buffer underflow exception happens
during readOutEdges() execution.

It can happen (and happens in our case) when a graph is only "static"
after step 0 (steps 1, 2, 3 etc.), but not in step 0 when "missing"
vertices are created via messages sent from vertices loaded during
step -1 (all such vertices have empty OutEdges and may not be
considered as explicit graph mutations).

I added writing of number of vertices to edges file on first
offloading and reading this value on subsequent loadings and it
helped.

However, I think that the problem with "isStaticGraph" lies within the
definition of "static" graph: can such graph be changed during step 0
due to messages to absent vertices or should it be loaded as a whole
during step -1 and be never changed during any other step?

Best Regards,
Denis Dudinski

2016-11-08 20:25 GMT+03:00 Hassan Eslami :
> Hi Denis,
>
> Thanks for trying this out without the "isStaticGraph" option. I'll soon
> create a JIRA and fix this issue.
>
> It is worth mentioning that the new OOC is designed in a way that it works
> even with a very low available memory. But, there are subtleties that should
> be considered. First, the Giraph itself (book-keeping, ZooKeeper, and other
> stuff not related to the graph structure or messages) would take about 2-3GB
> of space. Second, I strongly encourage you to use ParallelGC and set the
> OldGen size (tenured memory generation) to be about 90% of the available
> memory. Third, depending on the OldGen size, you would want to change some
> of the OOC parameters to have it working in a robust way. Let me give you an
> example. Let's say you have a 16GB-container (you may want to have about 10%
> more memory available on your system to accommodate JVM and off-heap message
> data). Let's say you set aside 2GB for the NewGen and 14GB for old gen.
> Assuming Giraph would take 3GB of data just for the non-graph-relevant
> book-keeping, that gives you 11GB of space only for graph structure. That
> means the job would fail if the amount of graph data in memory passes about
> 11GB or so!
>
> If you look at the ThresholdBasedOracle used as a default for the new OOC
> design, there is a flag called "giraph.memory.failPressure." You would want
> to set this flag to be 11/16 (or even slightly less) and scale the rest of
> the thresholds accordingly. So, in the example I just mentioned, here are
> the values you should set the flags:
> giraph.memory.failPressure=0.65
> giraph.memory.emergencyPressure=0.625
> giraph.memory.highPressure=0.6
> giraph.memory.optimalPressure=0.55
> giraph.memory.lowPressure=0.5
>
> You only need to tune these flags once for a given amount of memory in a
> container. Once tuned, the same set of thresholds should work for various
> applications.
>
> I hope it helps.
> Best,
> Hassan
>
> On Tue, Nov 8, 2016 at 4:25 AM, Denis Dudinski 
> wrote:
>>
>> Hi Hassan,
>>
>> Thank you for really quick response!
>>
>> I changed "giraph.isStaticGraph" to false and the error disappeared.
>> As expected iteration became slow and wrote to disk edges once again
>> in superstep 1.
>>
>> However, the computation failed at superstep 2 with error
>> "java.lang.OutOfMemoryError: GC overhead limit exceeded". It seems to
>> be unrelated to "isStaticGraph" issue, but I think it worth mentioning
>> to see the picture as a whole.
>>
>> Are there any other tests/information I am able to execute/check to
>> help to pinpoint "isStaticGraph" problem?
>>
>> Best Regards,
>> Denis Dudinski
>>
>>
>> 2016-11-07 20:00 GMT+03:00 Hassan Eslami :
>> > Hi Denis,
>> >
>> > Thanks for bringing up the issue. In the previous conversation thread,
>> > the
>> > similar problem is reported even with a simpler example connected
>> > component
>> > calculation. Although, back then, we were developing other
>> > performance-critical components of OOC.
>> >
>> > Let's debug this issue together to make the new OOC more stable. I
>> > suspect
>> > the problem is with "giraph.isStaticGraph=true" (as this is only an
>> > optimization and most of our end-to-end testing was on cases where the
>> > graph
>> > could change). Let's get rid of it for now and see if the problem still
>> > exists.
>> >
>> > Best,
>> > Hassan
>> >
>> > On Mon, Nov 7, 2016 at 6:24 AM, Denis Dudinski
>> > 
>> > wrote:
>> >>
>> >> Hello,
>> >>
>> >> We are trying to calculate PageRank on huge graph, which does not 

Re: Out of core computation fails with KryoException: Buffer underflow

[Denis Dudinski]

Hi Hassan,

Thank you for your help!

I'll try to tune GC as you suggested and play with different
ThresholdBasedOracle settings.

As for now I was able to run computation without OutOfMemory but with
really strict option giraph.maxPartitionsInMemory=1 (haven't tried
other values since it is really tedious to tune them).

Best Regards,
Denis Dudinski

2016-11-08 20:25 GMT+03:00 Hassan Eslami :
> Hi Denis,
>
> Thanks for trying this out without the "isStaticGraph" option. I'll soon
> create a JIRA and fix this issue.
>
> It is worth mentioning that the new OOC is designed in a way that it works
> even with a very low available memory. But, there are subtleties that should
> be considered. First, the Giraph itself (book-keeping, ZooKeeper, and other
> stuff not related to the graph structure or messages) would take about 2-3GB
> of space. Second, I strongly encourage you to use ParallelGC and set the
> OldGen size (tenured memory generation) to be about 90% of the available
> memory. Third, depending on the OldGen size, you would want to change some
> of the OOC parameters to have it working in a robust way. Let me give you an
> example. Let's say you have a 16GB-container (you may want to have about 10%
> more memory available on your system to accommodate JVM and off-heap message
> data). Let's say you set aside 2GB for the NewGen and 14GB for old gen.
> Assuming Giraph would take 3GB of data just for the non-graph-relevant
> book-keeping, that gives you 11GB of space only for graph structure. That
> means the job would fail if the amount of graph data in memory passes about
> 11GB or so!
>
> If you look at the ThresholdBasedOracle used as a default for the new OOC
> design, there is a flag called "giraph.memory.failPressure." You would want
> to set this flag to be 11/16 (or even slightly less) and scale the rest of
> the thresholds accordingly. So, in the example I just mentioned, here are
> the values you should set the flags:
> giraph.memory.failPressure=0.65
> giraph.memory.emergencyPressure=0.625
> giraph.memory.highPressure=0.6
> giraph.memory.optimalPressure=0.55
> giraph.memory.lowPressure=0.5
>
> You only need to tune these flags once for a given amount of memory in a
> container. Once tuned, the same set of thresholds should work for various
> applications.
>
> I hope it helps.
> Best,
> Hassan
>
> On Tue, Nov 8, 2016 at 4:25 AM, Denis Dudinski 
> wrote:
>>
>> Hi Hassan,
>>
>> Thank you for really quick response!
>>
>> I changed "giraph.isStaticGraph" to false and the error disappeared.
>> As expected iteration became slow and wrote to disk edges once again
>> in superstep 1.
>>
>> However, the computation failed at superstep 2 with error
>> "java.lang.OutOfMemoryError: GC overhead limit exceeded". It seems to
>> be unrelated to "isStaticGraph" issue, but I think it worth mentioning
>> to see the picture as a whole.
>>
>> Are there any other tests/information I am able to execute/check to
>> help to pinpoint "isStaticGraph" problem?
>>
>> Best Regards,
>> Denis Dudinski
>>
>>
>> 2016-11-07 20:00 GMT+03:00 Hassan Eslami :
>> > Hi Denis,
>> >
>> > Thanks for bringing up the issue. In the previous conversation thread,
>> > the
>> > similar problem is reported even with a simpler example connected
>> > component
>> > calculation. Although, back then, we were developing other
>> > performance-critical components of OOC.
>> >
>> > Let's debug this issue together to make the new OOC more stable. I
>> > suspect
>> > the problem is with "giraph.isStaticGraph=true" (as this is only an
>> > optimization and most of our end-to-end testing was on cases where the
>> > graph
>> > could change). Let's get rid of it for now and see if the problem still
>> > exists.
>> >
>> > Best,
>> > Hassan
>> >
>> > On Mon, Nov 7, 2016 at 6:24 AM, Denis Dudinski
>> > 
>> > wrote:
>> >>
>> >> Hello,
>> >>
>> >> We are trying to calculate PageRank on huge graph, which does not fit
>> >> into memory. For calculation to succeed we tried to turn on OutOfCore
>> >> feature of Giraph, but every launch we tried resulted in
>> >> com.esotericsoftware.kryo.KryoException: Buffer underflow.
>> >> Each time it happens on different servers but exactly right after
>> >> start of superstep 1.
>> >>
>> >> We are using Giraph 1.2.0 on hadoop 2.7.3 (our prod version, can't
>> >> back-step to Giraph's officially supported version and had to patch
>> >> Giraph a little) placed on 11 servers + 3 master servers (namenodes
>> >> etc.) with separate ZooKeeper cluster deployment.
>> >>
>> >> Our launch command:
>> >>
>> >> hadoop jar /opt/giraph-1.2.0/pr-job-jar-with-dependencies.jar
>> >> org.apache.giraph.GiraphRunner com.prototype.di.pr.PageRankComputation
>> >> \
>> >> -mc com.prototype.di.pr.PageRankMasterCompute \
>> >> -yj pr-job-jar-with-dependencies.jar \
>> >> -vif com.belprime.di.pr.input.HBLongVertexInputFormat \
>> >> -vof 

Re: Out of core computation fails with KryoException: Buffer underflow

[Denis Dudinski]

Hi Xenia,

Thank you! I'll check the thread you mentioned.

Best Regards,
Denis Dudinski

2016-11-08 14:16 GMT+03:00 Xenia Demetriou :
> Hi Denis,
>
> For the "java.lang.OutOfMemoryError: GC overhead limit exceeded" error  I
> hope that the  conversation in below link can help you.
>  www.mail-archive.com/user@giraph.apache.org/msg02938.html
>
> Regards,
> Xenia
>
> 2016-11-08 12:25 GMT+02:00 Denis Dudinski :
>>
>> Hi Hassan,
>>
>> Thank you for really quick response!
>>
>> I changed "giraph.isStaticGraph" to false and the error disappeared.
>> As expected iteration became slow and wrote to disk edges once again
>> in superstep 1.
>>
>> However, the computation failed at superstep 2 with error
>> "java.lang.OutOfMemoryError: GC overhead limit exceeded". It seems to
>> be unrelated to "isStaticGraph" issue, but I think it worth mentioning
>> to see the picture as a whole.
>>
>> Are there any other tests/information I am able to execute/check to
>> help to pinpoint "isStaticGraph" problem?
>>
>> Best Regards,
>> Denis Dudinski
>>
>>
>> 2016-11-07 20:00 GMT+03:00 Hassan Eslami :
>> > Hi Denis,
>> >
>> > Thanks for bringing up the issue. In the previous conversation thread,
>> > the
>> > similar problem is reported even with a simpler example connected
>> > component
>> > calculation. Although, back then, we were developing other
>> > performance-critical components of OOC.
>> >
>> > Let's debug this issue together to make the new OOC more stable. I
>> > suspect
>> > the problem is with "giraph.isStaticGraph=true" (as this is only an
>> > optimization and most of our end-to-end testing was on cases where the
>> > graph
>> > could change). Let's get rid of it for now and see if the problem still
>> > exists.
>> >
>> > Best,
>> > Hassan
>> >
>> > On Mon, Nov 7, 2016 at 6:24 AM, Denis Dudinski
>> > 
>> > wrote:
>> >>
>> >> Hello,
>> >>
>> >> We are trying to calculate PageRank on huge graph, which does not fit
>> >> into memory. For calculation to succeed we tried to turn on OutOfCore
>> >> feature of Giraph, but every launch we tried resulted in
>> >> com.esotericsoftware.kryo.KryoException: Buffer underflow.
>> >> Each time it happens on different servers but exactly right after
>> >> start of superstep 1.
>> >>
>> >> We are using Giraph 1.2.0 on hadoop 2.7.3 (our prod version, can't
>> >> back-step to Giraph's officially supported version and had to patch
>> >> Giraph a little) placed on 11 servers + 3 master servers (namenodes
>> >> etc.) with separate ZooKeeper cluster deployment.
>> >>
>> >> Our launch command:
>> >>
>> >> hadoop jar /opt/giraph-1.2.0/pr-job-jar-with-dependencies.jar
>> >> org.apache.giraph.GiraphRunner com.prototype.di.pr.PageRankComputation
>> >> \
>> >> -mc com.prototype.di.pr.PageRankMasterCompute \
>> >> -yj pr-job-jar-with-dependencies.jar \
>> >> -vif com.belprime.di.pr.input.HBLongVertexInputFormat \
>> >> -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
>> >> -op /user/hadoop/output/pr_test \
>> >> -w 10 \
>> >> -c com.prototype.di.pr.PRDoubleCombiner \
>> >> -wc com.prototype.di.pr.PageRankWorkerContext \
>> >> -ca hbase.rootdir=hdfs://namenode1.webmeup.com:8020/hbase \
>> >> -ca giraph.logLevel=info \
>> >> -ca hbase.mapreduce.inputtable=di_test \
>> >> -ca hbase.mapreduce.scan.columns=di:n \
>> >> -ca hbase.defaults.for.version.skip=true \
>> >> -ca hbase.table.row.textkey=false \
>> >> -ca giraph.yarn.task.heap.mb=48000 \
>> >> -ca giraph.isStaticGraph=true \
>> >> -ca giraph.SplitMasterWorker=false \
>> >> -ca giraph.oneToAllMsgSending=true \
>> >> -ca giraph.metrics.enable=true \
>> >> -ca giraph.jmap.histo.enable=true \
>> >> -ca
>> >> giraph.vertexIdClass=com.prototype.di.pr.DomainPartAwareLongWritable \
>> >> -ca
>> >> giraph.outgoingMessageValueClass=org.apache.hadoop.io.DoubleWritable \
>> >> -ca giraph.inputOutEdgesClass=org.apache.giraph.edge.LongNullArrayEdges
>> >> \
>> >> -ca giraph.useOutOfCoreGraph=true \
>> >> -ca giraph.waitForPerWorkerRequests=true \
>> >> -ca giraph.maxNumberOfUnsentRequests=1000 \
>> >> -ca
>> >>
>> >> giraph.vertexInputFilterClass=com.prototype.di.pr.input.PagesFromSameDomainLimiter
>> >> \
>> >> -ca giraph.useInputSplitLocality=true \
>> >> -ca hbase.mapreduce.scan.cachedrows=1 \
>> >> -ca giraph.minPartitionsPerComputeThread=60 \
>> >> -ca
>> >>
>> >> giraph.graphPartitionerFactoryClass=com.prototype.di.pr.DomainAwareGraphPartitionerFactory
>> >> \
>> >> -ca giraph.numInputThreads=1 \
>> >> -ca giraph.inputSplitSamplePercent=20 \
>> >> -ca giraph.pr.maxNeighborsPerVertex=50 \
>> >> -ca
>> >> giraph.partitionClass=org.apache.giraph.partition.ByteArrayPartition \
>> >> -ca giraph.vertexClass=org.apache.giraph.graph.ByteValueVertex \
>> >> -ca
>> >>
>> >> giraph.partitionsDirectory=/disk1/_bsp/_partitions,/disk2/_bsp/_partitions
>> >>
>> >> Logs excerpt:
>> >>
>> >> 16/11/06 15:47:15 INFO pr.PageRankWorkerContext: Pre superstep in

Re: Out of core computation fails with KryoException: Buffer underflow

[Denis Dudinski]

Hi Hassan,

Thank you for really quick response!

I changed "giraph.isStaticGraph" to false and the error disappeared.
As expected iteration became slow and wrote to disk edges once again
in superstep 1.

However, the computation failed at superstep 2 with error
"java.lang.OutOfMemoryError: GC overhead limit exceeded". It seems to
be unrelated to "isStaticGraph" issue, but I think it worth mentioning
to see the picture as a whole.

Are there any other tests/information I am able to execute/check to
help to pinpoint "isStaticGraph" problem?

Best Regards,
Denis Dudinski


2016-11-07 20:00 GMT+03:00 Hassan Eslami :
> Hi Denis,
>
> Thanks for bringing up the issue. In the previous conversation thread, the
> similar problem is reported even with a simpler example connected component
> calculation. Although, back then, we were developing other
> performance-critical components of OOC.
>
> Let's debug this issue together to make the new OOC more stable. I suspect
> the problem is with "giraph.isStaticGraph=true" (as this is only an
> optimization and most of our end-to-end testing was on cases where the graph
> could change). Let's get rid of it for now and see if the problem still
> exists.
>
> Best,
> Hassan
>
> On Mon, Nov 7, 2016 at 6:24 AM, Denis Dudinski 
> wrote:
>>
>> Hello,
>>
>> We are trying to calculate PageRank on huge graph, which does not fit
>> into memory. For calculation to succeed we tried to turn on OutOfCore
>> feature of Giraph, but every launch we tried resulted in
>> com.esotericsoftware.kryo.KryoException: Buffer underflow.
>> Each time it happens on different servers but exactly right after
>> start of superstep 1.
>>
>> We are using Giraph 1.2.0 on hadoop 2.7.3 (our prod version, can't
>> back-step to Giraph's officially supported version and had to patch
>> Giraph a little) placed on 11 servers + 3 master servers (namenodes
>> etc.) with separate ZooKeeper cluster deployment.
>>
>> Our launch command:
>>
>> hadoop jar /opt/giraph-1.2.0/pr-job-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner com.prototype.di.pr.PageRankComputation
>> \
>> -mc com.prototype.di.pr.PageRankMasterCompute \
>> -yj pr-job-jar-with-dependencies.jar \
>> -vif com.belprime.di.pr.input.HBLongVertexInputFormat \
>> -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
>> -op /user/hadoop/output/pr_test \
>> -w 10 \
>> -c com.prototype.di.pr.PRDoubleCombiner \
>> -wc com.prototype.di.pr.PageRankWorkerContext \
>> -ca hbase.rootdir=hdfs://namenode1.webmeup.com:8020/hbase \
>> -ca giraph.logLevel=info \
>> -ca hbase.mapreduce.inputtable=di_test \
>> -ca hbase.mapreduce.scan.columns=di:n \
>> -ca hbase.defaults.for.version.skip=true \
>> -ca hbase.table.row.textkey=false \
>> -ca giraph.yarn.task.heap.mb=48000 \
>> -ca giraph.isStaticGraph=true \
>> -ca giraph.SplitMasterWorker=false \
>> -ca giraph.oneToAllMsgSending=true \
>> -ca giraph.metrics.enable=true \
>> -ca giraph.jmap.histo.enable=true \
>> -ca giraph.vertexIdClass=com.prototype.di.pr.DomainPartAwareLongWritable \
>> -ca giraph.outgoingMessageValueClass=org.apache.hadoop.io.DoubleWritable \
>> -ca giraph.inputOutEdgesClass=org.apache.giraph.edge.LongNullArrayEdges \
>> -ca giraph.useOutOfCoreGraph=true \
>> -ca giraph.waitForPerWorkerRequests=true \
>> -ca giraph.maxNumberOfUnsentRequests=1000 \
>> -ca
>> giraph.vertexInputFilterClass=com.prototype.di.pr.input.PagesFromSameDomainLimiter
>> \
>> -ca giraph.useInputSplitLocality=true \
>> -ca hbase.mapreduce.scan.cachedrows=1 \
>> -ca giraph.minPartitionsPerComputeThread=60 \
>> -ca
>> giraph.graphPartitionerFactoryClass=com.prototype.di.pr.DomainAwareGraphPartitionerFactory
>> \
>> -ca giraph.numInputThreads=1 \
>> -ca giraph.inputSplitSamplePercent=20 \
>> -ca giraph.pr.maxNeighborsPerVertex=50 \
>> -ca giraph.partitionClass=org.apache.giraph.partition.ByteArrayPartition \
>> -ca giraph.vertexClass=org.apache.giraph.graph.ByteValueVertex \
>> -ca
>> giraph.partitionsDirectory=/disk1/_bsp/_partitions,/disk2/_bsp/_partitions
>>
>> Logs excerpt:
>>
>> 16/11/06 15:47:15 INFO pr.PageRankWorkerContext: Pre superstep in worker
>> context
>> 16/11/06 15:47:15 INFO graph.GraphTaskManager: execute: 60 partitions
>> to process with 1 compute thread(s), originally 1 thread(s) on
>> superstep 1
>> 16/11/06 15:47:15 INFO ooc.OutOfCoreEngine: startIteration: with 60
>> partitions in memory and 1 active threads
>> 16/11/06 15:47:15 INFO pr.PageRankComputation: Pre superstep1 in PR
>> computation
>> 16/11/06 15:47:15 INFO policy.ThresholdBasedOracle: getNextIOActions:
>> usedMemoryFraction = 0.75
>> 16/11/06 15:47:16 INFO ooc.OutOfCoreEngine:
>> updateActiveThreadsFraction: updating the number of active threads to
>> 1
>> 16/11/06 15:47:16 INFO policy.ThresholdBasedOracle:
>> updateRequestsCredit: updating the credit to 20
>> 16/11/06 15:47:17 INFO graph.GraphTaskManager: installGCMonitoring:
>> name = PS Scavenge, action = end of minor GC, cause = 

Re: Out of core computation fails with KryoException: Buffer underflow

[Hassan Eslami]

Hi Denis,

Thanks for bringing up the issue. In the previous conversation thread, the
similar problem is reported even with a simpler example connected component
calculation. Although, back then, we were developing other
performance-critical components of OOC.

Let's debug this issue together to make the new OOC more stable. I suspect
the problem is with "giraph.isStaticGraph=true" (as this is only an
optimization and most of our end-to-end testing was on cases where the
graph could change). Let's get rid of it for now and see if the problem
still exists.

Best,
Hassan

On Mon, Nov 7, 2016 at 6:24 AM, Denis Dudinski 
wrote:

> Hello,
>
> We are trying to calculate PageRank on huge graph, which does not fit
> into memory. For calculation to succeed we tried to turn on OutOfCore
> feature of Giraph, but every launch we tried resulted in
> com.esotericsoftware.kryo.KryoException: Buffer underflow.
> Each time it happens on different servers but exactly right after
> start of superstep 1.
>
> We are using Giraph 1.2.0 on hadoop 2.7.3 (our prod version, can't
> back-step to Giraph's officially supported version and had to patch
> Giraph a little) placed on 11 servers + 3 master servers (namenodes
> etc.) with separate ZooKeeper cluster deployment.
>
> Our launch command:
>
> hadoop jar /opt/giraph-1.2.0/pr-job-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner com.prototype.di.pr.PageRankComputation
> \
> -mc com.prototype.di.pr.PageRankMasterCompute \
> -yj pr-job-jar-with-dependencies.jar \
> -vif com.belprime.di.pr.input.HBLongVertexInputFormat \
> -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat \
> -op /user/hadoop/output/pr_test \
> -w 10 \
> -c com.prototype.di.pr.PRDoubleCombiner \
> -wc com.prototype.di.pr.PageRankWorkerContext \
> -ca hbase.rootdir=hdfs://namenode1.webmeup.com:8020/hbase \
> -ca giraph.logLevel=info \
> -ca hbase.mapreduce.inputtable=di_test \
> -ca hbase.mapreduce.scan.columns=di:n \
> -ca hbase.defaults.for.version.skip=true \
> -ca hbase.table.row.textkey=false \
> -ca giraph.yarn.task.heap.mb=48000 \
> -ca giraph.isStaticGraph=true \
> -ca giraph.SplitMasterWorker=false \
> -ca giraph.oneToAllMsgSending=true \
> -ca giraph.metrics.enable=true \
> -ca giraph.jmap.histo.enable=true \
> -ca giraph.vertexIdClass=com.prototype.di.pr.DomainPartAwareLongWritable \
> -ca giraph.outgoingMessageValueClass=org.apache.hadoop.io.DoubleWritable \
> -ca giraph.inputOutEdgesClass=org.apache.giraph.edge.LongNullArrayEdges \
> -ca giraph.useOutOfCoreGraph=true \
> -ca giraph.waitForPerWorkerRequests=true \
> -ca giraph.maxNumberOfUnsentRequests=1000 \
> -ca giraph.vertexInputFilterClass=com.prototype.di.pr.input.
> PagesFromSameDomainLimiter
> \
> -ca giraph.useInputSplitLocality=true \
> -ca hbase.mapreduce.scan.cachedrows=1 \
> -ca giraph.minPartitionsPerComputeThread=60 \
> -ca giraph.graphPartitionerFactoryClass=com.prototype.di.pr.
> DomainAwareGraphPartitionerFactory
> \
> -ca giraph.numInputThreads=1 \
> -ca giraph.inputSplitSamplePercent=20 \
> -ca giraph.pr.maxNeighborsPerVertex=50 \
> -ca giraph.partitionClass=org.apache.giraph.partition.ByteArrayPartition \
> -ca giraph.vertexClass=org.apache.giraph.graph.ByteValueVertex \
> -ca giraph.partitionsDirectory=/disk1/_bsp/_partitions,/disk2/
> _bsp/_partitions
>
> Logs excerpt:
>
> 16/11/06 15:47:15 INFO pr.PageRankWorkerContext: Pre superstep in worker
> context
> 16/11/06 15:47:15 INFO graph.GraphTaskManager: execute: 60 partitions
> to process with 1 compute thread(s), originally 1 thread(s) on
> superstep 1
> 16/11/06 15:47:15 INFO ooc.OutOfCoreEngine: startIteration: with 60
> partitions in memory and 1 active threads
> 16/11/06 15:47:15 INFO pr.PageRankComputation: Pre superstep1 in PR
> computation
> 16/11/06 15:47:15 INFO policy.ThresholdBasedOracle: getNextIOActions:
> usedMemoryFraction = 0.75
> 16/11/06 15:47:16 INFO ooc.OutOfCoreEngine:
> updateActiveThreadsFraction: updating the number of active threads to
> 1
> 16/11/06 15:47:16 INFO policy.ThresholdBasedOracle:
> updateRequestsCredit: updating the credit to 20
> 16/11/06 15:47:17 INFO graph.GraphTaskManager: installGCMonitoring:
> name = PS Scavenge, action = end of minor GC, cause = Allocation
> Failure, duration = 937ms
> 16/11/06 15:47:17 INFO policy.ThresholdBasedOracle: getNextIOActions:
> usedMemoryFraction = 0.72
> 16/11/06 15:47:18 INFO policy.ThresholdBasedOracle: getNextIOActions:
> usedMemoryFraction = 0.74
> 16/11/06 15:47:18 INFO ooc.OutOfCoreEngine:
> updateActiveThreadsFraction: updating the number of active threads to
> 1
> 16/11/06 15:47:18 INFO policy.ThresholdBasedOracle:
> updateRequestsCredit: updating the credit to 20
> 16/11/06 15:47:19 INFO policy.ThresholdBasedOracle: getNextIOActions:
> usedMemoryFraction = 0.76
> 16/11/06 15:47:19 INFO ooc.OutOfCoreEngine: doneProcessingPartition:
> processing partition 234 is done!
> 16/11/06 15:47:20 INFO policy.ThresholdBasedOracle: getNextIOActions:
>