Re: how to use out of core options

2013-10-19 Thread Jianqiang Ou
t command are you using to run the > job? > > > On Fri, Oct 18, 2013 at 9:03 AM, Jianqiang Ou wrote: > >> Thanks, I just tried another dataset, which could be successfully handled >> by my cluster within memory. However, exceptions still occurred with the >> -Dgiraph.use

Re: how to use out of core options

2013-10-18 Thread Jianqiang Ou
le in memory, using giraph defeats the purpose in > this case. > > > > On Thu, Oct 17, 2013 at 8:13 AM, Jianqiang Ou wrote: > >> Thanks very much, so are you saying if I use Dgiraph.maxPartitionsInMemory >> and Dgiraph.maxMessagesInMemory to make them both smaller number,

Re: how to use out of core options

2013-10-17 Thread Jianqiang Ou
computations tend to send large messages > along. Hence, it depends on your algorithm what feature to rely on more." > > Thanks > Sundi > > > On Wed, Oct 16, 2013 at 9:41 PM, Jianqiang Ou wrote: > >> Hi Sundi, >> >> I just tried your method, but somehow the

Re: how to use out of core options

2013-10-16 Thread Jianqiang Ou
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/andy/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/andy/output/page3 -w 3 -mc org.apache.giraph.examples.SimplePageRankComputation\$SimplePageRankMasterCompute Many thanks, Jianqiang On Wed, Oct 16, 2013 at 12:11 PM, Jianqiang Ou

Re: how to use out of core options

2013-10-16 Thread Jianqiang Ou
reMessages=true > -Dgiraph.useOutOfCoreGraph=true ... > > > > > On Wed, Oct 16, 2013 at 7:29 AM, Jianqiang Ou wrote: > >> Hi I have a question about the out of core giraph. It is said that, in >> order to use disk to store the partions, we need to use " >>

how to use out of core options

2013-10-16 Thread Jianqiang Ou
Hi I have a question about the out of core giraph. It is said that, in order to use disk to store the partions, we need to use " giraph.useOutOfCoreGraph=true", but where should I put this statement to? BTW, I am just trying to use the pagerank or shortestpath example to test the out of core perfo