Re: NegativeArraySizeException with large dataset

2014-09-09 Thread Andrew Munsell
Thanks for your help! I'll try it out. -- Andrew On Tue, Sep 9, 2014, at 10:25 AM, Pavan Kumar A wrote: yes, you should implement your own edge store. please take a look at ByteArrayEdges for example and modify it to use BigDataOutput & BigDataInput instead of ExtendedByteArrayOutput/Inpu

RE: NegativeArraySizeException with large dataset

2014-09-09 Thread Pavan Kumar A
yes, you should implement your own edge store.please take a look at ByteArrayEdges for example and modify it to use BigDataOutput & BigDataInput instead of ExtendedByteArrayOutput/Input. From: and...@wizardapps.net To: user@giraph.apache.org Subject: Re: NegativeArraySizeException with large dat

Re: NegativeArraySizeException with large dataset

2014-09-09 Thread Andrew Munsell
Great, thanks for pointing me in the right direction. All of the edge values are strings (in a Text object) and point to and from vertices with Text IDs, but none of the values should be greater than 60 bytes or so during the loading step. The size will increase during computation because I am modi