Re: Large Data Size in Row or Value?

2013-04-01 Thread Eric Newton
"What is the largest size that seems to work?" Tablet servers have been run in 64M JVMs without a problem, so long as there isn't any other pressure to swap that memory out (such as large map/reduce jobs). Since we've been keeping the New Generation size down ("-XX:NewSize=500m -XX:MaxNewSize=500

Re: Large Data Size in Row or Value?

2013-04-01 Thread Chris Sigman
I don't know that having the values being 128M chunks would make much difference if you still need to reassemble the chunk at a later time. The data is going to be stored in chunks smaller than that (unless the size of the data when its stored in HDFS is less than the block size), meaning that you'

Re: Large Data Size in Row or Value?

2013-04-01 Thread Josh Elser
Ignoring the actual size constraint necessary (I'm not entirely sure how that all adds up; it would be affected by concurrent query load and many other things), placing the large chunk into the Key will affect the size of the index inside of RFile (the file construct actually backing the data i

Large Data Size in Row or Value?

2013-04-01 Thread David Medinets
I have a chunk of data (let's say 400M) that I want to store in Accumulo. I can store the chunk in the ColumnFamily or in the Value. Does it make any difference to Accumulo which is used? My tserver is setup to use -Xmx3g. What is the largest size that seems to work? I have much more that I can a