Hurray! Thanks for following up, I really appreciate it. The upload speed should be better too.
I should some time to write this all down in a more readable format for the reference guide on the website. J-D On Tue, Feb 7, 2012 at 5:13 AM, Bryan Keller <[email protected]> wrote: > Just to follow up, this did indeed fix the problem I was having with hitting > the xceiver limit. Thanks a bunch for the help, I have a much better > understanding of how heap, memstore size, and number of regions all play a > role in performance and resource usage. > > On Feb 6, 2012, at 5:03 PM, Jean-Daniel Cryans wrote: > >> On Mon, Feb 6, 2012 at 4:47 PM, Bryan Keller <[email protected]> wrote: >>> I increased the max region file size to 4gb so I should have fewer than 200 >>> regions per node now, more like 25. With 2 column families that will be 50 >>> memstores per node. 5.6gb would then flush files of 112mb. Still not close >>> to the memstore limit but shouldn't I be much better off than before? >> >> Ah sorry I didn't understand that you were restarting with a new >> table. In that case go with even fewer regions and if you aren't >> relying on the block cache for anything fast reads then you could also >> expend the memstore global usage. >> >>> Inserting sequentially may or may not be an option for me. I am storing a >>> live feed of data from an external source so it could prove tricky. >> >> Thought so. >> >> J-D >
