On Sat, Dec 19, 2009 at 8:07 PM, Adam Silberstein <[email protected]>wrote:
> ... > -My bail out code should keep me from going to the bitter end. Yes. > I tried > bailing out at different numbers (2, 100). I'm seeing way better > throughput. Yeah. You want to balance rpc calls and over-fetching. > CPU still strikes me as high, but maybe I'm underestimating > what's going on under the covers. We're probably heavy-weight here. We're doing a bit of fancy footwork so we ride the scanner across regions and so we do right thing regards caching. > I have an 8-core machine. With 10 > parallel clients doing 100 record scans, usage across each core adds up to > 60-70% Does that seem reasonable? > I'm not sure. Can you hang a profiler on it to see where the CPU is being chewed up? Result is backed by a raw list of KeyValues. You have access to it even. I'd guess that working w/ the raw list, you'd save CPU but not sure you can do that via Scanner#next (maybe someone else knows better). Go easy, St.Ack
