Re: Parallelizing TableSnapshotScanner

2016-10-20 Thread Randy Fox
I can work on that. Since it is reading the restore data from a snapshot it is definitely not changing. I wanted to make sure there was not some inherent thread model under the scanner that prohibited this from working. On 10/20/16, 8:30 AM, "Ted Yu" wrote: >Can you come up with unit test

Re: Parallelizing TableSnapshotScanner

2016-10-20 Thread Ted Yu
Can you come up with unit test that shows what you observed ? You may have seen the javadoc for ClientSideRegionScanner : * A client scanner for a region opened for read-only on the client side. Assumes region data * is not changing. Cheers On Thu, Oct 20, 2016 at 8:26 AM, Randy Fox wrote:

Parallelizing TableSnapshotScanner

2016-10-20 Thread Randy Fox
Hi, I have attempted to parallelize the TableSnapshotScanner by taking the ClientSideRegionScanner’s it generates and sending them each to their own thread to read the results. I am finding that I do not get all the rows per region when I do this. It seems the more parallelism the less data I