Thanks Samarth and Jesse.
So, in effect setting the batch size (say, stmt.setFetchSize()) ensures that
only that much data is copied over the wire en-mass? And 'behind-the-scenes',
Phoenix driver would fetch next batch as each fetch size is exhausted?
Thanks,Sumit
From: Samarth Jain <[email protected]>
To: "[email protected]" <[email protected]>
Cc: Sumit Nigam <[email protected]>
Sent: Tuesday, October 6, 2015 9:20 PM
Subject: Re: ResultSet size
To add to what Jesse said, you can override the default scanner fetch size
programmatically via Phoenix by calling statement.setFetchSize(int).
On Tuesday, October 6, 2015, Jesse Yates <[email protected]> wrote:
So HBase (and by extension, Phoenix) does not do true "streaming" of rows -
rows are copied into memory from the HFiles and then eventually copied en-mass
onto the wire. On the client they are pulled off in chunks and paged through by
the client scanner. You can control the batch size (amount of rows in each
'page') via the usual HBase client configurations
On Tue, Oct 6, 2015 at 8:38 AM Sumit Nigam <[email protected]> wrote:
Hi,
Does Phoenix buffer the result set internally? I mean when I fire a huge skip
scan IN clause, then the data being returned may be too huge to contain in
memory. So, ideally I'd like to stream data through the resultset.next()
method. So, my question is does Phoenix really stream results?
And if so, is there a way to control how much is loads in one time in client
side before its next() fetches next batch of data from region servers to client?
Best regards,Sumit
