Your table is spooled to disk once a memory threshold is exceeded. The data will be spooled up to a configurable max and then the query would fail. We have a JIRA to instead pull the data over from the server as needed and it'd be great to have a volunteer to implement it: https://issues.apache.org/jira/browse/PHOENIX-539
Thanks, James On Tue, May 6, 2014 at 8:53 PM, anil gupta <[email protected]> wrote: > Thanks for the response. > Why do we need to store this data on disk? In some environments this is > undesirable because of security constraints/reasons. > If we do "select * from test_table;" then i am assuming that we are not > trying to pull the entire table at client side. We will just show first > 50-100 rows(1 page) and keep on serving subsequent pages on next page > requests. > Note: I was using sqlline to do that query. In Squirell, i think by default > for every query limit is set to 100. > > > On Tue, May 6, 2014 at 12:10 PM, James Taylor <[email protected] > >wrote: > > > Yes, and just to add to that, those files should be deleted when the scan > > is closed. If that's not the case, please file a JIRA. > > Thanks, > > James > > > > > > On Tue, May 6, 2014 at 12:08 AM, Gabriel Reid <[email protected] > > >wrote: > > > > > Yes, Phoenix writes the results of scanners to an in-memory store that > > > spools into file storage once it gets too big (see [1]). > > > > > > 1. > > > > > > https://github.com/apache/incubator-phoenix/blob/c22539d940fa0afc790cc0f25c2cdbbb255f80a3/phoenix-core/src/main/java/org/apache/phoenix/iterate/SpoolingResultIterator.java#L95 > > > > > > On Sun, May 4, 2014 at 8:19 AM, anil gupta <[email protected]> > > wrote: > > > > Hi, > > > > > > > > I would like to know why phoenix creates "ResultSpoolerxxxxxxxx.bin" > > > file? > > > > What is the content of these? > > > > > > > > I was using sqlline with phoenix3.0 and ran a "select * from table" > > > query. > > > > That table had around 10 million rows. Is phoenix really trying to > read > > > > entire table? > > > > > > > > Eventually this exception was thrown by phoenix: > > > > java.lang.Runtime Exception: > > > > org.apache.phoenix.exception.PhoenixIOException: > > > > org.apache.phoenix.exception.PhoenixIOException: No space left on > > device > > > > at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2440) > > > > at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2074) > > > > at sqlline.SqlLine.print(SqlLine.java:1735) > > > > at sqlline.SqlLine$Commands.execute(SqlLine.java:3683) > > > > at sqlline.SqlLine$Commands.sql(SqlLine.java:3584) > > > > at sqlline.SqlLine.dispatch(SqlLine.java:821) > > > > at sqlline.SqlLine.begin(SqlLine.java:699) > > > > at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:441) > > > > at sqlline.SqlLine.main(SqlLine.java:424) > > > > > > > > -- > > > > Thanks & Regards, > > > > Anil Gupta > > > > > > > > > -- > Thanks & Regards, > Anil Gupta >
