[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327060#comment-14327060 ]
Lars Hofhansl commented on HBASE-11544: --------------------------------------- Excellent. Minor comments: bq. When I changed the default value of caching to Integer.MAX_VALUE I was running into OOME on the server since caching is used to presize the ArrayList that holds results. Meh :) Sorry I missed that when I did the simple change in HBASE-12976. Yeah, simple solution is good. I'd actually like to change that size in all branches now, even before we finish this. Any objections? Quick note on small scans: They do not necessarily finish in one RPC. The difference is that they do not hold any state on the server, which means each RPC needs to re-setup the scanner stack, which is expensive, they also do p-reads by default. But you're absolutely right, when need partial results you sure-as-hell do not want to use a small scan. This: "The setting of Scan#setMaxResultSize will now operate at the cell level rather than the row level. " does not compute with this: "whereas the server will only ever see partial Results for very large rows." Is the limit per cell or per row? > [Ergonomics] hbase.client.scanner.caching is dogged and will try to return > batch even if it means OOME > ------------------------------------------------------------------------------------------------------ > > Key: HBASE-11544 > URL: https://issues.apache.org/jira/browse/HBASE-11544 > Project: HBase > Issue Type: Bug > Reporter: stack > Assignee: Jonathan Lawlor > Priority: Critical > Labels: beginner > Attachments: HBASE-11544-v1.patch > > > Running some tests, I set hbase.client.scanner.caching=1000. Dataset has > large cells. I kept OOME'ing. > Serverside, we should measure how much we've accumulated and return to the > client whatever we've gathered once we pass out a certain size threshold > rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)