[ https://issues.apache.org/jira/browse/HBASE-14598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-14598: ----------------------------------- Fix Version/s: 0.98.16 1.0.3 This applies cleanly to 0.98 and to branch-1.0 with a small fixup needed in ByteBufferOutputStream. Picked back to both. I checked that the complete unit test suite passes on the respective branches locally before pushing. > ByteBufferOutputStream grows its HeapByteBuffer beyond JVM limitations > ---------------------------------------------------------------------- > > Key: HBASE-14598 > URL: https://issues.apache.org/jira/browse/HBASE-14598 > Project: HBase > Issue Type: Bug > Affects Versions: 0.98.12 > Reporter: Ian Friedman > Assignee: Ian Friedman > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16 > > Attachments: 14598.txt, hbase-14598-v1 (1).patch, > hbase-14598-v1.patch, hbase-14598-v1.patch, hbase-14598-v1.patch > > > We noticed that in returning a Scan against a region containing particularly > large (wide) rows that it is possible during > ByteBufferOutputStream.checkSizeAndGrow() to attempt to create a new > ByteBuffer larger than the JVM allows which then throws a OutOfMemoryError. > The code currently caps it at Integer.MAX_VALUE which is actually larger than > the JVM allows. This lead to us dealing with cascading region server death as > the RegionServer hosting the region died, opened on a new server, the client > retried the scan, and the new RS died as well. > I believe ByteBufferOutputStream should not try to create ByteBuffers that > large and instead throw an exception back up if it needs to grow any bigger. > The limit should probably be something like Integer.MAX_VALUE-8, as that is > what ArrayList uses. ref: > http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/util/ArrayList.java#221 -- This message was sent by Atlassian JIRA (v6.3.4#6332)