[
https://issues.apache.org/jira/browse/ACCUMULO-4708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ACCUMULO-4708:
-------------------------------------
Labels: pull-request-available (was: )
> Limit RFile block size to 2GB
> -----------------------------
>
> Key: ACCUMULO-4708
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4708
> Project: Accumulo
> Issue Type: Bug
> Components: core
> Reporter: Nick Felts
> Labels: pull-request-available
>
> In core/file/rfile/bcfile/BCFile.java, the block size is determined by the
> size of a DataOutputStream, which returns an int. The javadoc for size()
> states "If the counter overflows, it will be wrapped to Integer.MAX_VALUE."
> This can be a problem when RFiles are saving block regions, because Accumulo
> has no way of knowing how large the block actually is when the size is
> Integer.MAX_VALUE.
> To fix this, a check can be put in to throw an exception if Integer.MAX_VALUE
> bytes (or more) have been written. A check can also be made when appending
> to the block to make sure the key/value won't create a block that is too
> large.
> A check should also be made when reading in TABLE_FILE_COMPRESSED_BLOCK_SIZE
> and TABLE_FILE_COMPRESSED_BLOCK_SIZE_INDEX to make sure nobody mistakenly
> configures larger than Integer.MAX_VALUE.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
