[ https://issues.apache.org/jira/browse/TEZ-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hitesh Shah updated TEZ-2256: ----------------------------- Assignee: Cyrille Chépélov > Avoid use of BufferTooSmallException to signal end of buffer in > UnorderedPartitionedKVWriter > -------------------------------------------------------------------------------------------- > > Key: TEZ-2256 > URL: https://issues.apache.org/jira/browse/TEZ-2256 > Project: Apache Tez > Issue Type: Improvement > Affects Versions: 0.6.0, 0.7.0 > Reporter: Cyrille Chépélov > Assignee: Cyrille Chépélov > Priority: Minor > Original Estimate: 6h > Remaining Estimate: 6h > > UnorderedPartitionedKVWriter delegates serialization to the application, > passing it a private ByteArrayOutputStream. In case the buffer is exhausted, > ByteArrayOutputStream signals that with a private BufferTooSmallException, > which can be seen but not dealt with by the application. As [~cwensel] > pointed out, when the application is in fact a complex framework, there is no > way to distinguish this exception from a real failure, which compels logging > the full stack even for reasonable events such as "buffer complete". > Suggested approach: set a "complete" flag in ByteArrayOutputStream that > disables any further output, and replace BufferTooSmallException (BTSE) > handling by checking that flag. > [~sseth] suggested checking out SortedOutput as well, as the mechanisms there > should be similar. > I'll give this a go this week. -- This message was sent by Atlassian JIRA (v6.3.4#6332)