[
https://issues.apache.org/jira/browse/HIVE-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13726507#comment-13726507
]
Brock Noland commented on HIVE-4838:
------------------------------------
Hey can you explain a little bit more? We aren't writing out the metadata
per-key or anything like that, we are passing the metadata down into new
read/write methods. AFAICT the current approach did the static stuff because
they were using the Externalizable interface which didn't not allow any
push-down metadata during seralization or deserialization. If you look at
MapJoinTableContainer read and write in the new patch you'll see us pushing the
metadata (called context) down into the *new* read/write methods and the
corresponding read/write methods are not serializing that metadata.
> Refactor MapJoin HashMap code to improve testability and readability
> --------------------------------------------------------------------
>
> Key: HIVE-4838
> URL: https://issues.apache.org/jira/browse/HIVE-4838
> Project: Hive
> Issue Type: Bug
> Reporter: Brock Noland
> Assignee: Brock Noland
> Attachments: HIVE-4838.patch, HIVE-4838.patch, HIVE-4838.patch,
> HIVE-4838.patch, HIVE-4838.patch
>
>
> MapJoin is an essential component for high performance joins in Hive and the
> current code has done great service for many years. However, the code is
> showing it's age and currently suffers from the following issues:
> * Uses static state via the MapJoinMetaData class to pass serialization
> metadata to the Key, Row classes.
> * The api of a logical "Table Container" is not defined and therefore it's
> unclear what apis HashMapWrapper
> needs to publicize. Additionally HashMapWrapper has many used public methods.
> * HashMapWrapper contains logic to serialize, test memory bounds, and
> implement the table container. Ideally these logical units could be seperated
> * HashTableSinkObjectCtx has unused fields and unused methods
> * CommonJoinOperator and children use ArrayList on left hand side when only
> List is required
> * There are unused classes MRU, DCLLItemm and classes which duplicate
> functionality MapJoinSingleKey and MapJoinDoubleKeys
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
