[
https://issues.apache.org/jira/browse/PHOENIX-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831897#comment-15831897
]
Josh Mahonin commented on PHOENIX-3600:
---------------------------------------
Thanks for the review [~elserj]
- Will clean up the minor issues. My original intent was to do a
mostly-verbatim copy of the (presumably) vetted code in phoenix-hive.
- Good catch re: null {{regionLocation}} . I suspect safer behaviour is to just
return an empty array if regionLocation is null.
- Will update docs with new 'phoenix.mapreduce.split.by.stats' parameter
> Core MapReduce classes don't provide location info
> --------------------------------------------------
>
> Key: PHOENIX-3600
> URL: https://issues.apache.org/jira/browse/PHOENIX-3600
> Project: Phoenix
> Issue Type: Improvement
> Affects Versions: 4.8.0
> Reporter: Josh Mahonin
> Assignee: Josh Mahonin
> Attachments: PHOENIX-3600.patch
>
>
> The core MapReduce classes {{org.apache.phoenix.mapreduce.PhoenixInputSplit}}
> and {{org.apache.phoenix.mapreduce.PhoenixInputFormat}} don't provide region
> size or location information, leaving the execution engine (MR, Spark, etc.)
> to randomly assign splits to nodes.
> Interestingly, the phoenix-hive module has reimplemented these classes,
> including the node-aware functionality. We should port a subset of those
> changes back to the core code so that other engines can make use of them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
