[
https://issues.apache.org/jira/browse/CRUNCH-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572370#comment-13572370
]
Josh Wills commented on CRUNCH-162:
-----------------------------------
Delegate the sort-post function to PTable.keys, that one seems easy. I'd make
your "sort-pre" a distinct library function and not worry about the existing
sort-pre impl, since we have plans to (eventually) rewrite the sort stuff to
use a custom partitioner to support using multiple reducers. The trickier issue
seems to be what to name this functionality-- should it have its own lib.*
class, or should it drop in to one of the existing classes?
> Add utility function for merging output by identity reduce
> ----------------------------------------------------------
>
> Key: CRUNCH-162
> URL: https://issues.apache.org/jira/browse/CRUNCH-162
> Project: Crunch
> Issue Type: Improvement
> Components: MapReduce Patterns
> Affects Versions: 0.4.0
> Reporter: Dave Beech
> Priority: Minor
>
> Something I find myself doing reasonably often in mapreduce is to use
> the reduce step as nothing more than a means to merge data into larger
> files (using the identity reducer).
> There doesn't appear to be a neat way to do this with Crunch at the moment.
> Ref:
> http://mail-archives.apache.org/mod_mbox/incubator-crunch-user/201302.mbox/%3CCAFZSZPsXRxWT45c9w4ef7Ruij2exE4HP2CDNMjd%2BVc%3D9RWX-Jw%40mail.gmail.com%3E
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
