[
https://issues.apache.org/jira/browse/CRUNCH-455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093231#comment-14093231
]
Gabriel Reid commented on CRUNCH-455:
-------------------------------------
I just ran a little benchmark with the before and after version of this patch
on a 5-node cluster with a word count on 2 billion records, and there was no
time difference between the two. Actually, thinking about it a bit more, this
makes more sense than I thought it would, as the objects being created and
discarded (AvroKeys) are super lightweight and just a wrapper around the real
value.
Anyhow, I'm now officially not worried about object reuse and +1 on this patch.
> Sort.sort doesn't work with ReverseAvroComparator in MemPipeline
> ----------------------------------------------------------------
>
> Key: CRUNCH-455
> URL: https://issues.apache.org/jira/browse/CRUNCH-455
> Project: Crunch
> Issue Type: Bug
> Components: Core
> Reporter: David Whiting
> Assignee: Josh Wills
> Priority: Minor
> Attachments: CRUNCH-455.patch
>
>
> The mem Shuffler class discards the config that arrives with the
> GroupingOptions and only uses the unmodified Conifguration from the pipeline
> object, which means that "crunch.schema" is not set and causes a
> NullPointerException when you try and execute it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
