[
https://issues.apache.org/jira/browse/CRUNCH-455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093121#comment-14093121
]
Gabriel Reid commented on CRUNCH-455:
-------------------------------------
The removal of the re-use of the single AvroKey instance AvroKeyConverter
worries me a (really tiny) little bit, but that's just me fearing the unknown.
I'd love to try some benchmarking to see if this makes a real performance
difference at all -- I'm pretty convinced it doesn't, but the fact that this
stuff is done everywhere in MR code makes me wonder.
I was looking at Shuffler to see if there's any other way to do it, but the
only thing I see is having a special case for handling Avro where we make a
copy of the AvroKey object before putting it in the map, which wouldn't be too
cool either.
> Sort.sort doesn't work with ReverseAvroComparator in MemPipeline
> ----------------------------------------------------------------
>
> Key: CRUNCH-455
> URL: https://issues.apache.org/jira/browse/CRUNCH-455
> Project: Crunch
> Issue Type: Bug
> Components: Core
> Reporter: David Whiting
> Assignee: Josh Wills
> Priority: Minor
> Attachments: CRUNCH-455.patch
>
>
> The mem Shuffler class discards the config that arrives with the
> GroupingOptions and only uses the unmodified Conifguration from the pipeline
> object, which means that "crunch.schema" is not set and causes a
> NullPointerException when you try and execute it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
