[
https://issues.apache.org/jira/browse/CRUNCH-90?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabriel Reid updated CRUNCH-90:
-------------------------------
Attachment: CRUNCH-90-reflect.patch
Thanks for figuring that out Josh! And sorry for getting you to do all my
monkey work for me, I feel pretty guilty about that now (but it's definitely
extra motivation for me to get my act together in terms of figuring Scala out).
I reworked the patch a bit to pass the configuration in via the
PType#initialize method instead of PType#getDetachedValue. My dream (or at
least my intention) is to have the PType get initialized and automatically made
available within DoFns. Sound good to you?
> Object reuse is not accounted for in mapper fusion
> --------------------------------------------------
>
> Key: CRUNCH-90
> URL: https://issues.apache.org/jira/browse/CRUNCH-90
> Project: Crunch
> Issue Type: Bug
> Reporter: Gabriel Reid
> Assignee: Gabriel Reid
> Fix For: 0.4.0
>
> Attachments: CRUNCH-90.patch, CRUNCH-90-reflect.patch,
> CRUNCH-90-reflect.patch
>
>
> When multiple DoFns are run over the same output (i.e. in the case of mapper
> fusion), the same value object is passed to multiple underlying DoFns. If the
> state of that value object is changed by one DoFn, other DoFns are called
> with the updated object.
> This is a situation that can happen quite easily when the input of a DoFn is
> simply updated and then emitted. In general, this bug will only affect values
> whose type is the same as the underlying serialization type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
