[
https://issues.apache.org/jira/browse/AVRO-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795394#action_12795394
]
Todd Lipcon commented on AVRO-266:
----------------------------------
bq. I'm not sure that you'd actually want object-reuse, since you might be
carrying around that 4MB payload in one of those records for way longer than
you need to
Good point about leakage here - reminds me of the recent Hadoop JIRA with the
IPC buffers never shrinking.
I think the solution here is that we should document the reuse behavior, and
suggest that, if the payloads are large and memory is tight, they should not
reuse objects. Passing null into the deserialization calls allocates new
objects, currently, and we should leave that option.
> Union as java.lang.Object prevents object reuse
> -----------------------------------------------
>
> Key: AVRO-266
> URL: https://issues.apache.org/jira/browse/AVRO-266
> Project: Avro
> Issue Type: Improvement
> Affects Versions: 1.2.0
> Reporter: Todd Lipcon
>
> Because Unions end up deserializing as java.lang.Object and using the
> object's type to differentiate the union constituents, object reuse is pretty
> hard to achieve. I don't have a specific benchmark, but I think this will
> hurt performance for logging applications where every record in a large file
> is a union, and the type tends to change for each record.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
