[
https://issues.apache.org/jira/browse/PIG-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159696#comment-13159696
]
Scott Carey commented on PIG-2359:
----------------------------------
Yes, using something like ASM (http://asm.ow2.org/) to fill out the fields and
get() methods by extending an abstract class that is written in Java and has
most of the work already done.
A custom TupleFactory could have a static ThreadLocal<WeakHashMap<Schema,
TupleBuilder>> or something similar, where the TupleBuilder and the Tuple it
creates is created by dynamic class generation.
The resulting objects would be memory compact and very fast. I am tempted to
try and implement it if I have some spare time. I have been thinking about
something similar for Avro, but the use case for Pig is significantly simpler.
It would re-use the other changes above to the Tuple contract -- getLong()
getInteger() etc. I really like the idea of moving Tuple to have intrinsic
getters. There are some questions I have about the contract however -- what
should be done if getLong is called for a field index containg an Integer?
promote? Should it follow the rules in the Pig specification for type casts or
throw an exception?
> Support more efficient Tuples when schemas are known
> ----------------------------------------------------
>
> Key: PIG-2359
> URL: https://issues.apache.org/jira/browse/PIG-2359
> Project: Pig
> Issue Type: New Feature
> Reporter: Dmitriy V. Ryaboy
> Assignee: Dmitriy V. Ryaboy
> Attachments: PIG-2359.1.patch
>
>
> Pig Tuples have significant overhead due to the fact that all the fields are
> Objects.
> When a Tuple only contains primitive fields (ints, longs, etc), it's possible
> to avoid this overhead, which would result in significant memory savings.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
