[
https://issues.apache.org/jira/browse/PIG-5400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058229#comment-17058229
]
Koji Noguchi commented on PIG-5400:
-----------------------------------
I believe this feature(\?) of dropping struct(tuple) layer was added in
PIG-3294 when common logic in OrcStorage and HiveUDF were combined into
{{HiveUtils.}} and there was a section of code passing array(bag) of primitive
types to HiveUDF in which Pig had to pass with Bag(Tuple(Primitive_types)).
This had a side-effect of making the OrcStorage do the same when storing
single field Tuples.
> OrcStorage dropping struct(tuple) when it only holds a single field
> -------------------------------------------------------------------
>
> Key: PIG-5400
> URL: https://issues.apache.org/jira/browse/PIG-5400
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Priority: Minor
>
> I was asked by a user that they were seeing inconsistent schema when stored
> on OrcStorage. Sample code
> {code}
> A = load 'input.txt' as (a0:long);
> B = GROUP A by a0;
> STORE B into 'filename' using OrcStorage();
> {code}
> Pig's schema {{B: {group: long,A: bag: { tuple(a0: long)}}}}.
> Expected Orc schema {{struct<group:bigint,A:array<struct<bigint>>>}}
> Actual Orc schema {{struct<group:bigint,A:array<bigint>>}}
> _This only happens when a tuple contains a single field._
> Current schema without struct(tuple) is better in saving space but it would
> be nice to have an option to keep the extra struct(tuple) layer if user
> expects schema evolution within that tuple by adding more fields in the
> future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
