Daniel Dai created PIG-4130:
-------------------------------
Summary: Store/Load the same file fails for
AvroStorage/OrcStorage, etc
Key: PIG-4130
URL: https://issues.apache.org/jira/browse/PIG-4130
Project: Pig
Issue Type: Bug
Components: impl
Reporter: Daniel Dai
Assignee: Daniel Dai
Priority: Minor
Fix For: 0.14.0
The following script fail:
{code}
a = load '/user/pig/tests/data/singlefile/studenttab10k' as (name:chararray,
age:int, gpa:float);
store a into 'Avro.intermediate' using OrcStorage();
b = load 'Avro.intermediate' using OrcStorage();
c = filter b by age < 30;
store c into 'ooo';
{code}
Message:
<file Avro_1.pig, line 7, column 16> Invalid field projection. Projected field
\[age\] does not exist.
If put a "exec" after the first store, the script success.
Pig does compile the script into two MR job, and correctly figure out the
dependency of the two, but it still need to goes for "Avro.intermediate" for
the schema of b when compiling, and at this time "Avro.intermediate" does not
exist. This also happens to other Loaders which need to get the schema from
input file, such as OrcStorage, etc.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
