-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/361/
-----------------------------------------------------------
Review request for pig and Richard Ding.
Summary
-------
Currently Pig interprets the result type of a relation as a bag. However the
schema of the relation directly contains the schema describing the fields in
the tuples for the relation. However when a udf wants to return a bag or if
there is a bag in input data or if the user creates a bag constant, the schema
of the bag has one field schema which is that of the tuple. The Tuple's schema
has the types of the fields. To be able to access the fields from the bag
directly in such a case by using something like <bagname>.<fieldname> or
<bag>.<fieldposition>, the schema of the bag should have the twoLevelAccess set
to true so that pig's type system can get traverse the tuple schema and get to
the field in question. This is confusing - we should try and see if we can
avoid needing this extra flag. A possible solution is to treat bags the same
way - whether they represent relations or real bags. Another way is to
introduce a special "relation" datatype for the result type of a relation and
bag type would be used only for true bags. In this case, we would always need
bag schema to have a tuple schema which would describe the fields.
This addresses bug PIG-847.
https://issues.apache.org/jira/browse/PIG-847
Diffs
-----
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/ResourceSchema.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/TOKENIZE.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/DataType.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/LOForEach.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/LOProject.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/logicalLayer/schema/Schema.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/Util.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/DereferenceExpression.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/ExpToPhyTranslationVisitor.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/expression/ProjectExpression.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOCogroup.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOGenerate.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LOInnerLoad.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/relational/LogicalSchema.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/DuplicateForEachColumnRewrite.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/InputOutputFileValidator.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestLogicalPlanMigrationVisitor.java
1063925
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestSchema.java
1063925
Diff: https://reviews.apache.org/r/361/diff
Testing
-------
Test-patch:
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 7 new or
modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
[exec]
[exec] -1 javac. The applied patch generated 964 javac compiler
warnings (more than the trunk's current 962 warnings).
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec]
[exec] -1 release audit. The applied patch generated 489 release
audit warnings (more than the trunk's current 485 warnings).
Two javac warnings are depredation warnings in QueryParser.jjt. I cannot
suppress them otherwise javacc will complain. There is no new file added so
ignore release audit warning.
Unit-test:
all pass
End-to-end test:
all pass
Thanks,
Daniel
