[
https://issues.apache.org/jira/browse/PIG-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640938#action_12640938
]
Shubham Chopra commented on PIG-502:
------------------------------------
Illustrate doesn't currently support Limit, Split(Both implicit and explicit),
nested foreach and maps data types. These need to be handled separately as they
don't follow the equivalence class model like other operators. I am currently
working on these and the PigPen development environment. I will try to get them
in the trunk as soon as possible.
> Limit and Illustrate do not work together
> -----------------------------------------
>
> Key: PIG-502
> URL: https://issues.apache.org/jira/browse/PIG-502
> Project: Pig
> Issue Type: Improvement
> Components: tools
> Affects Versions: types_branch
> Environment: Hadoop 18
> Reporter: Viraj Bhat
>
> Suppose a user wants to do an illustrate command after limiting his data to a
> certain number of records, it does not seem to work..
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> {code}
> MYDATA = load 'testfilelarge.txt' as (f1, f2, f3, f4, f5);
> MYDATA = limit MYDATA 10;
> describe MYDATA;
> illustrate MYDATA;
> {code}
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> Running this script produces the following output and error
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> MYDATA: {f1: bytearray,f2: bytearray,f3: bytearray,f4: bytearray,f5:
> bytearray}
> 2008-10-18 02:14:26,900 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to hadoop fil
> e system at: hdfs://localhost:9000
> 2008-10-18 02:14:27,013 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
> to map-reduce
> job tracker at: localhost:9001
> java.lang.RuntimeException: Unrecognized logical operator.
> at
> org.apache.pig.pen.EquivalenceClasses.GetEquivalenceClasses(EquivalenceClasses.java:60)
> at
> org.apache.pig.pen.DerivedDataVisitor.evaluateOperator(DerivedDataVisitor.java:368)
> at
> org.apache.pig.pen.DerivedDataVisitor.visit(DerivedDataVisitor.java:273)
> at org.apache.pig.impl.logicalLayer.LOLimit.visit(LOLimit.java:71)
> at org.apache.pig.impl.logicalLayer.LOLimit.visit(LOLimit.java:10)
> at
> org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at
> org.apache.pig.pen.LineageTrimmingVisitor.init(LineageTrimmingVisitor.java:98)
> at
> org.apache.pig.pen.LineageTrimmingVisitor.<init>(LineageTrimmingVisitor.java:90)
> at
> org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:106)
> at org.apache.pig.PigServer.getExamples(PigServer.java:630)
> at
> org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:279)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:183)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
> at org.apache.pig.Main.main(Main.java:306)
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> If I remove the illustrate and replace it with "dump MYDATA;" it works..
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> {code}
> MYDATA = load 'testfilelarge.txt' as (f1, f2, f3, f4, f5);
> MYDATA = limit MYDATA 10;
> describe MYDATA;
> -- illustrate MYDATA;
> dump MYDATA;
> {code}
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
