[ https://issues.apache.org/jira/browse/PIG-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020039#comment-13020039 ]
jirapos...@reviews.apache.org commented on PIG-1979: ---------------------------------------------------- ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/587/#review461 ----------------------------------------------------------- Ship it! +1 - thejas On 2011-04-13 00:11:37, Daniel Dai wrote: bq. bq. ----------------------------------------------------------- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/587/ bq. ----------------------------------------------------------- bq. bq. (Updated 2011-04-13 00:11:37) bq. bq. bq. Review request for pig and thejas. bq. bq. bq. Summary bq. ------- bq. bq. See PIG-1979 bq. bq. bq. This addresses bug PIG-1979. bq. https://issues.apache.org/jira/browse/PIG-1979 bq. bq. bq. Diffs bq. ----- bq. bq. http://svn.apache.org/repos/asf/pig/branches/branch-0.8/src/org/apache/pig/newplan/logical/expression/DereferenceExpression.java 1091122 bq. http://svn.apache.org/repos/asf/pig/branches/branch-0.8/test/org/apache/pig/test/TestEvalPipeline2.java 1091122 bq. bq. Diff: https://reviews.apache.org/r/587/diff bq. bq. bq. Testing bq. ------- bq. bq. Test-patch: bq. [exec] +1 overall. bq. [exec] bq. [exec] +1 @author. The patch does not contain any @author tags. bq. [exec] bq. [exec] +1 tests included. The patch appears to include 3 new or modified tests. bq. [exec] bq. [exec] +1 javadoc. The javadoc tool did not generate any warning messages. bq. [exec] bq. [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. bq. [exec] bq. [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. bq. [exec] bq. [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. bq. bq. Unit-test: bq. all pass bq. bq. bq. Thanks, bq. bq. Daniel bq. bq. > New logical plan failing with ERROR 2229: Couldn't find matching uid -1 > ------------------------------------------------------------------------ > > Key: PIG-1979 > URL: https://issues.apache.org/jira/browse/PIG-1979 > Project: Pig > Issue Type: Bug > Components: impl > Affects Versions: 0.8.0, 0.9.0 > Reporter: Vivek Padmanabhan > Assignee: Daniel Dai > Fix For: 0.8.0 > > Attachments: PIG-1979-1-trunk.patch, PIG-1979-1.patch > > > The below is my script > {code} > register myudf.jar; > c01 = LOAD 'input' USING org.test.MyTableLoader(''); > c02 = FILTER c01 BY result == 'OK' AND formatted IS NOT NULL AND formatted > != '' ; > c03 = FOREACH c02 GENERATE url, formatted, FLATTEN(usage); > c04 = FOREACH c03 GENERATE usage::domain AS domain, url, formatted; > doc_001 = FOREACH c04 GENERATE domain,url, FLATTEN(MyExtractor(formatted)) AS > category; > doc_004_1 = GROUP doc_001 BY (domain,url); > doc_005 = FOREACH doc_004_1 GENERATE group.domain as domain, group.url as > url, doc_001.category as category; > STORE doc_005 INTO 'out_final' USING PigStorage(); > review1 = FOREACH c04 GENERATE domain,url, MyExtractor(formatted) AS rev; > review2 = FILTER review1 BY SIZE(rev)>0; > joinresult = JOIN review2 by (domain,url), doc_005 by (domain,url); > finalresult = FOREACH joinresult GENERATE doc_005::category; > STORE finalresult INTO 'out_final' using PigStorage(); > {code} > The script is failing in building the plan, while applying for logical > optimization rule for AddForEach. > ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2229: Couldn't find matching > uid -1 for project (Name: Project Type: bytearray Uid: 106 Input: 0 Column: 5) > The problem is happening when I try to include doc_005::category in the > projection for relation finalresult. This is field is orginated from the udf > org.vivek.udfs.MyExtractor (source given below). > {code} > import java.io.IOException; > import org.apache.pig.EvalFunc; > import org.apache.pig.data.*; > import org.apache.pig.impl.logicalLayer.FrontendException; > import org.apache.pig.impl.logicalLayer.schema.Schema; > import org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema; > public class MyExtractor extends EvalFunc<DataBag> > { > @Override > public Schema outputSchema(Schema arg0) { > try { > return Schema.generateNestedSchema(DataType.BAG, > DataType.CHARARRAY); > } catch (FrontendException e) { > System.err.println("Error while generating schema. "+e); > return new Schema(new FieldSchema(null, DataType.BAG)); > } > } > @Override > public DataBag exec(Tuple inputTuple) > throws IOException > { > try { > Tuple tp2 = TupleFactory.getInstance().newTuple(1); > tp2.set(0, (inputTuple.get(0).toString()+inputTuple.hashCode())); > DataBag retBag = BagFactory.getInstance().newDefaultBag(); > retBag.add(tp2); > return retBag; > } > catch (Exception e) { > throw new IOException(" Caught exception", e); > } > } > } > {code} > The script goes through fine if I disable AddForEach rule by -t AddForEach -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira