[
https://issues.apache.org/jira/browse/PIG-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020039#comment-13020039
]
[email protected] commented on PIG-1979:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/587/#review461
-----------------------------------------------------------
Ship it!
+1
- thejas
On 2011-04-13 00:11:37, Daniel Dai wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/587/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-13 00:11:37)
bq.
bq.
bq. Review request for pig and thejas.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. See PIG-1979
bq.
bq.
bq. This addresses bug PIG-1979.
bq. https://issues.apache.org/jira/browse/PIG-1979
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
http://svn.apache.org/repos/asf/pig/branches/branch-0.8/src/org/apache/pig/newplan/logical/expression/DereferenceExpression.java
1091122
bq.
http://svn.apache.org/repos/asf/pig/branches/branch-0.8/test/org/apache/pig/test/TestEvalPipeline2.java
1091122
bq.
bq. Diff: https://reviews.apache.org/r/587/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Test-patch:
bq. [exec] +1 overall.
bq. [exec]
bq. [exec] +1 @author. The patch does not contain any @author tags.
bq. [exec]
bq. [exec] +1 tests included. The patch appears to include 3 new or
modified tests.
bq. [exec]
bq. [exec] +1 javadoc. The javadoc tool did not generate any warning
messages.
bq. [exec]
bq. [exec] +1 javac. The applied patch does not increase the total
number of javac compiler warnings.
bq. [exec]
bq. [exec] +1 findbugs. The patch does not introduce any new
Findbugs warnings.
bq. [exec]
bq. [exec] +1 release audit. The applied patch does not increase the
total number of release audit warnings.
bq.
bq. Unit-test:
bq. all pass
bq.
bq.
bq. Thanks,
bq.
bq. Daniel
bq.
bq.
> New logical plan failing with ERROR 2229: Couldn't find matching uid -1
> ------------------------------------------------------------------------
>
> Key: PIG-1979
> URL: https://issues.apache.org/jira/browse/PIG-1979
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
> Assignee: Daniel Dai
> Fix For: 0.8.0
>
> Attachments: PIG-1979-1-trunk.patch, PIG-1979-1.patch
>
>
> The below is my script
> {code}
> register myudf.jar;
> c01 = LOAD 'input' USING org.test.MyTableLoader('');
> c02 = FILTER c01 BY result == 'OK' AND formatted IS NOT NULL AND formatted
> != '' ;
> c03 = FOREACH c02 GENERATE url, formatted, FLATTEN(usage);
> c04 = FOREACH c03 GENERATE usage::domain AS domain, url, formatted;
> doc_001 = FOREACH c04 GENERATE domain,url, FLATTEN(MyExtractor(formatted)) AS
> category;
> doc_004_1 = GROUP doc_001 BY (domain,url);
> doc_005 = FOREACH doc_004_1 GENERATE group.domain as domain, group.url as
> url, doc_001.category as category;
> STORE doc_005 INTO 'out_final' USING PigStorage();
> review1 = FOREACH c04 GENERATE domain,url, MyExtractor(formatted) AS rev;
> review2 = FILTER review1 BY SIZE(rev)>0;
> joinresult = JOIN review2 by (domain,url), doc_005 by (domain,url);
> finalresult = FOREACH joinresult GENERATE doc_005::category;
> STORE finalresult INTO 'out_final' using PigStorage();
> {code}
> The script is failing in building the plan, while applying for logical
> optimization rule for AddForEach.
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2229: Couldn't find matching
> uid -1 for project (Name: Project Type: bytearray Uid: 106 Input: 0 Column: 5)
> The problem is happening when I try to include doc_005::category in the
> projection for relation finalresult. This is field is orginated from the udf
> org.vivek.udfs.MyExtractor (source given below).
> {code}
> import java.io.IOException;
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.*;
> import org.apache.pig.impl.logicalLayer.FrontendException;
> import org.apache.pig.impl.logicalLayer.schema.Schema;
> import org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema;
> public class MyExtractor extends EvalFunc<DataBag>
> {
> @Override
> public Schema outputSchema(Schema arg0) {
> try {
> return Schema.generateNestedSchema(DataType.BAG,
> DataType.CHARARRAY);
> } catch (FrontendException e) {
> System.err.println("Error while generating schema. "+e);
> return new Schema(new FieldSchema(null, DataType.BAG));
> }
> }
> @Override
> public DataBag exec(Tuple inputTuple)
> throws IOException
> {
> try {
> Tuple tp2 = TupleFactory.getInstance().newTuple(1);
> tp2.set(0, (inputTuple.get(0).toString()+inputTuple.hashCode()));
> DataBag retBag = BagFactory.getInstance().newDefaultBag();
> retBag.add(tp2);
> return retBag;
> }
> catch (Exception e) {
> throw new IOException(" Caught exception", e);
> }
> }
> }
> {code}
> The script goes through fine if I disable AddForEach rule by -t AddForEach
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
