[jira] [Updated] (PIG-3622) Allow casting bytearray fileds to bytearray type

2013-12-19 Thread Redis Liu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Redis Liu updated PIG-3622:
---

Attachment: 3622-v2.patch

this works on our cluster

 Allow casting bytearray fileds to bytearray type
 

 Key: PIG-3622
 URL: https://issues.apache.org/jira/browse/PIG-3622
 Project: Pig
  Issue Type: Improvement
 Environment: 0.12
Reporter: Redis Liu
Assignee: Redis Liu
Priority: Minor
 Attachments: 3622-v1.patch, 3622-v2.patch


 test.pig:
 AA = load '1.txt' USING PigStorage(' ') as (a:bytearray, b:chararray, 
 c:chararray);
 AA1 = filter AA by a == '1';
 AA2 = foreach AA1 generate *, ( a == '1' ? a : null ) as myd;
 dump AA2;
 the INPUT file 1.txt is as below:
 a b c
 1 2 3
 4 5 6
 2 3 4
 b a c
 c a b
 run the pig script in this way:
 # pig -x local test.pig
 It'll fail with this error message:
 Pig Stack Trace
 ---
 ERROR 1051: Cannot cast to bytearray
 org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
 open iterator for alias AA2
   at org.apache.pig.PigServer.openIterator(PigServer.java:882)
   at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
   at 
 org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
   at 
 org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
   at 
 org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
   at org.apache.pig.Main.run(Main.java:607)
   at org.apache.pig.Main.main(Main.java:156)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
 Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias AA2
   at org.apache.pig.PigServer.storeEx(PigServer.java:984)
   at org.apache.pig.PigServer.store(PigServer.java:944)
   at org.apache.pig.PigServer.openIterator(PigServer.java:857)
   ... 12 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 1059: 
 file test.pig, line 7, column 6 Problem while reconciling output schema of 
 ForEach
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.throwTypeCheckerException(TypeCheckingRelVisitor.java:142)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:182)
   at 
 org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:76)
   at 
 org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at org.apache.pig.PigServer$Graph.compile(PigServer.java:1733)
   at org.apache.pig.PigServer$Graph.compile(PigServer.java:1710)
   at org.apache.pig.PigServer$Graph.access$200(PigServer.java:1411)
   at org.apache.pig.PigServer.storeEx(PigServer.java:979)
   ... 14 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 2216: 
 file test.pig, line 7, column 34 Problem getting fieldSchema for (Name: 
 Cast Type: bytearray Uid: 17)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:603)
   at 
 org.apache.pig.newplan.logical.expression.BinCondExpression.accept(BinCondExpression.java:84)
   at 
 org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visitExpressionPlan(TypeCheckingRelVisitor.java:191)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:157)
   at 
 org.apache.pig.newplan.logical.relational.LOGenerate.accept(LOGenerate.java:242)
   at 
 org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:174)
   ... 21 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 1051: Cannot cast to bytearray
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:494)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.insertCast(TypeCheckingExpVisitor.java:472)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:599)
   ... 30 more

[jira] [Updated] (PIG-3622) Allow casting bytearray fileds to bytearray type

2013-12-19 Thread Redis Liu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Redis Liu updated PIG-3622:
---

Attachment: (was: 3622-v1.patch)

 Allow casting bytearray fileds to bytearray type
 

 Key: PIG-3622
 URL: https://issues.apache.org/jira/browse/PIG-3622
 Project: Pig
  Issue Type: Improvement
 Environment: 0.12
Reporter: Redis Liu
Assignee: Redis Liu
Priority: Minor
 Attachments: 3622-v2.patch


 test.pig:
 AA = load '1.txt' USING PigStorage(' ') as (a:bytearray, b:chararray, 
 c:chararray);
 AA1 = filter AA by a == '1';
 AA2 = foreach AA1 generate *, ( a == '1' ? a : null ) as myd;
 dump AA2;
 the INPUT file 1.txt is as below:
 a b c
 1 2 3
 4 5 6
 2 3 4
 b a c
 c a b
 run the pig script in this way:
 # pig -x local test.pig
 It'll fail with this error message:
 Pig Stack Trace
 ---
 ERROR 1051: Cannot cast to bytearray
 org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
 open iterator for alias AA2
   at org.apache.pig.PigServer.openIterator(PigServer.java:882)
   at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
   at 
 org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
   at 
 org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
   at 
 org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
   at org.apache.pig.Main.run(Main.java:607)
   at org.apache.pig.Main.main(Main.java:156)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
 Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias AA2
   at org.apache.pig.PigServer.storeEx(PigServer.java:984)
   at org.apache.pig.PigServer.store(PigServer.java:944)
   at org.apache.pig.PigServer.openIterator(PigServer.java:857)
   ... 12 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 1059: 
 file test.pig, line 7, column 6 Problem while reconciling output schema of 
 ForEach
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.throwTypeCheckerException(TypeCheckingRelVisitor.java:142)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:182)
   at 
 org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:76)
   at 
 org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at org.apache.pig.PigServer$Graph.compile(PigServer.java:1733)
   at org.apache.pig.PigServer$Graph.compile(PigServer.java:1710)
   at org.apache.pig.PigServer$Graph.access$200(PigServer.java:1411)
   at org.apache.pig.PigServer.storeEx(PigServer.java:979)
   ... 14 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 2216: 
 file test.pig, line 7, column 34 Problem getting fieldSchema for (Name: 
 Cast Type: bytearray Uid: 17)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:603)
   at 
 org.apache.pig.newplan.logical.expression.BinCondExpression.accept(BinCondExpression.java:84)
   at 
 org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visitExpressionPlan(TypeCheckingRelVisitor.java:191)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:157)
   at 
 org.apache.pig.newplan.logical.relational.LOGenerate.accept(LOGenerate.java:242)
   at 
 org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:174)
   ... 21 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 1051: Cannot cast to bytearray
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:494)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.insertCast(TypeCheckingExpVisitor.java:472)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:599)
   ... 30 more

[jira] [Updated] (PIG-3622) Allow casting bytearray fileds to bytearray type

2013-12-13 Thread Redis Liu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Redis Liu updated PIG-3622:
---

Attachment: 3622-v1.patch

 Allow casting bytearray fileds to bytearray type
 

 Key: PIG-3622
 URL: https://issues.apache.org/jira/browse/PIG-3622
 Project: Pig
  Issue Type: Improvement
 Environment: 0.12
Reporter: Redis Liu
Priority: Minor
 Attachments: 3622-v1.patch


 test.pig:
 AA = load '1.txt' USING PigStorage(' ') as (a:bytearray, b:chararray, 
 c:chararray);
 AA1 = filter AA by a == '1';
 AA2 = foreach AA1 generate *, ( a == '1' ? a : null ) as myd;
 dump AA2;
 the INPUT file 1.txt is as below:
 a b c
 1 2 3
 4 5 6
 2 3 4
 b a c
 c a b
 run the pig script in this way:
 # pig -x local test.pig
 It'll fail with this error message:
 Pig Stack Trace
 ---
 ERROR 1051: Cannot cast to bytearray
 org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
 open iterator for alias AA2
   at org.apache.pig.PigServer.openIterator(PigServer.java:882)
   at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
   at 
 org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
   at 
 org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
   at 
 org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
   at org.apache.pig.Main.run(Main.java:607)
   at org.apache.pig.Main.main(Main.java:156)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
 Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias AA2
   at org.apache.pig.PigServer.storeEx(PigServer.java:984)
   at org.apache.pig.PigServer.store(PigServer.java:944)
   at org.apache.pig.PigServer.openIterator(PigServer.java:857)
   ... 12 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 1059: 
 file test.pig, line 7, column 6 Problem while reconciling output schema of 
 ForEach
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.throwTypeCheckerException(TypeCheckingRelVisitor.java:142)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:182)
   at 
 org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:76)
   at 
 org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at org.apache.pig.PigServer$Graph.compile(PigServer.java:1733)
   at org.apache.pig.PigServer$Graph.compile(PigServer.java:1710)
   at org.apache.pig.PigServer$Graph.access$200(PigServer.java:1411)
   at org.apache.pig.PigServer.storeEx(PigServer.java:979)
   ... 14 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 2216: 
 file test.pig, line 7, column 34 Problem getting fieldSchema for (Name: 
 Cast Type: bytearray Uid: 17)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:603)
   at 
 org.apache.pig.newplan.logical.expression.BinCondExpression.accept(BinCondExpression.java:84)
   at 
 org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visitExpressionPlan(TypeCheckingRelVisitor.java:191)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:157)
   at 
 org.apache.pig.newplan.logical.relational.LOGenerate.accept(LOGenerate.java:242)
   at 
 org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:174)
   ... 21 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 1051: Cannot cast to bytearray
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:494)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.insertCast(TypeCheckingExpVisitor.java:472)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:599)
   ... 30 more

[jira] [Updated] (PIG-3622) Allow casting bytearray fileds to bytearray type

2013-12-13 Thread Alan Gates (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PIG-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated PIG-3622:


Assignee: Redis Liu

 Allow casting bytearray fileds to bytearray type
 

 Key: PIG-3622
 URL: https://issues.apache.org/jira/browse/PIG-3622
 Project: Pig
  Issue Type: Improvement
 Environment: 0.12
Reporter: Redis Liu
Assignee: Redis Liu
Priority: Minor
 Attachments: 3622-v1.patch


 test.pig:
 AA = load '1.txt' USING PigStorage(' ') as (a:bytearray, b:chararray, 
 c:chararray);
 AA1 = filter AA by a == '1';
 AA2 = foreach AA1 generate *, ( a == '1' ? a : null ) as myd;
 dump AA2;
 the INPUT file 1.txt is as below:
 a b c
 1 2 3
 4 5 6
 2 3 4
 b a c
 c a b
 run the pig script in this way:
 # pig -x local test.pig
 It'll fail with this error message:
 Pig Stack Trace
 ---
 ERROR 1051: Cannot cast to bytearray
 org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to 
 open iterator for alias AA2
   at org.apache.pig.PigServer.openIterator(PigServer.java:882)
   at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
   at 
 org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
   at 
 org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
   at 
 org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
   at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
   at org.apache.pig.Main.run(Main.java:607)
   at org.apache.pig.Main.main(Main.java:156)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
 Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias AA2
   at org.apache.pig.PigServer.storeEx(PigServer.java:984)
   at org.apache.pig.PigServer.store(PigServer.java:944)
   at org.apache.pig.PigServer.openIterator(PigServer.java:857)
   ... 12 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 1059: 
 file test.pig, line 7, column 6 Problem while reconciling output schema of 
 ForEach
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.throwTypeCheckerException(TypeCheckingRelVisitor.java:142)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:182)
   at 
 org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:76)
   at 
 org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at org.apache.pig.PigServer$Graph.compile(PigServer.java:1733)
   at org.apache.pig.PigServer$Graph.compile(PigServer.java:1710)
   at org.apache.pig.PigServer$Graph.access$200(PigServer.java:1411)
   at org.apache.pig.PigServer.storeEx(PigServer.java:979)
   ... 14 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 2216: 
 file test.pig, line 7, column 34 Problem getting fieldSchema for (Name: 
 Cast Type: bytearray Uid: 17)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:603)
   at 
 org.apache.pig.newplan.logical.expression.BinCondExpression.accept(BinCondExpression.java:84)
   at 
 org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visitExpressionPlan(TypeCheckingRelVisitor.java:191)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:157)
   at 
 org.apache.pig.newplan.logical.relational.LOGenerate.accept(LOGenerate.java:242)
   at 
 org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
   at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:174)
   ... 21 more
 Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: 
 ERROR 1051: Cannot cast to bytearray
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:494)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.insertCast(TypeCheckingExpVisitor.java:472)
   at 
 org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:599)
   ... 30 more