[
https://issues.apache.org/jira/browse/PIG-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai resolved PIG-2489.
-----------------------------
Resolution: Fixed
Fix Version/s: 0.11
0.10
Assignee: Daniel Dai
Hadoop Flags: Reviewed
Unit tests pass. test-patch:
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or
modified tests.
[exec]
[exec] -1 javadoc. The javadoc tool appears to have generated 1
warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number
of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs
warnings.
[exec]
[exec] -1 release audit. The applied patch generated 527 release
audit warnings (more than the trunk's current 524 warnings).
javadoc warning is unrelated. No new file added, ignore release audit warning.
Patch committed to 0.10/trunk.
> Input Path Globbing{} not working with PigStorageSchema or PigStorage('\t',
> '-schema');
> ---------------------------------------------------------------------------------------
>
> Key: PIG-2489
> URL: https://issues.apache.org/jira/browse/PIG-2489
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0, 0.9.1, 0.10
> Reporter: Mitesh Singh Jat
> Assignee: Daniel Dai
> Fix For: 0.10, 0.11
>
> Attachments: PIG-2489-1.patch
>
>
> {code:title=test.pig}
> -- For Pig 0.9
> --A = LOAD 'input/PigStorageSchema/Temp{1,2}/pss*' USING
> org.apache.pig.piggybank.storage.PigStorageSchema();
> -- For Pig 0.10
> A = LOAD 'input/PigStorageSchema/Temp{1,2}/pss*' USING PigStorage('\t',
> '-schema');
> DESCRIBE A;
> DUMP A
> {code}
> Schema file _input/PigStorageSchema/Temp{1,2}.pig_schema_
> {code}
> {"fields":[{"name":"name","type":55,"schema":null,"description":"autogenerated
> from Pig Field
> Schema"},{"name":"val","type":10,"schema":null,"description":"autogenerated
> from Pig Field Schema"}],"version":0,"sortKeys":[],"sortKeyOrders":[]}
> {code}
> Header file _input/PigStorageSchema/Temp{1,2}/.pig_header_
> {code}
> name val
> {code}
> Sample input file _input/PigStorageSchema/Temp1/pss.in_
> {code}
> peter 1
> samir 3
> michael 4
> peter 2
> peter 4
> samir 1
> {code}
> On running the above pig script _test.pig_ with pig 0.10, the following error
> is received.
> {noformat}
> 012-01-24 04:07:42,210 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 1131: Could not find schema file for
> hdfs://nameNode:8020/user/mitesh/input/PigStorageSchema/Temp{1,2}/pss*
> {noformat}
> Pig Stack Trace
> {noformat}
> Pig Stack Trace
> ---------------
> ERROR 1131: Could not find schema file for
> hdfs://nameNode:8020/user/mitesh/input/PigStorageSchema/Temp{1,2}/pss*
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias A
> at org.apache.pig.PigServer.openIterator(PigServer.java:858)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:655)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
> at org.apache.pig.Main.run(Main.java:567)
> at org.apache.pig.Main.main(Main.java:111)
> Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias A
> at org.apache.pig.PigServer.storeEx(PigServer.java:957)
> at org.apache.pig.PigServer.store(PigServer.java:920)
> at org.apache.pig.PigServer.openIterator(PigServer.java:833)
> ... 7 more
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2245:
> <file test.pig, line 2, column 4> Cannot get schema from loadFunc
> org.apache.pig.builtin.PigStorage
> at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:154)
> at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:109)
> at
> org.apache.pig.newplan.logical.relational.LOStore.getSchema(LOStore.java:68)
> at
> org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:60)
> at
> org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.visit(SchemaAliasVisitor.java:84)
> at
> org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)
> at
> org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
> at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
> at org.apache.pig.PigServer$Graph.compile(PigServer.java:1618)
> at org.apache.pig.PigServer$Graph.compile(PigServer.java:1612)
> at org.apache.pig.PigServer$Graph.access$200(PigServer.java:1335)
> at org.apache.pig.PigServer.storeEx(PigServer.java:952)
> ... 9 more
> Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1131:
> Could not find schema file for
> hdfs://nameNode:8020/user/mitesh/input/PigStorageSchema/Temp{1,2}/pss*
> at
> org.apache.pig.builtin.JsonMetadata.nullOrException(JsonMetadata.java:222)
> at org.apache.pig.builtin.JsonMetadata.getSchema(JsonMetadata.java:191)
> at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:438)
> at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:150)
> {noformat}
> Whereas PigStorageSchema() or PigStorage('\t', '-schema') works with
> *wildcard* *.
> For example, following script works
> {code:title=test2.pig}
> A = LOAD 'input/PigStorageSchema/Temp*/pss*' USING PigStorage('\t',
> '-schema');
> DESCRIBE A;
> DUMP A;
> {code}
> As a workaround to make Temp{1,2} globbing work, the *,(comma)* separated
> multiple input paths (with no globbing)
> can given as input.
> {code:title=test2.pig}
> A = LOAD
> 'input/PigStorageSchema/Temp1/pss*,input/PigStorageSchema/Temp2/pss*' USING
> PigStorage('\t', '-schema');
> DESCRIBE A;
> DUMP A;
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
