[ https://issues.apache.org/jira/browse/PIG-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202008#comment-13202008 ]
Thejas M Nair commented on PIG-2489: ------------------------------------ +1 > Input Path Globbing{} not working with PigStorageSchema or PigStorage('\t', > '-schema'); > --------------------------------------------------------------------------------------- > > Key: PIG-2489 > URL: https://issues.apache.org/jira/browse/PIG-2489 > Project: Pig > Issue Type: Bug > Affects Versions: 0.9.0, 0.9.1, 0.10 > Reporter: Mitesh Singh Jat > Attachments: PIG-2489-1.patch > > > {code:title=test.pig} > -- For Pig 0.9 > --A = LOAD 'input/PigStorageSchema/Temp{1,2}/pss*' USING > org.apache.pig.piggybank.storage.PigStorageSchema(); > -- For Pig 0.10 > A = LOAD 'input/PigStorageSchema/Temp{1,2}/pss*' USING PigStorage('\t', > '-schema'); > DESCRIBE A; > DUMP A > {code} > Schema file _input/PigStorageSchema/Temp{1,2}.pig_schema_ > {code} > {"fields":[{"name":"name","type":55,"schema":null,"description":"autogenerated > from Pig Field > Schema"},{"name":"val","type":10,"schema":null,"description":"autogenerated > from Pig Field Schema"}],"version":0,"sortKeys":[],"sortKeyOrders":[]} > {code} > Header file _input/PigStorageSchema/Temp{1,2}/.pig_header_ > {code} > name val > {code} > Sample input file _input/PigStorageSchema/Temp1/pss.in_ > {code} > peter 1 > samir 3 > michael 4 > peter 2 > peter 4 > samir 1 > {code} > On running the above pig script _test.pig_ with pig 0.10, the following error > is received. > {noformat} > 012-01-24 04:07:42,210 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR > 1131: Could not find schema file for > hdfs://nameNode:8020/user/mitesh/input/PigStorageSchema/Temp{1,2}/pss* > {noformat} > Pig Stack Trace > {noformat} > Pig Stack Trace > --------------- > ERROR 1131: Could not find schema file for > hdfs://nameNode:8020/user/mitesh/input/PigStorageSchema/Temp{1,2}/pss* > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to > open iterator for alias A > at org.apache.pig.PigServer.openIterator(PigServer.java:858) > at > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:655) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164) > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) > at org.apache.pig.Main.run(Main.java:567) > at org.apache.pig.Main.main(Main.java:111) > Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias A > at org.apache.pig.PigServer.storeEx(PigServer.java:957) > at org.apache.pig.PigServer.store(PigServer.java:920) > at org.apache.pig.PigServer.openIterator(PigServer.java:833) > ... 7 more > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2245: > <file test.pig, line 2, column 4> Cannot get schema from loadFunc > org.apache.pig.builtin.PigStorage > at > org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:154) > at > org.apache.pig.newplan.logical.relational.LOLoad.getSchema(LOLoad.java:109) > at > org.apache.pig.newplan.logical.relational.LOStore.getSchema(LOStore.java:68) > at > org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.validate(SchemaAliasVisitor.java:60) > at > org.apache.pig.newplan.logical.visitor.SchemaAliasVisitor.visit(SchemaAliasVisitor.java:84) > at > org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77) > at > org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75) > at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50) > at org.apache.pig.PigServer$Graph.compile(PigServer.java:1618) > at org.apache.pig.PigServer$Graph.compile(PigServer.java:1612) > at org.apache.pig.PigServer$Graph.access$200(PigServer.java:1335) > at org.apache.pig.PigServer.storeEx(PigServer.java:952) > ... 9 more > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1131: > Could not find schema file for > hdfs://nameNode:8020/user/mitesh/input/PigStorageSchema/Temp{1,2}/pss* > at > org.apache.pig.builtin.JsonMetadata.nullOrException(JsonMetadata.java:222) > at org.apache.pig.builtin.JsonMetadata.getSchema(JsonMetadata.java:191) > at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:438) > at > org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:150) > {noformat} > Whereas PigStorageSchema() or PigStorage('\t', '-schema') works with > *wildcard* *. > For example, following script works > {code:title=test2.pig} > A = LOAD 'input/PigStorageSchema/Temp*/pss*' USING PigStorage('\t', > '-schema'); > DESCRIBE A; > DUMP A; > {code} > As a workaround to make Temp{1,2} globbing work, the *,(comma)* separated > multiple input paths (with no globbing) > can given as input. > {code:title=test2.pig} > A = LOAD > 'input/PigStorageSchema/Temp1/pss*,input/PigStorageSchema/Temp2/pss*' USING > PigStorage('\t', '-schema'); > DESCRIBE A; > DUMP A; > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira