[
https://issues.apache.org/jira/browse/PIG-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Le Dem resolved PIG-2654.
--------------------------------
Resolution: Duplicate
> LoadFunc and StoreFunc implementations should be instanciated only once in
> the FE and once in the BE per call in a given script
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-2654
> URL: https://issues.apache.org/jira/browse/PIG-2654
> Project: Pig
> Issue Type: Improvement
> Reporter: Julien Le Dem
>
> I'm running the following script:
> {code}
> A = LOAD 'foo' USING Loader();
> STORE A INTO 'bar' USING Storer();
> {code}
> Using instrumented Loader an Storer I get the following result:
> * 7 instances of Loader
> * 6 instances of Storer
> each instance gets an ID and logs the calls to its methods:
> {code}
> Loader(1).<init> called by
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:791)
> Loader(2).<init> called by
> org.apache.pig.parser.LogicalPlanBuilder.getAbolutePathForLoad(LogicalPlanBuilder.java:417)
> Loader(2).relativeToAbsolutePath called by
> org.apache.pig.parser.LogicalPlanBuilder.getAbolutePathForLoad(LogicalPlanBuilder.java:421)
> Loader(3).<init> called by
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:791)
> Loader(4).<init> called by
> org.apache.pig.parser.LogicalPlanBuilder.getAbolutePathForLoad(LogicalPlanBuilder.java:417)
> Storer(1).<init> called by
> org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:791)
> Storer(2).<init> called by
> org.apache.pig.parser.LogicalPlanBuilder.getAbolutePathForStore(LogicalPlanBuilder.java:478)
> Storer(2).setStoreFuncUDFContextSignature called by
> org.apache.pig.parser.LogicalPlanBuilder.getAbolutePathForStore(LogicalPlanBuilder.java:481)
> Storer(2).relToAbsPathForStoreLocation called by
> org.apache.pig.parser.LogicalPlanBuilder.getAbolutePathForStore(LogicalPlanBuilder.java:484)
> Storer(3).<init> called by
> org.apache.pig.newplan.logical.relational.LOStore.<init>(LOStore.java:55)
> Storer(3).setStoreFuncUDFContextSignature called by
> org.apache.pig.newplan.logical.relational.LOStore.setSignature(LOStore.java:129)
> Loader(5).<init> called by
> org.apache.pig.newplan.logical.relational.LOLoad.getLoadFunc(LOLoad.java:77)
> Loader(5).setUDFContextSignature called by
> org.apache.pig.newplan.logical.relational.LOLoad.getLoadFunc(LOLoad.java:78)
> Loader(5).getSchema called by
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> Loader(5).getSchema called by
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> Loader(5).getSchema called by
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> Loader(5).getSchema called by
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> Loader(5).getSchema called by
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> Loader(5).getSchema called by
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> Storer(3).setStoreFuncUDFContextSignature called by
> org.apache.pig.newplan.logical.relational.LOStore.setSignature(LOStore.java:129)
> Loader(5).getSchema called by
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> Loader(5).getSchema called by
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> Loader(5).getSchema called by
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> Storer(3).setStoreLocation called by
> org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:68)
> Storer(3).getOutputFormat called by
> org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80)
> Loader(5).getSchema called by
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:151)
> Loader(5).setLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:389)
> Storer(4).<init> called by
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getStoreFunc(POStore.java:232)
> Storer(4).setStoreFuncUDFContextSignature called by
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getStoreFunc(POStore.java:233)
> Storer(4).setStoreLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:480)
> Storer(5).<init> called by
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getStoreFunc(POStore.java:232)
> Storer(5).setStoreFuncUDFContextSignature called by
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getStoreFunc(POStore.java:233)
> Storer(5).setStoreLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setLocation(PigOutputFormat.java:170)
> Storer(5).getOutputFormat called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecsHelper(PigOutputFormat.java:203)
> Loader(6).<init> called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:254)
> Loader(6).setUDFContextSignature called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.passLoadSignature(PigInputFormat.java:187)
> Loader(6).setLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:266)
> Loader(6).getInputFormat called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:272)
> Storer(6).<init> called by
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getStoreFunc(POStore.java:232)
> Storer(6).setStoreFuncUDFContextSignature called by
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getStoreFunc(POStore.java:233)
> Storer(6).setStoreLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setLocation(PigOutputFormat.java:170)
> Storer(6).getOutputFormat called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:92)
> Storer(6).setStoreLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setLocation(PigOutputFormat.java:170)
> Loader(7).<init> called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getLoadFunc(PigInputFormat.java:158)
> Loader(7).setUDFContextSignature called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.passLoadSignature(PigInputFormat.java:187)
> Loader(7).setLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.mergeSplitSpecificConf(PigInputFormat.java:138)
> Loader(7).getInputFormat called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:117)
> Storer(6).setStoreLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setLocation(PigOutputFormat.java:170)
> Storer(6).getOutputFormat called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
> Storer(6).prepareToWrite called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125)
> Loader(7).setUDFContextSignature called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.passLoadSignature(PigInputFormat.java:187)
> Loader(7).prepareToRead called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initialize(PigRecordReader.java:182)
> Loader(7).getNext called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:194)
> Storer(6).putNext called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
> Loader(7).getNext called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:194)
> Storer(6).putNext called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
> Loader(7).getNext called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:194)
> Storer(6).putNext called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
> Loader(7).getNext called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:194)
> Storer(6).setStoreLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setLocation(PigOutputFormat.java:170)
> Storer(6).setStoreLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setLocation(PigOutputFormat.java:170)
> Storer(4).setStoreLocation called by
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setLocation(PigOutputFormat.java:170)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
