[
https://issues.apache.org/jira/browse/PIG-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477108#comment-13477108
]
Bill Graham commented on PIG-2983:
----------------------------------
As a percentage of the execution time I'd think the time to build the plan
would be small. How long is this taking for you?
For a unique signature see {{pig.logical.plan.signature}}, which is a hash of
the logical plan.
> Enable storing/loading an execution plan
> ----------------------------------------
>
> Key: PIG-2983
> URL: https://issues.apache.org/jira/browse/PIG-2983
> Project: Pig
> Issue Type: Wish
> Components: impl
> Reporter: Mathias Herberts
>
> I have no idea at the moment if this is feasible or not, but here is the idea.
> When running Pig in production it happens quite frequently that the building
> of the plan takes a fair amount of time compared to the actual execution of
> the job on the Hadoop cluster.
> For jobs which are run periodically without modification of the underlying
> pig script it would be nice to be able to store the execution plan when it is
> first built, then load that persisted plan for all other executions.
> A unique signature of the script could be computed, for example by removing
> comments and blank lines and computing a digest, and the computed plan stored
> under that name with a signature mechanism to ensure it's correct.
> This would speed up lots of executions of Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
