[
https://issues.apache.org/jira/browse/PIG-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-4703:
----------------------------
Status: Patch Available (was: Open)
> TezOperator.stores shall not ship to backend
> --------------------------------------------
>
> Key: PIG-4703
> URL: https://issues.apache.org/jira/browse/PIG-4703
> Project: Pig
> Issue Type: Bug
> Components: tez
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.16.0, 0.15.1
>
> Attachments: PIG-4703-1.patch
>
>
> We end up of shipping all physical plan to the backend. One direct result is
> UDFContext is not set in OutputCommitter. Here is how:
> 1. PigGraceShuffleVertexManager deserialize "pig.tez.plan".
> TezOperator.stores reference physical plan so physical plan get deserialize.
> 2. When deserialize POUserFunc, POUserFunc.instantiateFunc is invoked. Inside
> it, POUserFunc.setFuncInputSchema would use UDFContext and leave an entry
> there (though it is empty map).
> 3. MapRedUtil.setupUDFContext would only deserialize if UDFContext is empty,
> since we already touch UDFContext with some maps, UDFContext is not
> deserialized
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
