[ 
https://issues.apache.org/jira/browse/PIG-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-4703:
----------------------------
    Status: Patch Available  (was: Open)

> TezOperator.stores shall not ship to backend
> --------------------------------------------
>
>                 Key: PIG-4703
>                 URL: https://issues.apache.org/jira/browse/PIG-4703
>             Project: Pig
>          Issue Type: Bug
>          Components: tez
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.16.0, 0.15.1
>
>         Attachments: PIG-4703-1.patch
>
>
> We end up of shipping all physical plan to the backend. One direct result is 
> UDFContext is not set in OutputCommitter. Here is how:
> 1. PigGraceShuffleVertexManager deserialize "pig.tez.plan". 
> TezOperator.stores reference physical plan so physical plan get deserialize.
> 2. When deserialize POUserFunc, POUserFunc.instantiateFunc is invoked. Inside 
> it, POUserFunc.setFuncInputSchema would use UDFContext and leave an entry 
> there (though it is empty map).
> 3. MapRedUtil.setupUDFContext would only deserialize if UDFContext is empty, 
> since we already touch UDFContext with some maps, UDFContext is not 
> deserialized



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to