[
https://issues.apache.org/jira/browse/PIG-4232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222778#comment-14222778
]
liyunzhang_intel commented on PIG-4232:
---------------------------------------
Upload PIG-4232_4.patch. In this patch, spark worker will fetch the udf script
like scriptingudf.py, scriptingudfs.rb and so on. Using this patch, PIG-4207
has been fixed. e2e tests about Scripting_[1,3,5,6-8],
StreamingPythonUDFs_[1-12] all passed in spark cluster env.
> UDFContext is not initialized in executors when running on Spark cluster
> ------------------------------------------------------------------------
>
> Key: PIG-4232
> URL: https://issues.apache.org/jira/browse/PIG-4232
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: Praveen Rachabattuni
> Assignee: liyunzhang_intel
> Attachments: PIG-4232.patch, PIG-4232_1.patch, PIG-4232_2.patch,
> PIG-4232_3.patch, PIG-4232_4.patch, pouserfunc.readObject.stacktrace.txt
>
>
> UDFContext is used in lot of features across pig code base. For example its
> used in PigStorage to pass columns information between the frontend and the
> backend code.
> https://github.com/apache/pig/blob/spark/src/org/apache/pig/builtin/PigStorage.java#L246-L247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
