[
https://issues.apache.org/jira/browse/PIG-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-4554:
------------------------------------
Attachment: PIG-4554-4.patch
Fixed the failing tests in Sandeep's patch. [~daijy], can you review so that I
can commit. It is just a minor change from his patch, but since it is a change
need someone else's +1 before commit.
Working on couple of patches (PIG-4653, PIG-4697) to cut down the size of
data sent to Tez AM as we keep hitting RPC limit exceeded issue for many
scripts. This is one of the required ones and would like to wrap this up.
> Compress pig.script before encoding
> -----------------------------------
>
> Key: PIG-4554
> URL: https://issues.apache.org/jira/browse/PIG-4554
> Project: Pig
> Issue Type: Improvement
> Affects Versions: 0.14.0
> Reporter: Rohini Palaniswamy
> Assignee: Sandeep Samdaria
> Labels: newbie
> Fix For: 0.16.0
>
> Attachments: PIG-4554-2.patch, PIG-4554-3.patch, PIG-4554-4.patch,
> PIG-4554.patch
>
>
> Currently we truncate the pig script (maxScriptSize = 10240) and base64
> encode it and store in config. We should remove the truncation and store the
> full script by compressing and then doing base64 encoding. We already do that
> for udfcontext, etc. It will save space as it will compress really well and
> will also give the full pig script while debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
