[ https://issues.apache.org/jira/browse/PIG-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14076924#comment-14076924 ]
Daniel Dai commented on PIG-4054: --------------------------------- PIG-4054-5.patch committed to trunk. Thanks Cheolsoo! > Do not create job.jar when submitting job > ----------------------------------------- > > Key: PIG-4054 > URL: https://issues.apache.org/jira/browse/PIG-4054 > Project: Pig > Issue Type: Improvement > Components: impl > Reporter: Daniel Dai > Assignee: Daniel Dai > Fix For: 0.14.0 > > Attachments: PIG-4054-1.patch, PIG-4054-2.patch, PIG-4054-3.patch, > PIG-4054-4.patch, PIG-4054-5.patch > > > Currently Pig creates job.jar per job when submitting mapreduce job. There > are several disadvantages: > 1. job.jar varies job by job, job.jar will not get reused even if jar cache > is used (PIG-2672). > 2. Before job submission, we need to pack a job.jar which are mostly > repacking of existing jars, this is a waste of time > 3. job.jar is a uber jar which makes debug harder and could lead to jar > conflicting issue (eg, PIG-3039) > On tez side, situation is similar, the consequence is worse since container > will not be reused. > So instead of job.jar, I would like to ship individual jar to distributed > cache. Note this issue is in essence independent of PIG-4047, however, > PIG-4047 would make the picture more complete in that we don't have any uber > jars. -- This message was sent by Atlassian JIRA (v6.2#6252)