[ https://issues.apache.org/jira/browse/PIG-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073536#comment-14073536 ]
Rohini Palaniswamy commented on PIG-4047: ----------------------------------------- Few comments: - pig pom file might require some changes to explicitly specify dependencies as pig-withouthadoop.jar is split up now. I don't see a pig.pom in trunk anymore. Haven't checked how it has been the pom file is produced now, but it would be good to check once if that works fine. - I am wondering if we should still keep the pig-withouthadoop.jar for some time giving folks time to migrate (For eg: In oozie now you will have to copy more jars instead of just the pig-withouthadoop.jar). Getting rid of the fat pig.jar is ok. The new jar should probably be called pig-core.jar so that after mavenization we can have pig-tez.jar and pig-spark.jar. - I like the idea of lib, lib/hadoop1 and lib/hadoop2. What other jars are produced in the base directory apart from pig.jar? Aren't dependent jars under lib. {code} + <fileset dir="${pig.dir}"> + <include name="*.jar"/> + </fileset> {code} > Break up pig withouthadoop and fat jar > -------------------------------------- > > Key: PIG-4047 > URL: https://issues.apache.org/jira/browse/PIG-4047 > Project: Pig > Issue Type: Improvement > Components: build > Affects Versions: site > Reporter: fang fang chen > Assignee: fang fang chen > Labels: build > Fix For: 0.14.0 > > Attachments: PIG-4047-1.patch, PIG-4047-2.patch, PIG-4047.patch > > > pig-withouthadoop jar is packaging pig core and pig core dependencies. But > this jar should be removed due to following items: > 1. the name is confusing. User did not know what the jar is used for at a > glance. > 2. it is not absolutely clear for user what the core dependencies are. > 3. it is hard to maintain dependencies, like dependencies version update. > Maybe user want to try different version avro without repackaging. > It is better to not use pig-withouthadoop jar, instead: > 1. devided without hadoop jar into pig core and pig core dependencies. > 2. save jars in 1# in lib directory > 3. in pig script, always add all the jars in lib directory into classpath and > add pig core jar into classpath. > I used pig in this way since version 0.8.1 via launching pig grunt. No issue > found yet. > Current branch-0.13 is packaging following jars into pig-withouhadoop jar: > <include name="antlr-runtime-${antlr.version}.jar"/> > <include name="ST4-${stringtemplate.version}.jar"/> > <include name="jline-${jline.version}.jar"/> > <include name="jackson-mapper-asl-${jackson.version}.jar"/> > <include name="jackson-core-asl-${jackson.version}.jar"/> > <include name="joda-time-${joda-time.version}.jar"/> > <include name="guava-${guava.version}.jar"/> > <include name="automaton-${automaton.version}.jar"/> > <include name="jansi-${jansi.version}.jar"/> > <include name="avro-${avro.version}.jar"/> > <include name="avro-mapred-${avro.version}.jar"/> > <include name="trevni-core-${avro.version}.jar"/> > <include name="trevni-avro-${avro.version}.jar"/> > <include name="snappy-java-${snappy. version}.jar"/> > We could save upper jars and pig-core jar into lib directory. -- This message was sent by Atlassian JIRA (v6.2#6252)