[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17302581#comment-17302581 ] Zhenbo Li commented on PIG-2599: I tested [~nielsbasjes]'s script After several changes, it's okay to compile pig-script as a maven project [https://github.com/Endle/pig] Is this an open ticket now? > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando >Priority: Major > Labels: gsoc2014 > Fix For: 0.18.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511345#comment-16511345 ] Rohini Palaniswamy commented on PIG-2599: - [~nielsbasjes], Will look over the proposal and comment on it today or tomorrow. bq. I still think Gradle is a way to go since now it's 2018. My vote is for maven. One reason is that all hadoop projects are in maven. But the main one is PIG-4539 which is a better version of pigunit (yaml configuration based testing) with maven integration that we use internally and it would be nice if it could be made as part of Pig itself. > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando >Priority: Major > Labels: gsoc2014 > Fix For: 0.18.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16507161#comment-16507161 ] Vlad Korolev commented on PIG-2599: --- It's been on my todo list for over a year. I still think Gradle is a way to go since now it's 2018. Let me know if you want to cooperate on this > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando >Priority: Major > Labels: gsoc2014 > Fix For: 0.18.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506911#comment-16506911 ] Niels Basjes commented on PIG-2599: --- HI all, I decided to try and see how far I would get with this and also try to understand why this (for me) very desirable change remains stuck for so long.. My current attempt is based upon previous attempts by others and myself and can be found here [https://github.com/nielsbasjes/MavenizePig] What I have right now is that I am able to compile and build Pig as a normal multi module Maven project with Spark 2 (not spark 1 yet). At this point some tests work and many do not yet work. Or they do work when run in isolation but not as part of a full build cycle (due to static MiniGenericCluster instance). There are however a few situations that are hard in Maven and easy in Ant: * Building against multiple versions of (for example) Spark. So my current attempt is only capable of building with Spark 2. * Running the unit tests that are MR/Tez/Spark specific. Here the specific subset of tests is started with a specific environment variable. Now the problem is that this variable gets stuck in the MiniGenericCluster as the static variable INSTANCE and as such running Tez/Spark/Mr tests sequentially becomes really hard. Now in order to make this actually work I propose the following steps # Step by step change the tests to no longer rely on ant and those nasty environment variables. These should be commits that can just be committed. # Then as a 'big bang' rearrange the entire source tree with a script similar to what I have right now. # Have a few small 'cleanup' commits if needed. I think the spark shims problem can be solved by making several version dedicated modules that eventually get shaded into the final jar file. This way we can have several steps that all keeps the code compiling and only a single "massive tree change" that rearranges everything. Now the key question is: Do you guys agree with this? Do you think this would work? If so then I'm going to actually try to see how far I can get and put up some patches. > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando >Priority: Major > Labels: gsoc2014 > Fix For: 0.18.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15390861#comment-15390861 ] Vlad Korolev commented on PIG-2599: --- Ok, I started to work on this. Here is what I got so far https://github.com/vladistan/pig/tree/gradle-conv It is only primary steps. My approach is to import the Ant build script into Gradle and slowly convert tasks one-by-one. Also, I am planning to keep build.xml which will simply forward all the tasks into Gradle. This is the least intrusive way and all the old CI servers and other infrastructure will not break. At this point I am looking for initial critique of the approach and suggestions. Reason to choose Gradle over maven is that the existing Ant script is quite complex for Maven's simple declarative approach. Gradle is as flexible as Ant so there will not be a problem in this regard. > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando > Labels: gsoc2014 > Fix For: 0.17.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15268822#comment-15268822 ] Niels Basjes commented on PIG-2599: --- The current 'ant' setup is really hard to build correctly and near impossible to load into IntelliJ to do any debugging on a piggybank feature. So yes, from me there is a wish to have this. I think the most important thing is that "it just works" when loading the project from scratch into something like the IntelliJ IDE. >From my perspective Maven would be fine. > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando > Labels: gsoc2014 > Fix For: 0.17.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15130470#comment-15130470 ] Eyal Allweil commented on PIG-2599: --- I've worked with countless Maven projects, and it almost always works out-of-the-box. I've worked with one Gradle project - DataFu - and I couldn't get it to build either on Windows or Linux. My vote is for Maven, but I feel embarrassed to write it, since I'm not the one offering to continue work on this feature. > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando > Labels: gsoc2014 > Fix For: 0.16.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15116551#comment-15116551 ] Vlad Korolev commented on PIG-2599: --- Is there still interest in it. I am getting back into hacking on the PIG and was thinking about converting the builds to Gradle. Gradle is newer tool and has strengths of both Maven and PIG. Anybody wants to chime in on this? > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando > Labels: gsoc2014 > Fix For: 0.16.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225006#comment-14225006 ] Daniel Dai commented on PIG-2599: - [~brocknoland], I don't think anyone is actively working on it. Do you have interest to help? > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando > Labels: gsoc2014 > Fix For: 0.15.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14205321#comment-14205321 ] Brock Noland commented on PIG-2599: --- I see there has not been much progress on this project in a while. Is someone still planning on working on this? > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando > Labels: gsoc2014 > Fix For: 0.15.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096025#comment-14096025 ] Giridharan Kesavan commented on PIG-2599: - [~vimuth] [~cheolsoo] this is awesome, let me take a look > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando > Labels: gsoc2014 > Fix For: 0.14.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PIG-2599) Mavenize Pig
[
https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096017#comment-14096017
]
Cheolsoo Park commented on PIG-2599:
Ok, I merged trunk into maven branch one more time. Now {{mvn clean install}}
succeeds except piggybank with {{\-Phadoop2}} that fails with the following
error-
{code}
[ERROR]
/Users/cheolsoop/workspace/pig-svn-maven/piggybank/src/main/java/org/apache/pig/piggybank/storage/partition/PathPartitionHelper.java:[183,50]
cannot find symbol
[ERROR] symbol: method newInstance()
[ERROR] location: class javax.el.ExpressionFactory
[ERROR] -> [Help 1]
{code}
I believe that this is because Hadoop 1 and 2 pull in different versions of jsp
jars.
> Mavenize Pig
>
>
> Key: PIG-2599
> URL: https://issues.apache.org/jira/browse/PIG-2599
> Project: Pig
> Issue Type: New Feature
> Components: build
>Reporter: Daniel Dai
>Assignee: Vimuth Fernando
> Labels: gsoc2014
> Fix For: 0.14.0
>
> Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml
>
>
> Switch Pig build system from ant to maven.
> This is a candidate project for Google summer of code 2014. More information
> about the program can be found at
> https://cwiki.apache.org/confluence/display/PIG/GSoc2014
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095911#comment-14095911 ] Daniel Dai commented on PIG-2599: - Thanks [~vimuth], [~cheolsoo], that's a really good start! > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai >Assignee: Vimuth Fernando > Labels: gsoc2014 > Fix For: 0.14.0 > > Attachments: PIG-2599-wip.zip, maven-pig.1.zip, maven-wip.xml > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PIG-2599) Mavenize Pig
[
https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043909#comment-14043909
]
Cheolsoo Park commented on PIG-2599:
Thank you Vimuth. I can run {{mvn package}} with your pom.
Besides adding build profiles (hadoop 1/2, jar, jar-withouthadoop, jar-all), we
need to migrate test/package targets such as test, test-commit, e2e-test,
package, etc. Also, we will need a pom to build piggybank.
Next time, please upload a diff file named as {{PIG-2599-?.patch}}. It would be
nice if you could also delete ant/ivy files moving forward.
> Mavenize Pig
>
>
> Key: PIG-2599
> URL: https://issues.apache.org/jira/browse/PIG-2599
> Project: Pig
> Issue Type: New Feature
> Components: build
>Reporter: Daniel Dai
>Assignee: Cheolsoo Park
> Labels: gsoc2014
> Fix For: 0.14.0
>
> Attachments: maven-pig.1.zip, maven-wip.xml
>
>
> Switch Pig build system from ant to maven.
> This is a candidate project for Google summer of code 2014. More information
> about the program can be found at
> https://cwiki.apache.org/confluence/display/PIG/GSoc2014
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935986#comment-13935986 ] Vimuth commented on PIG-2599: - Hi all, Im a third year computer science student from the university of moratuwa. Im interested in taking this on as my gsoc project. I've already checked out the code and built it and followed some of the tutorials. Is there any possible mentors for this project? I want to discuss how to proceed further and get some help on my proposal. Thanks > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai > Labels: gsoc2014 > Fix For: 0.13.0 > > Attachments: maven-pig.1.zip > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2014. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2014 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13525797#comment-13525797 ] Julien Le Dem commented on PIG-2599: Hey, that sounds like a great start. Usually attaching a patch to the JIRA and optionnaly posting a review to https://reviews.apache.org/dashboard/ is how a review is done. If you need help you can ping the pig-dev mailing list or comment on the JIRA. a shell script sounds good to me. Is it intended to be a one time move that is then checked in to replace the current layout? Why do you need jdo to be installed in your local maven repo, isn't maven going to do it? could you provide a short description of each folder? Not all of them are clear to me. Do you deal with hadoop 20 vs 23 ? I think zebra has had issues for a while. I'm not sure what the status of this is right now. Maybe Olga knows. fixing checkstyle and findbugs later sound ok to me. It should be relatively easy to do. what about the shim layer ? Anyways, thanks for looking into this. > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai > Labels: gsoc2012 > Attachments: maven-pig.1.zip > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2012. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2012 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506578#comment-13506578 ] major seitan commented on PIG-2599: --- [coming soon to a theater near you] [maven-pig.1.zip|https://issues.apache.org/jira/secure/attachment/12555367/maven-pig.1.zip] Attached is a preview of the effort to port pig from ant to maven in hopes of getting feedback. So if folks can kick the tires, it would be great. q. Is there a recommended way to do such review? q. Is there someone that I can correspond with if I need help? [no code was harmed] The approach taken was to migrate the build.xml to a set of pom.xml. A shell script is used to move around directories and fields, and no code was altered in the migration. So far the pom's are 100% maven and no ant is used. q. Should it be ant instead of a shell script? [let's get the party started] Instructions: tested : Ubuntu, MacOS 1. Set JAVA_HOME (tools.jar) 2. Download [jdo2-api-2.3-ec.jar|http://www.datanucleus.org/downloads/maven2/javax/jdo/jdo2-api/2.3-ec/jdo2-api-2.3-ec.jar] 3. mvn install:install-file -DgroupId=javax.jdo -DartifactId=jdo2-api -Dversion=2.3-ec -Dpackaging=jar -Dfile=jdo2-api-2.3-ec.jar 4. Create a work directory e.g. mkdir pig-2599 && cd pig-2599 5. Download and unzip : maven-pig.1.zip 6. Run ./mvn-pig.sh 2>&1 [remains of the day] If the script ran sucessfuly there will be two directories: pig-ant : containing the ant version pig-mvn : containing the mvn version The project has the following layout pig (pig-mvn) - pig-core - pig-bzip2 - pig-shock - pig-piggybank - pig-zebra - pig-tutorial q. Are the names reasonable? q. Is the layout reasonable? [issues] There is work to be done and feedback is appreciated. Here are some of the known issues: - pig-core needs the "mvn clean test" target to run the tests - jdo2-api2.3-jar has to be manually installed - e2e, documentation are not migrated yet - zebra tests are not completed (also zebra fails to compile) - checkstyle, findbugs not supported q. Did I miss any thing? > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai > Labels: gsoc2012 > Attachments: maven-pig.1.zip > > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2012. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2012 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503940#comment-13503940 ] Julien Le Dem commented on PIG-2599: Another way to incrementally change the build is to switch to the Maven Ant Tasks for dependency resolution in the current ant based build. > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai > Labels: gsoc2012 > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2012. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2012 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503938#comment-13503938 ] Julien Le Dem commented on PIG-2599: for the shim layer refactoring you can use this jira: PIG-2686 If you want to change the source level compatibility you should open a JIRA for this. > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai > Labels: gsoc2012 > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2012. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2012 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503325#comment-13503325 ] major seitan commented on PIG-2599: --- Did some digging looks like the shims issue is PIG-2388 The source level is 1.5 build.xml: but org.apache.hadoop:hadoop-annotations:jar:2.0.0-alpha requires jdk.tools:jdk.tools:jar:1.6 what is the recommended source to set the compiler to in maven? > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai > Labels: gsoc2012 > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2012. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2012 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2599) Mavenize Pig
[
https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503317#comment-13503317
]
major seitan commented on PIG-2599:
---
As maven is biased towards having one src directory
the shims used to support hadoop23 and hadoop20
most likely have to be changed. I would like to
recommend the refactoring the shims to be maven sub
projects corresponding to implementations needed for
the hadoop versions. (e.g.).
pig-hadoop23
pig-hadoop20
The following context with be placed in the main
pig project
{code:title=ShimContext.java|borderStyle=solid}
ShimContext {
// handle the reflection errors
private static final IShimFactory SHIM_FACTORY =
Class.forName("org.apache.pig.backend.hadoop.executionengine.shims.ShimFactoryImpl").newInstance();
static IShimFactory getInstance(){ return SHIM_FACTORY; }
}
{code}
And each project will have its own ShimFactoryImpl implementating
IShimFactory. Where IShimFactory gives access to the the implementations
of the classes (and inner classes) :
PigJobControl
PigMapReduce
PigMapBase
HadoopShims
This is a substantial change so I would need some consensus/direction on how
best to proceed.
> Mavenize Pig
>
>
> Key: PIG-2599
> URL: https://issues.apache.org/jira/browse/PIG-2599
> Project: Pig
> Issue Type: New Feature
> Components: build
>Reporter: Daniel Dai
> Labels: gsoc2012
>
> Switch Pig build system from ant to maven.
> This is a candidate project for Google summer of code 2012. More information
> about the program can be found at
> https://cwiki.apache.org/confluence/display/PIG/GSoc2012
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247476#comment-13247476 ] Daniel Dai commented on PIG-2599: - Looks good, need to make sure current targets works: 1. build and package for both hadoop 20 and 23 2. run unit tests, e2e tests 3. build tarball, rpm/deb 4. publish maven artifacts Looking forward to see your proposal. > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai > Labels: gsoc2012 > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2012. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2012 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2599) Mavenize Pig
[ https://issues.apache.org/jira/browse/PIG-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13244778#comment-13244778 ] Dimitris Bousis commented on PIG-2599: -- Hi all, My name is Dimitris Bousis, currently doing my Master in Computer Engineering & Informatics in University of Patras, Greece. My research interests include cloud & distributed computing with related technologies such as Hadoop, HBase, Cassandra , Pig & Hive. Though i have not started any research activity with the technologies (I plan to do so after the summer), i have taken an elective course in Hadoop,HDFS, HBase & Cassandra during my undergraduate studies. I am interested in applying for this GsoC 2012 project. I have used Maven in several Jave EE projects i have been involved and i have some experience with Ant as well. I downloaded Pig and explored it's source. Switching to maven would roughly consist of these main tasks : 1) Provide projects dependencies in the main pom.xml file taken from the Ivy's configuration. 2) Process any required resources. 3) Build pig and any required components with the compile scope. 4) Build and run pig tests 5) Provide a packaged version of pig for component reuse through the maven repository distribution. Please comment this post for anything you consider necessary. Looking forward working with you this summer. Dimitris Bousis > Mavenize Pig > > > Key: PIG-2599 > URL: https://issues.apache.org/jira/browse/PIG-2599 > Project: Pig > Issue Type: New Feature > Components: build >Reporter: Daniel Dai > Labels: gsoc2012 > > Switch Pig build system from ant to maven. > This is a candidate project for Google summer of code 2012. More information > about the program can be found at > https://cwiki.apache.org/confluence/display/PIG/GSoc2012 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
