Patrick, I assume an RC3 will be out for folks like me to test the distribution. As usual, I will run the tests when you have a new distribution. Cheers <k/>
On Fri, Jul 3, 2015 at 4:38 PM, Patrick Wendell <pwend...@gmail.com> wrote: > Patch that added test-jar dependencies: > https://github.com/apache/spark/commit/bfe74b34 > > Patch that originally disabled dependency reduced poms: > > https://github.com/apache/spark/commit/984ad60147c933f2d5a2040c87ae687c14eb1724 > > Patch that reverted the disabling of dependency reduced poms: > > https://github.com/apache/spark/commit/bc51bcaea734fe64a90d007559e76f5ceebfea9e > > On Fri, Jul 3, 2015 at 4:36 PM, Patrick Wendell <pwend...@gmail.com> > wrote: > > Okay I did some forensics with Sean Owen. Some things about this bug: > > > > 1. The underlying cause is that we added some code to make the tests > > of sub modules depend on the core tests. For unknown reasons this > > causes Spark to hit MSHADE-148 for *some* combinations of build > > profiles. > > > > 2. MSHADE-148 can be worked around by disabling building of > > "dependency reduced poms" because then the buggy code path is > > circumvented. Andrew Or did this in a patch on the 1.4 branch. > > However, that is not a tenable option for us because our *published* > > pom files require dependency reduction to substitute in the scala > > version correctly for the poms published to maven central. > > > > 3. As a result, Andrew Or reverted his patch recently, causing some > > package builds to start failing again (but publishing works now). > > > > 4. The reason this is not detected in our test harness or release > > build is that it is sensitive to the profiles enabled. The combination > > of profiles we enable in the test harness and release builds do not > > trigger this bug. > > > > The best path I see forward right now is to do the following: > > > > 1. Disable creation of dependency reduced poms by default (this > > doesn't matter for people doing a package build) so typical users > > won't have this bug. > > > > 2. Add a profile that re-enables that setting. > > > > 3. Use the above profile when publishing release artifacts to maven > central. > > > > 4. Hope that we don't hit this bug for publishing. > > > > - Patrick > > > > On Fri, Jul 3, 2015 at 3:51 PM, Tarek Auel <tarek.a...@gmail.com> wrote: > >> Doesn't change anything for me. > >> > >> On Fri, Jul 3, 2015 at 3:45 PM Patrick Wendell <pwend...@gmail.com> > wrote: > >>> > >>> Can you try using the built in maven "build/mvn..."? All of our builds > >>> are passing on Jenkins so I wonder if it's a maven version issue: > >>> > >>> https://amplab.cs.berkeley.edu/jenkins/view/Spark-QA-Compile/ > >>> > >>> - Patrick > >>> > >>> On Fri, Jul 3, 2015 at 3:14 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >>> > Please take a look at SPARK-8781 > >>> > (https://github.com/apache/spark/pull/7193) > >>> > > >>> > Cheers > >>> > > >>> > On Fri, Jul 3, 2015 at 3:05 PM, Tarek Auel <tarek.a...@gmail.com> > wrote: > >>> >> > >>> >> I found a solution, there might be a better one. > >>> >> > >>> >> https://github.com/apache/spark/pull/7217 > >>> >> > >>> >> On Fri, Jul 3, 2015 at 2:28 PM Robin East <robin.e...@xense.co.uk> > >>> >> wrote: > >>> >>> > >>> >>> Yes me too > >>> >>> > >>> >>> On 3 Jul 2015, at 22:21, Ted Yu <yuzhih...@gmail.com> wrote: > >>> >>> > >>> >>> This is what I got (the last line was repeated non-stop): > >>> >>> > >>> >>> [INFO] Replacing original artifact with shaded artifact. > >>> >>> [INFO] Replacing > >>> >>> /home/hbase/spark/bagel/target/spark-bagel_2.10-1.5.0-SNAPSHOT.jar > >>> >>> with > >>> >>> > >>> >>> > /home/hbase/spark/bagel/target/spark-bagel_2.10-1.5.0-SNAPSHOT-shaded.jar > >>> >>> [INFO] Dependency-reduced POM written at: > >>> >>> /home/hbase/spark/bagel/dependency-reduced-pom.xml > >>> >>> [INFO] Dependency-reduced POM written at: > >>> >>> /home/hbase/spark/bagel/dependency-reduced-pom.xml > >>> >>> > >>> >>> On Fri, Jul 3, 2015 at 1:13 PM, Tarek Auel <tarek.a...@gmail.com> > >>> >>> wrote: > >>> >>>> > >>> >>>> Hi all, > >>> >>>> > >>> >>>> I am trying to build the master, but it stucks and prints > >>> >>>> > >>> >>>> [INFO] Dependency-reduced POM written at: > >>> >>>> /Users/tarek/test/spark/bagel/dependency-reduced-pom.xml > >>> >>>> > >>> >>>> build command: mvn -DskipTests clean package > >>> >>>> > >>> >>>> Do others have the same issue? > >>> >>>> > >>> >>>> Regards, > >>> >>>> Tarek > >>> >>> > >>> >>> > >>> >>> > >>> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > >
