Hello there, So I just took a quick look at the pom and I see two problems with it.
- "activatedByDefault" does not work like you think it does. It only "activates by default" if you do not explicitly activate other profiles. So if you do "mvn package", scala-2.10 will be activated; but if you do "mvn -Pyarn package", it will not. - you need to duplicate the "activation" stuff everywhere where the profile is declared, not just in the root pom. (I spent quite some time yesterday fighting a similar issue...) My suggestion here is to change the activation of scala-2.10 to look like this: <activation> <property> <name>!scala-2.11</name> </property> </activation> And change the scala-2.11 profile to do this: <properties> <scala-2.11>true</scala-2.11> </properties> I haven't tested, but in my experience this will activate the scala-2.10 profile by default, unless you explicitly activate the 2.11 profile, in which case that property will be set and scala-2.10 will not activate. If you look at examples/pom.xml, that's the same strategy used to choose which hbase profile to activate. Ah, and just to reinforce, the activation logic needs to be copied to other places (e.g. examples/pom.xml, repl/pom.xml, and any other place that has scala-2.x profiles). On Wed, Nov 12, 2014 at 11:14 PM, Patrick Wendell <pwend...@gmail.com> wrote: > I actually do agree with this - let's see if we can find a solution > that doesn't regress this behavior. Maybe we can simply move the one > kafka example into its own project instead of having it in the > examples project. > > On Wed, Nov 12, 2014 at 11:07 PM, Sandy Ryza <sandy.r...@cloudera.com> wrote: >> Currently there are no mandatory profiles required to build Spark. I.e. >> "mvn package" just works. It seems sad that we would need to break this. >> >> On Wed, Nov 12, 2014 at 10:59 PM, Patrick Wendell <pwend...@gmail.com> >> wrote: >>> >>> I think printing an error that says "-Pscala-2.10 must be enabled" is >>> probably okay. It's a slight regression but it's super obvious to >>> users. That could be a more elegant solution than the somewhat >>> complicated monstrosity I proposed on the JIRA. >>> >>> On Wed, Nov 12, 2014 at 10:37 PM, Prashant Sharma <scrapco...@gmail.com> >>> wrote: >>> > One thing we can do it is print a helpful error and break. I don't know >>> > about how this can be done, but since now I can write groovy inside >>> > maven >>> > build so we have more control. (Yay!!) >>> > >>> > Prashant Sharma >>> > >>> > >>> > >>> > On Thu, Nov 13, 2014 at 12:05 PM, Patrick Wendell <pwend...@gmail.com> >>> > wrote: >>> >> >>> >> Yeah Sandy and I were chatting about this today and din't realize >>> >> -Pscala-2.10 was mandatory. This is a fairly invasive change, so I was >>> >> thinking maybe we could try to remove that. Also if someone doesn't >>> >> give -Pscala-2.10 it fails in a way that is initially silent, which is >>> >> bad because most people won't know to do this. >>> >> >>> >> https://issues.apache.org/jira/browse/SPARK-4375 >>> >> >>> >> On Wed, Nov 12, 2014 at 10:29 PM, Prashant Sharma >>> >> <scrapco...@gmail.com> >>> >> wrote: >>> >> > Thanks Patrick, I have one suggestion that we should make passing >>> >> > -Pscala-2.10 mandatory for maven users. I am sorry for not mentioning >>> >> > this >>> >> > before. There is no way around not passing that option for maven >>> >> > users(only). However, this is unnecessary for sbt users because it is >>> >> > added >>> >> > automatically if -Pscala-2.11 is absent. >>> >> > >>> >> > >>> >> > Prashant Sharma >>> >> > >>> >> > >>> >> > >>> >> > On Wed, Nov 12, 2014 at 3:53 PM, Sean Owen <so...@cloudera.com> >>> >> > wrote: >>> >> > >>> >> >> - Tip: when you rebase, IntelliJ will temporarily think things like >>> >> >> the >>> >> >> Kafka module are being removed. Say 'no' when it asks if you want to >>> >> >> remove >>> >> >> them. >>> >> >> - Can we go straight to Scala 2.11.4? >>> >> >> >>> >> >> On Wed, Nov 12, 2014 at 5:47 AM, Patrick Wendell >>> >> >> <pwend...@gmail.com> >>> >> >> wrote: >>> >> >> >>> >> >> > Hey All, >>> >> >> > >>> >> >> > I've just merged a patch that adds support for Scala 2.11 which >>> >> >> > will >>> >> >> > have some minor implications for the build. These are due to the >>> >> >> > complexities of supporting two versions of Scala in a single >>> >> >> > project. >>> >> >> > >>> >> >> > 1. The JDBC server will now require a special flag to build >>> >> >> > -Phive-thriftserver on top of the existing flag -Phive. This is >>> >> >> > because some build permutations (only in Scala 2.11) won't support >>> >> >> > the >>> >> >> > JDBC server yet due to transitive dependency conflicts. >>> >> >> > >>> >> >> > 2. The build now uses non-standard source layouts in a few >>> >> >> > additional >>> >> >> > places (we already did this for the Hive project) - the repl and >>> >> >> > the >>> >> >> > examples modules. This is just fine for maven/sbt, but it may >>> >> >> > affect >>> >> >> > users who import the build in IDE's that are using these projects >>> >> >> > and >>> >> >> > want to build Spark from the IDE. I'm going to update our wiki to >>> >> >> > include full instructions for making this work well in IntelliJ. >>> >> >> > >>> >> >> > If there are any other build related issues please respond to this >>> >> >> > thread and we'll make sure they get sorted out. Thanks to Prashant >>> >> >> > Sharma who is the author of this feature! >>> >> >> > >>> >> >> > - Patrick >>> >> >> > >>> >> >> > >>> >> >> > --------------------------------------------------------------------- >>> >> >> > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >>> >> >> > For additional commands, e-mail: dev-h...@spark.apache.org >>> >> >> > >>> >> >> > >>> >> >> >>> > >>> > >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >>> For additional commands, e-mail: dev-h...@spark.apache.org >>> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org