Why not install them? It doesn't take any work and is the only correct way to do it. mvn install is all you need. On Nov 20, 2014 2:35 AM, "Yiming (John) Zhang" <sdi...@gmail.com> wrote:
> Hi Sean, > > Thank you for your reply. I was wondering whether there is a method of > reusing locally-built components without installing them? That is, if I > have successfully built the spark project as a whole, how should I > configure it so that I can incrementally build (only) the "spark-examples" > sub project without the need of downloading or installation? > > Thank you! > > Cheers, > Yiming > > -----邮件原件----- > 发件人: Sean Owen [mailto:so...@cloudera.com] > 发送时间: 2014年11月17日 17:40 > 收件人: yiming zhang > 抄送: Marcelo Vanzin; user@spark.apache.org > 主题: Re: How to incrementally compile spark examples using mvn > > The downloads just happen once so this is not a problem. > > If you are just building one module in a project, it needs a compiled copy > of other modules. It will either use your locally-built and > locally-installed artifact, or, download one from the repo if possible. > > This isn't needed if you are compiling all modules at once. If you want to > compile everything and reuse the local artifacts later, you need 'install' > not 'package'. > > On Mon, Nov 17, 2014 at 12:27 AM, Yiming (John) Zhang <sdi...@gmail.com> > wrote: > > Thank you Marcelo. I tried your suggestion (# mvn -pl > :spark-examples_2.10 compile), but it required to download many spark > components (as listed below), which I have already compiled on my server. > > > > Downloading: > > https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/1.1.0/ > > spark-core_2.10-1.1.0.pom > > ... > > Downloading: > > https://repo1.maven.org/maven2/org/apache/spark/spark-streaming_2.10/1 > > .1.0/spark-streaming_2.10-1.1.0.pom > > ... > > Downloading: > > https://repository.jboss.org/nexus/content/repositories/releases/org/a > > pache/spark/spark-hive_2.10/1.1.0/spark-hive_2.10-1.1.0.pom > > ... > > > > This problem didn't happen when I compiled the whole project using ``mvn > -DskipTests package''. I guess some configurations have to be made to tell > mvn the dependencies are local. Any idea for that? > > > > Thank you for your help! > > > > Cheers, > > Yiming > > > > -----邮件原件----- > > 发件人: Marcelo Vanzin [mailto:van...@cloudera.com] > > 发送时间: 2014年11月16日 10:26 > > 收件人: sdi...@gmail.com > > 抄送: user@spark.apache.org > > 主题: Re: How to incrementally compile spark examples using mvn > > > > I haven't tried scala:cc, but you can ask maven to just build a > particular sub-project. For example: > > > > mvn -pl :spark-examples_2.10 compile > > > > On Sat, Nov 15, 2014 at 5:31 PM, Yiming (John) Zhang <sdi...@gmail.com> > wrote: > >> Hi, > >> > >> > >> > >> I have already successfully compile and run spark examples. My > >> problem is that if I make some modifications (e.g., on SparkPi.scala > >> or > >> LogQuery.scala) I have to use “mvn -DskipTests package” to rebuild > >> the whole spark project and wait a relatively long time. > >> > >> > >> > >> I also tried “mvn scala:cc” as described in > >> http://spark.apache.org/docs/latest/building-with-maven.html, but I > >> could only get infinite stop like: > >> > >> [INFO] --- scala-maven-plugin:3.2.0:cc (default-cli) @ spark-parent > >> --- > >> > >> [INFO] wait for files to compile... > >> > >> > >> > >> Is there any method to incrementally compile the examples using mvn? > >> Thank you! > >> > >> > >> > >> Cheers, > >> > >> Yiming > > > > > > > > -- > > Marcelo > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For > > additional commands, e-mail: user-h...@spark.apache.org > > > >
