RE: Re: Spark assembly in Maven repo?

2015-12-14 Thread Xiaoyong Zhu
Thanks for the info! Xiaoyong From: Sean Owen [mailto:so...@cloudera.com] Sent: Monday, December 14, 2015 12:20 AM To: Xiaoyong Zhu Cc: user Subject: Re: Re: Spark assembly in Maven repo? Yes, though I think the Maven Central repository is more canonical. http://repo1.maven.org/maven2/org

Re: Re: Spark assembly in Maven repo?

2015-12-14 Thread Sean Owen
> > https://repository.apache.org/content/repositories/releases/org/apache/spark/spark-core_2.10/1.5.1/ > > > > Xiaoyong > > > > *From:* Sean Owen [mailto:so...@cloudera.com] > *Sent:* Saturday, December 12, 2015 12:45 AM > *To:* Xiaoyong Zhu > *Cc:* user >

RE: Re: Spark assembly in Maven repo?

2015-12-13 Thread Xiaoyong Zhu
: user Subject: Re: Re: Spark assembly in Maven repo? That's exactly what the various artifacts in the Maven repo are for. The API classes for core are in the core artifact and so on. You don't need an assembly. On Sat, Dec 12, 2015 at 12:32 AM, Xiaoyong Zhu mailto:xiaoy...@microsoft.c

Re: Re: Spark assembly in Maven repo?

2015-12-12 Thread Sean Owen
* Mark Hamstra > *Cc:* Xiaoyong Zhu ; Jeff Zhang ; > user ; Zhaomin Xu ; Joe Zhang > (SDE) > *Subject:* Re: Re: Spark assembly in Maven repo? > > > > Agree with you that assembly jar is not good to publish. However, what he > really need is to fetch > > an up

RE: Re: Spark assembly in Maven repo?

2015-12-11 Thread Xiaoyong Zhu
To: Mark Hamstra Cc: Xiaoyong Zhu ; Jeff Zhang ; user ; Zhaomin Xu ; Joe Zhang (SDE) Subject: Re: Re: Spark assembly in Maven repo? Agree with you that assembly jar is not good to publish. However, what he really need is to fetch an updatable maven jar file

Re: Re: Spark assembly in Maven repo?

2015-12-11 Thread fightf...@163.com
; Joe Zhang (SDE) Subject: RE: Spark assembly in Maven repo? Sorry – I didn’t make it clear. It’s actually not a “dependency” – it’s actually that we are building a certain plugin for IntelliJ where we want to distribute this jar. But since the jar is updated frequently we don't want to distribu

Re: RE: Spark assembly in Maven repo?

2015-12-10 Thread Mark Hamstra
> *To:* Jeff Zhang > *CC:* user@spark.apache.org; Zhaomin Xu ; Joe Zhang > (SDE) > *Subject:* RE: Spark assembly in Maven repo? > > Sorry – I didn’t make it clear. It’s actually not a “dependency” – it’s > actually that we are building a certain plugin for IntelliJ where

Re: RE: Spark assembly in Maven repo?

2015-12-10 Thread fightf...@163.com
; Joe Zhang (SDE) Subject: RE: Spark assembly in Maven repo? Sorry – I didn’t make it clear. It’s actually not a “dependency” – it’s actually that we are building a certain plugin for IntelliJ where we want to distribute this jar. But since the jar is updated frequently we don't want to distr

RE: Spark assembly in Maven repo?

2015-12-10 Thread Xiaoyong Zhu
nload it via Maven. In this case what’s the recommended way? Xiaoyong From: Jeff Zhang [mailto:zjf...@gmail.com] Sent: Thursday, December 10, 2015 11:03 PM To: Xiaoyong Zhu Cc: user@spark.apache.org Subject: Re: Spark assembly in Maven repo? I don't think make the assembly jar as depende

Re: Spark assembly in Maven repo?

2015-12-10 Thread Jeff Zhang
I don't think make the assembly jar as dependency a good practice. You may meet jar hell issue in that case. On Fri, Dec 11, 2015 at 2:46 PM, Xiaoyong Zhu wrote: > Hi Experts, > > > > We have a project which has a dependency for the following jar > > > > spark-assembly--hadoop.jar > > for exampl