Thanks Luciano.
This seems to work now although some spurious errors!
*spark-submit --packages amplab:succinct:0.1.5 --class "SimpleApp" --master
local target/scala-2.10/simple-project_2.10-1.0.jar*
Ivy Default Cache set to: /home/hduser/.ivy2/cache
The jars for the packages stored in: /home/hduser/.ivy2/jars
:: loading settings :: url =
jar:file:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/spark-assembly-1.5.2-hadoop2.6.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
amplab#succinct added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
confs: [default]
found amplab#succinct;0.1.5 in list
found junit#junit;3.8.1 in list
found net.sf.trove4j#trove4j;3.0.3 in list
:: resolution report :: resolve 314ms :: artifacts dl 13ms
:: modules in use:
amplab#succinct;0.1.5 from list in [default]
junit#junit;3.8.1 from list in [default]
net.sf.trove4j#trove4j;3.0.3 from list in [default]
---------------------------------------------------------------------
| | modules || artifacts
|
| conf | number| search|dwnlded|evicted||
number|dwnlded|
---------------------------------------------------------------------
| default | 3 | 0 | 0 | 0 || 3 | 0
|
---------------------------------------------------------------------
:: problems summary ::
:::: ERRORS
unknown resolver sbt-chain
unknown resolver sbt-chain
unknown resolver sbt-chain
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
:: retrieving :: org.apache.spark#spark-submit-parent
confs: [default]
0 artifacts copied, 3 already retrieved (0kB/9ms)
Some(/home/hduser/dba/bin/scala/target/scala-2.10/simple-project_2.10-1.0.jar)
Lines with a: 60, Lines with b: 29
[B@3b81c435
>>>
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 4 March 2016 at 16:08, Luciano Resende <luckbr1...@gmail.com> wrote:
> Have you tried adding it as --packages at the beginning of you
> spark-submit ?
>
> --packages amplab:succinct:0.1.5
>
> Also, I would usually have the Spark dependencies as "provided" in the
> build.sbt
>
>
> On Fri, Mar 4, 2016 at 6:46 AM, Mich Talebzadeh <mich.talebza...@gmail.com
> > wrote:
>
>> Hi,
>>
>> I have a simple Scala code that I want to use it in an sbt project.
>>
>> It is pretty simple but imports the following:
>>
>> // Import SuccinctRDD
>> import edu.berkeley.cs.succinct._
>>
>>
>> name := "Simple Project"
>> version := "1.0"
>> scalaVersion := "2.10.5"
>> libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.0"
>> libraryDependencies += "org.apache.spark" %% "spark-sql" % "1.0.0"
>> libraryDependencies += "org.apache.spark" %% "spark-hive" % "1.5.0"
>> libraryDependencies += "amplab" % "succinct" % "0.1.5"
>>
>> The jar file is created OK but at execution time it fails
>>
>>
>> *sbt package*[info] Set current project to Simple Project (in build
>> file:/home/hduser/dba/bin/scala/)
>> [success] Total time: 1 s, completed Mar 4, 2016 2:50:16 PM
>>
>> *spark-submit --class "SimpleApp" --master local
>> target/scala-2.10/simple-project_2.10-1.0.jar*
>> Some(/home/hduser/dba/bin/scala/target/scala-2.10/simple-project_2.10-1.0.jar)
>> Lines with a: 60, Lines with b: 29
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> edu/berkeley/cs/succinct/package$
>>
>>
>> I suspect I have not got the Library dependencies corrects?
>>
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn *
>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>>
>
>
>
> --
> Luciano Resende
> http://people.apache.org/~lresende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>
