Re: Intellij IDEA build issues

2013-12-16 Thread Nick Pentreath 
Thanks Evan, I tried it and the new SBT direct import seems to work well,
though I did run into issues with some yarn imports on Spark.

n


On Thu, Dec 12, 2013 at 7:03 PM, Evan Chan  wrote:

> Nick, have you tried using the latest Scala plug-in, which features native
> SBT project imports?   ie you no longer need to run gen-idea.
>
>
> On Sat, Dec 7, 2013 at 4:15 AM, Nick Pentreath  >wrote:
>
> > Hi Spark Devs,
> >
> > Hoping someone cane help me out. No matter what I do, I cannot get
> Intellij
> > to build Spark from source. I am using IDEA 13. I run sbt gen-idea and
> > everything seems to work fine.
> >
> > When I try to build using IDEA, everything compiles but I get the error
> > below.
> >
> > Have any of you come across the same?
> >
> > ==
> >
> > Internal error: (java.lang.AssertionError)
> > java/nio/channels/FileChannel$MapMode already declared as
> > ch.epfl.lamp.fjbg.JInnerClassesAttribute$Entry@1b5b798b
> > java.lang.AssertionError: java/nio/channels/FileChannel$MapMode already
> > declared as ch.epfl.lamp.fjbg.JInnerClassesAttribute$Entry@1b5b798b
> > at
> >
> >
> ch.epfl.lamp.fjbg.JInnerClassesAttribute.addEntry(JInnerClassesAttribute.java:74)
> > at
> >
> >
> scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator$$anonfun$addInnerClasses$3.apply(GenJVM.scala:738)
> > at
> >
> >
> scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator$$anonfun$addInnerClasses$3.apply(GenJVM.scala:733)
> > at
> >
> >
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
> > at scala.collection.immutable.List.foreach(List.scala:76)
> > at
> >
> >
> scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator.addInnerClasses(GenJVM.scala:733)
> > at
> >
> >
> scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator.emitClass(GenJVM.scala:200)
> > at
> >
> >
> scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator.genClass(GenJVM.scala:355)
> > at
> >
> >
> scala.tools.nsc.backend.jvm.GenJVM$JvmPhase$$anonfun$run$4.apply(GenJVM.scala:86)
> > at
> >
> >
> scala.tools.nsc.backend.jvm.GenJVM$JvmPhase$$anonfun$run$4.apply(GenJVM.scala:86)
> > at
> >
> >
> scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:104)
> > at
> >
> >
> scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:104)
> > at scala.collection.Iterator$class.foreach(Iterator.scala:772)
> > at
> scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:157)
> > at
> >
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:190)
> > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:45)
> > at scala.collection.mutable.HashMap$$anon$2.foreach(HashMap.scala:104)
> > at scala.tools.nsc.backend.jvm.GenJVM$JvmPhase.run(GenJVM.scala:86)
> > at scala.tools.nsc.Global$Run.compileSources(Global.scala:953)
> > at scala.tools.nsc.Global$Run.compile(Global.scala:1041)
> > at xsbt.CachedCompiler0.run(CompilerInterface.scala:123)
> > at xsbt.CachedCompiler0.liftedTree1$1(CompilerInterface.scala:99)
> > at xsbt.CachedCompiler0.run(CompilerInterface.scala:99)
> > at xsbt.CompilerInterface.run(CompilerInterface.scala:27)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:601)
> > at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:102)
> > at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:48)
> > at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:41)
> > at
> >
> >
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3$$anonfun$apply$1.apply$mcV$sp(AggressiveCompile.scala:106)
> > at
> >
> >
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3$$anonfun$apply$1.apply(AggressiveCompile.scala:106)
> > at
> >
> >
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3$$anonfun$apply$1.apply(AggressiveCompile.scala:106)
> > at
> >
> >
> sbt.compiler.AggressiveCompile.sbt$compiler$AggressiveCompile$$timed(AggressiveCompile.scala:173)
> > at
> >
> >
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3.apply(AggressiveCompile.scala:105)
> > at
> >
> >
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3.apply(AggressiveCompile.scala:102)
> > at scala.Option.foreach(Option.scala:236)
> > at
> >
> >
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1.apply(AggressiveCompile.scala:102)
> > at
> >
> >
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1.apply(AggressiveCompile.scala:102)
> > at scala.Option.foreach(Option.scala:236)
> > at
> >
> >
> sbt.compiler.AggressiveCompile$$anonfun$6.compileScala$1(AggressiveCompile.scala:102)
> > at
> >
> >
> sbt.compiler.AggressiveComp

Re: Intellij IDEA build issues

2013-12-12 Thread Evan Chan 
Nick, have you tried using the latest Scala plug-in, which features native
SBT project imports?   ie you no longer need to run gen-idea.


On Sat, Dec 7, 2013 at 4:15 AM, Nick Pentreath wrote:

> Hi Spark Devs,
>
> Hoping someone cane help me out. No matter what I do, I cannot get Intellij
> to build Spark from source. I am using IDEA 13. I run sbt gen-idea and
> everything seems to work fine.
>
> When I try to build using IDEA, everything compiles but I get the error
> below.
>
> Have any of you come across the same?
>
> ==
>
> Internal error: (java.lang.AssertionError)
> java/nio/channels/FileChannel$MapMode already declared as
> ch.epfl.lamp.fjbg.JInnerClassesAttribute$Entry@1b5b798b
> java.lang.AssertionError: java/nio/channels/FileChannel$MapMode already
> declared as ch.epfl.lamp.fjbg.JInnerClassesAttribute$Entry@1b5b798b
> at
>
> ch.epfl.lamp.fjbg.JInnerClassesAttribute.addEntry(JInnerClassesAttribute.java:74)
> at
>
> scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator$$anonfun$addInnerClasses$3.apply(GenJVM.scala:738)
> at
>
> scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator$$anonfun$addInnerClasses$3.apply(GenJVM.scala:733)
> at
>
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
> at scala.collection.immutable.List.foreach(List.scala:76)
> at
>
> scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator.addInnerClasses(GenJVM.scala:733)
> at
>
> scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator.emitClass(GenJVM.scala:200)
> at
>
> scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator.genClass(GenJVM.scala:355)
> at
>
> scala.tools.nsc.backend.jvm.GenJVM$JvmPhase$$anonfun$run$4.apply(GenJVM.scala:86)
> at
>
> scala.tools.nsc.backend.jvm.GenJVM$JvmPhase$$anonfun$run$4.apply(GenJVM.scala:86)
> at
>
> scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:104)
> at
>
> scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:104)
> at scala.collection.Iterator$class.foreach(Iterator.scala:772)
> at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:157)
> at
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:190)
> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:45)
> at scala.collection.mutable.HashMap$$anon$2.foreach(HashMap.scala:104)
> at scala.tools.nsc.backend.jvm.GenJVM$JvmPhase.run(GenJVM.scala:86)
> at scala.tools.nsc.Global$Run.compileSources(Global.scala:953)
> at scala.tools.nsc.Global$Run.compile(Global.scala:1041)
> at xsbt.CachedCompiler0.run(CompilerInterface.scala:123)
> at xsbt.CachedCompiler0.liftedTree1$1(CompilerInterface.scala:99)
> at xsbt.CachedCompiler0.run(CompilerInterface.scala:99)
> at xsbt.CompilerInterface.run(CompilerInterface.scala:27)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:102)
> at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:48)
> at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:41)
> at
>
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3$$anonfun$apply$1.apply$mcV$sp(AggressiveCompile.scala:106)
> at
>
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3$$anonfun$apply$1.apply(AggressiveCompile.scala:106)
> at
>
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3$$anonfun$apply$1.apply(AggressiveCompile.scala:106)
> at
>
> sbt.compiler.AggressiveCompile.sbt$compiler$AggressiveCompile$$timed(AggressiveCompile.scala:173)
> at
>
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3.apply(AggressiveCompile.scala:105)
> at
>
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3.apply(AggressiveCompile.scala:102)
> at scala.Option.foreach(Option.scala:236)
> at
>
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1.apply(AggressiveCompile.scala:102)
> at
>
> sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1.apply(AggressiveCompile.scala:102)
> at scala.Option.foreach(Option.scala:236)
> at
>
> sbt.compiler.AggressiveCompile$$anonfun$6.compileScala$1(AggressiveCompile.scala:102)
> at
>
> sbt.compiler.AggressiveCompile$$anonfun$6.apply(AggressiveCompile.scala:151)
> at
> sbt.compiler.AggressiveCompile$$anonfun$6.apply(AggressiveCompile.scala:89)
> at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:39)
> at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:37)
> at sbt.inc.Incremental$.cycle(Incremental.scala:75)
> at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:34)
> at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:33)
> a

Intellij IDEA build issues

2013-12-07 Thread Nick Pentreath 
Hi Spark Devs,

Hoping someone cane help me out. No matter what I do, I cannot get Intellij
to build Spark from source. I am using IDEA 13. I run sbt gen-idea and
everything seems to work fine.

When I try to build using IDEA, everything compiles but I get the error
below.

Have any of you come across the same?

==

Internal error: (java.lang.AssertionError)
java/nio/channels/FileChannel$MapMode already declared as
ch.epfl.lamp.fjbg.JInnerClassesAttribute$Entry@1b5b798b
java.lang.AssertionError: java/nio/channels/FileChannel$MapMode already
declared as ch.epfl.lamp.fjbg.JInnerClassesAttribute$Entry@1b5b798b
at
ch.epfl.lamp.fjbg.JInnerClassesAttribute.addEntry(JInnerClassesAttribute.java:74)
at
scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator$$anonfun$addInnerClasses$3.apply(GenJVM.scala:738)
at
scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator$$anonfun$addInnerClasses$3.apply(GenJVM.scala:733)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
at scala.collection.immutable.List.foreach(List.scala:76)
at
scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator.addInnerClasses(GenJVM.scala:733)
at
scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator.emitClass(GenJVM.scala:200)
at
scala.tools.nsc.backend.jvm.GenJVM$BytecodeGenerator.genClass(GenJVM.scala:355)
at
scala.tools.nsc.backend.jvm.GenJVM$JvmPhase$$anonfun$run$4.apply(GenJVM.scala:86)
at
scala.tools.nsc.backend.jvm.GenJVM$JvmPhase$$anonfun$run$4.apply(GenJVM.scala:86)
at
scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:104)
at
scala.collection.mutable.HashMap$$anon$2$$anonfun$foreach$3.apply(HashMap.scala:104)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:157)
at
scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:190)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:45)
at scala.collection.mutable.HashMap$$anon$2.foreach(HashMap.scala:104)
at scala.tools.nsc.backend.jvm.GenJVM$JvmPhase.run(GenJVM.scala:86)
at scala.tools.nsc.Global$Run.compileSources(Global.scala:953)
at scala.tools.nsc.Global$Run.compile(Global.scala:1041)
at xsbt.CachedCompiler0.run(CompilerInterface.scala:123)
at xsbt.CachedCompiler0.liftedTree1$1(CompilerInterface.scala:99)
at xsbt.CachedCompiler0.run(CompilerInterface.scala:99)
at xsbt.CompilerInterface.run(CompilerInterface.scala:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:102)
at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:48)
at sbt.compiler.AnalyzingCompiler.compile(AnalyzingCompiler.scala:41)
at
sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3$$anonfun$apply$1.apply$mcV$sp(AggressiveCompile.scala:106)
at
sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3$$anonfun$apply$1.apply(AggressiveCompile.scala:106)
at
sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3$$anonfun$apply$1.apply(AggressiveCompile.scala:106)
at
sbt.compiler.AggressiveCompile.sbt$compiler$AggressiveCompile$$timed(AggressiveCompile.scala:173)
at
sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3.apply(AggressiveCompile.scala:105)
at
sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1$$anonfun$apply$3.apply(AggressiveCompile.scala:102)
at scala.Option.foreach(Option.scala:236)
at
sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1.apply(AggressiveCompile.scala:102)
at
sbt.compiler.AggressiveCompile$$anonfun$6$$anonfun$compileScala$1$1.apply(AggressiveCompile.scala:102)
at scala.Option.foreach(Option.scala:236)
at
sbt.compiler.AggressiveCompile$$anonfun$6.compileScala$1(AggressiveCompile.scala:102)
at
sbt.compiler.AggressiveCompile$$anonfun$6.apply(AggressiveCompile.scala:151)
at
sbt.compiler.AggressiveCompile$$anonfun$6.apply(AggressiveCompile.scala:89)
at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:39)
at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:37)
at sbt.inc.Incremental$.cycle(Incremental.scala:75)
at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:34)
at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:33)
at sbt.inc.Incremental$.manageClassfiles(Incremental.scala:42)
at sbt.inc.Incremental$.compile(Incremental.scala:33)
at sbt.inc.IncrementalCompile$.apply(Compile.scala:27)
at sbt.compiler.AggressiveCompile.compile2(AggressiveCompile.scala:164)
at sbt.compiler.AggressiveCompile.compile1(AggressiveCompile.scala:73)
at
org.jetbrains.jps.incremental.scala.local.CompilerImpl.compile(CompilerImpl.scala:61)
at
org.jetbrains.j