Spark build error

[金国栋]

Hi!

I tried to build spark source code from github, and I successfully built it
from command line using `*sbt/sbt assembly*`. While I encountered an error
when compiling the project in Intellij IDEA(V14.1.5).


The error log is below:
*Error:scala: *
* while compiling:
/Users/ray/Documents/P01_Project/Spark-Github/spark/sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala*
*during phase: jvm*
 library version: version 2.10.5
compiler version: version 2.10.5
  reconstructed args: -nobootcp -javabootclasspath : -deprecation -feature
-classpath

Re: Spark build error

[Ted Yu]

Is the Scala version in Intellij the same as the one used by sbt ?

Cheers

On Tue, Nov 17, 2015 at 6:45 PM, 金国栋  wrote:

> Hi!
>
> I tried to build spark source code from github, and I successfully built
> it from command line using `*sbt/sbt assembly*`. While I encountered an
> error when compiling the project in Intellij IDEA(V14.1.5).
>
>
> The error log is below:
> *Error:scala: *
> * while compiling:
> /Users/ray/Documents/P01_Project/Spark-Github/spark/sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala*
> *during phase: jvm*
>  library version: version 2.10.5
> compiler version: version 2.10.5
>   reconstructed args: -nobootcp -javabootclasspath : -deprecation -feature
> -classpath
> 

Re: Spark build error

[Jeff Zhang]

This also bother me for a long time. I suspect the intellij builder
conflicts with the sbt/maven builder.

I resolve this issue by rebuild spark in intellij.  You may meet
compilation issue when building it in intellij.
For that you need to put external/flume-sink/target/java on the source
build path.



On Wed, Nov 18, 2015 at 12:02 PM, Ted Yu  wrote:

> Is the Scala version in Intellij the same as the one used by sbt ?
>
> Cheers
>
> On Tue, Nov 17, 2015 at 6:45 PM, 金国栋  wrote:
>
>> Hi!
>>
>> I tried to build spark source code from github, and I successfully built
>> it from command line using `*sbt/sbt assembly*`. While I encountered an
>> error when compiling the project in Intellij IDEA(V14.1.5).
>>
>>
>> The error log is below:
>> *Error:scala: *
>> * while compiling:
>> /Users/ray/Documents/P01_Project/Spark-Github/spark/sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala*
>> *during phase: jvm*
>>  library version: version 2.10.5
>> compiler version: version 2.10.5
>>   reconstructed args: -nobootcp -javabootclasspath : -deprecation
>> -feature -classpath
>> 

Spark build error

[Priya Ch]

Hi,

I am trying to build jars using the command :

mvn -Pyarn -Phadoop-2.2 -Dhadoop.version=2.2.0 -DskipTests clean package

Execution of the above command is throwing the following error:

[INFO] Spark Project Core . FAILURE [  0.295 s]
[INFO] Spark Project Bagel  SKIPPED
[INFO] Spark Project GraphX ... SKIPPED
[INFO] Spark Project ML Library ... SKIPPED
[INFO] Spark Project Streaming  SKIPPED
[INFO] Spark Project Tools  SKIPPED
[INFO] Spark Project Catalyst . SKIPPED
[INFO] Spark Project SQL .. SKIPPED
[INFO] Spark Project Hive . SKIPPED
[INFO] Spark Project REPL . SKIPPED
[INFO] Spark Project YARN Parent POM .. SKIPPED
[INFO] Spark Project YARN Stable API .. SKIPPED
[INFO] Spark Project Assembly . SKIPPED
[INFO] Spark Project External Twitter . SKIPPED
[INFO] Spark Project External Kafka ... SKIPPED
[INFO] Spark Project External Flume ... SKIPPED
[INFO] Spark Project External ZeroMQ .. SKIPPED
[INFO] Spark Project External MQTT  SKIPPED
[INFO] Spark Project Examples . SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 3.748 s
[INFO] Finished at: 2014-08-07T01:00:48+05:30
[INFO] Final Memory: 24M/175M
[INFO] 
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(default) on project spark-core_2.10: Execution default of goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
failed: For artifact {null:null:null:jar}: The groupId cannot be
empty. - [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(default) on project spark-core_2.10: Execution default of goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
failed: For artifact {null:null:null:jar}: The groupId cannot be
empty.
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:224)
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at 
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at 
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at 
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:347)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:154)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at 
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at 
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at 
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution
default of goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
failed: For artifact {null:null:null:jar}: The groupId cannot be
empty.
at 
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:143)
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 19 more
Caused by: org.apache.maven.artifact.InvalidArtifactRTException: For
artifact {null:null:null:jar}: The groupId cannot be empty.



Can someone help me on this ?