I'm still getting acquainted with the Spark ecosystem, and wanted to make sure
my understanding of the different API layers is correct.
Is this an accurate picture of the major API layers, and their associated
client support?
Thanks,
-Lincoln
Spark Core:
- Scala
- Java
-
Sorry, only indirectly Spark-related.
I've attempting to create a .NET proxy for spark-core, using JNI4NET. At the
moment I'm stuck with the following error when running the proxy generator:
java.lang.NoClassDefFoundError:
org.apache.hadoop.mapreduce.lib.input.FileInputFormat
I've resolved
WARN Executor: Told to re-register on heartbeat is logged repeatedly in the
spark shell, which is very distracting and corrupts the display of whatever set
of commands I'm currently typing out.
Is there an option to disable the logging of this message?
Thanks,
-Lincoln
Brilliant! Thanks.
From: Feynman Liang [mailto:fli...@databricks.com]
Sent: Wednesday, July 08, 2015 2:15 PM
To: Lincoln Atkinson
Cc: user@spark.apache.org
Subject: Re: Disable heartbeat messages in REPL
I was thinking the same thing! Try sc.setLogLevel(ERROR)
On Wed, Jul 8, 2015 at 2:01 PM
I'm trying to build Spark from source on Windows 8.1, using a recent Cygwin
install and JDK 8u45. From the root of my enlistment, I'm running `build/mvn
-Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package`
The build moves along just fine for a while, until it builds Spark
That solved it. Thanks!
From: Ted Yu [mailto:yuzhih...@gmail.com]
Sent: Tuesday, July 07, 2015 10:21 AM
To: Lincoln Atkinson
Cc: user@spark.apache.org
Subject: Re: Windows - endless Dependency-reduced POM written... in Bagel
build
Looks like a workaround has gone in:
[SPARK-8819] Fix build
