Hello,
I have a Spark app which I run with master local[3]. When running without
any persist calls, it seems to work fine, but as soon as I add persist calls
(at default storage level), it fails at the first persist call with the message
below. Unfortunately, I can't post the code.
Hello,
My SBT pulls in, among others, the following dependency for Spark 1.1.0:
akka-actor_2.10-2.2.3-shaded-protobuf.jar
What is this? How is this different from the regular Akka Actor JAR? How do I
reconcile with other libs that use Akka, such as Play?
Thanks!
Best,
Hello,
Suppose I want to use Spark from an application that I already submit to run
in another container (e.g. Tomcat). Is this at all possible? Or do I have to
split the app into two components, and submit one to Spark and one to the other
container? In that case, what is the
the script. Thanks!
Best, Oliver
From: Matei Zaharia [mailto:matei.zaha...@gmail.com]
Sent: Tuesday, September 16, 2014 1:31 PM
To: Ruebenacker, Oliver A; user@spark.apache.org
Subject: Re: Spark as a Library
If you want to run the computation on just one machine (using Spark's local
mode
, September 04, 2014 2:11 PM
To: Ruebenacker, Oliver A
Cc: Akhil Das; Wonha Ryu; user@spark.apache.org
Subject: Re: Web UI
Hi all,
The JSON version of the web UI is not officially supported; I don't believe
this is documented anywhere.
The alternative is to set `spark.eventLog.enabled` to true
Hello,
Can this be used as a library from within another application?
Thanks!
Best, Oliver
From: Matt Chu [mailto:m...@kabam.com]
Sent: Thursday, September 04, 2014 2:46 AM
To: Vicky Kak
Cc: user
Subject: Re: Programatically running of the Spark Jobs.
, 2014 3:09 AM
To: Ruebenacker, Oliver A
Cc: Wonha Ryu; user@spark.apache.org
Subject: Re: Web UI
Hi
You can see this
dochttps://spark.apache.org/docs/latest/spark-standalone.html#configuring-ports-for-network-security
for all the available webUI ports.
Yes there are ways to get the data metrics
Hello,
Is cluster manager mentioned
herehttps://spark.apache.org/docs/latest/cluster-overview.html the same thing
as master mentioned
herehttps://spark.apache.org/docs/latest/spark-standalone.html#starting-a-cluster-manually?
Thanks!
Best, Oliver
Oliver Ruebenacker | Solutions
Hello,
I'm running Spark on Windows 7 as standalone, with everything on the same
machine. No Hadoop installed. My app throws exception and worker reports:
Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
I had the same problem earlier when deploying local. I
Hello,
In the app below, when I run it with local[1] or local [3], I get the
expected result - a list of the square roots of the numbers from 1 to 20.
When I try the same app as standalone with one or two workers on the same
machine, it will only print 1.0.
Adding print statements
field, it works fine.
Best, Oliver
From: Ruebenacker, Oliver A [mailto:oliver.ruebenac...@altisource.com]
Sent: Thursday, September 04, 2014 4:15 PM
To: user@spark.apache.org
Subject: Reduce truncates RDD in standalone, but fine when local.
Hello,
In the app below, when I run
Hello,
What is included in the Spark web UI? What are the available URLs? Can the
information be obtained in a machine-readable way (e.g. JSON, XML, etc)?
Thanks!
Best,
Oliver
Oliver Ruebenacker | Solutions Architect
Altisource(tm)
290 Congress St, 7th Floor | Boston,
]
Sent: Wednesday, September 03, 2014 3:36 PM
To: Ruebenacker, Oliver A
Cc: user@spark.apache.org
Subject: Re: Web UI
Hi Oliver,
Spark standalone master and worker support '/json' endpoint in web UI, which
returns some of the information in JSON format.
I wasn't able to find relevant documentation
.
Best,
Oliver
From: Wonha Ryu [mailto:wonha@gmail.com]
Sent: Wednesday, September 03, 2014 4:27 PM
To: Ruebenacker, Oliver A
Cc: user@spark.apache.org
Subject: Re: Web UI
Hey Oliver,
IIRC there's no JSON endpoint for application web UI. They only exist for
cluster master and worker
Hello,
If launched with local as master, where are master and workers? Do they
each have a web UI? How can they be monitored?
Thanks!
Best,
Oliver
Oliver Ruebenacker | Solutions Architect
Altisource(tm)
290 Congress St, 7th Floor | Boston, Massachusetts 02210
P: (617)
How can that single process be monitored? Thanks!
-Original Message-
From: Marcelo Vanzin [mailto:van...@cloudera.com]
Sent: Wednesday, September 03, 2014 6:32 PM
To: Ruebenacker, Oliver A
Cc: user@spark.apache.org
Subject: Re: If master is local, where are master and workers?
local
16 matches
Mail list logo