The UI code is the same in both, but one possibility is that your executors
were given less memory on YARN. Can you check that? Or otherwise, how do you
know that some RDDs were cached?
Matei
On Jul 12, 2014, at 4:12 PM, Koert Kuipers ko...@tresata.com wrote:
hey shuo,
so far all stage
I just tested a long lived application (that we normally run in standalone
mode) on yarn in client mode.
it looks to me like cached rdds are missing in the storage tap of the ui.
accessing the rdd storage information via the spark context shows rdds as
fully cached but they are missing on
i was testing using the acl for spark ui in secure mode on yarn in client
mode.
it works great. my spark 1.0.0 configuration has:
spark.authenticate = true
spark.ui.acls.enable = true
spark.ui.view.acls = koert
spark.ui.filters
About more succeeded tasks than total tasks:
- This can happen if you have enabled speculative execution. Some
partitions can get processed multiple times.
- More commonly, the result of the stage may be used in a later
calculation, and has to be recalculated. This happens if some of the
results
Daniel,
Thanks for the explanation.
On Wed, Jun 11, 2014 at 8:57 AM, Daniel Darabos
daniel.dara...@lynxanalytics.com wrote:
About more succeeded tasks than total tasks:
- This can happen if you have enabled speculative execution. Some
partitions can get processed multiple times.
- More
Using MEMORY_AND_DISK_SER to persist the input RDD[Rating] seems to work
right for me now. I'm testing on a larger dataset and will see how it goes.
On Wed, Jun 11, 2014 at 9:56 AM, Neville Li neville@gmail.com wrote:
Does cache eviction affect disk storage level too? I tried cranking up
The executors shown CANNOT FIND ADDRESS are not listed in the Executors Tab
on the top of the Spark UI.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Information-on-Spark-UI-tp7354p7355.html
Sent from the Apache Spark User List mailing list archive
PM, coderxiang shuoxiang...@gmail.com wrote:
The executors shown CANNOT FIND ADDRESS are not listed in the Executors
Tab
on the top of the Spark UI.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Information-on-Spark-UI-tp7354p7355.html
Sent from
201 - 208 of 208 matches
Mail list logo