jiaan.geng created SPARK-26641:
----------------------------------
Summary: Seperate capacity Configurations of different event queue.
Key: SPARK-26641
URL: https://issues.apache.org/jira/browse/SPARK-26641
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.4.0, 2.3.0, 3.0.0
Reporter: jiaan.geng
I always maintance a spark on yarn cluster on line.I always found the error:
`Dropping event from queue eventLog. This likely means one of the listeners is
too slow and cannot keep up with the rate at which tasks are being started by
the scheduler.`
Spark event log write to a hdfs cluster on line, and this hdfs cluster support
write a few TB.The performance issuse of write frequently, lead to the
`EventLoggingListener` exists the bottleneck.
But other event queue appear the problem rarely,so I think seperate the
configurations of different event queue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
