Hi all, This error happens because we receive a "completed" event for a particular stage that we don't know about, i.e. a stage we haven't received a "submitted" event for. The root cause of this, as Baoxu explained, is usually because the event queue is full and the listener begins to drop events. In this case we are dropping the "submitted" event. This particular exception should be fixed in the latest master, as we now check for whether the key exists before indexing directly into it. Unfortunately, this is not in Spark 1.0.1, but will be fixed in Spark 1.1. There is currently no bullet-proof workaround for this issue, but you might try to reduce the number of concurrently running tasks (partitions) to avoid emitting too many events. The root cause of the listener queue taking too much time to process events is recorded in SPARK-2316, which we also intend to fix by Spark 1.1.
Andrew 2014-07-21 10:23 GMT-07:00 mrm <ma...@skimlinks.com>: > I have the same error! Did you manage to fix it? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/LiveListenerBus-throws-exception-and-weird-web-UI-bug-tp8330p10324.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
