Hey all,
Just noticed this when some of our tests started to fail. SPARK-4072 added
a new method to the "SparkListener" trait, and even though it has a default
implementation, it doesn't seem like that applies retroactively.
Namely, if you have an existing, compiled app that has an implementation
One related note here is that we have a Java version of this that is
an abstract class - in the doc it says that it exists more or less to
allow for binary compatibility (it says it's for Java users, but
really Scala could use this also):
https://github.com/apache/spark/blob/master/core/src/main/j
Actually the java one is a concrete class.
On Wed, Jul 15, 2015 at 12:14 PM, Patrick Wendell wrote:
> One related note here is that we have a Java version of this that is
> an abstract class - in the doc it says that it exists more or less to
> allow for binary compatibility (it says it's for Jav
It's bad that expose a trait - even though we want to mixin stuff. We
should really audit all of these and expose only abstract classes for
anything beyond an extremely simple interface. That itself however would
break binary compatibility.
On Wed, Jul 15, 2015 at 12:15 PM, Patrick Wendell
wrote
Hmm, the Java listener was added in 1.3, so I think it will work for my
needs.
Might be worth it to make it clear in the SparkListener documentation that
people should avoid using it directly. Or follow Reynold's suggestion.
On Wed, Jul 15, 2015 at 12:14 PM, Patrick Wendell
wrote:
> One related
Or, alternatively, the bus could catch that error and ignore / log it,
instead of stopping the context...
On Wed, Jul 15, 2015 at 12:20 PM, Marcelo Vanzin
wrote:
> Hmm, the Java listener was added in 1.3, so I think it will work for my
> needs.
>
> Might be worth it to make it clear in the Spark
