One related note here is that we have a Java version of this that is an abstract class - in the doc it says that it exists more or less to allow for binary compatibility (it says it's for Java users, but really Scala could use this also):
https://github.com/apache/spark/blob/master/core/src/main/java/org/apache/spark/JavaSparkListener.java#L23 I think it might be reasonable that the Scala trait provides only source compatibitly and the Java class provides binary compatibility. - Patrick On Wed, Jul 15, 2015 at 11:47 AM, Marcelo Vanzin <van...@cloudera.com> wrote: > Hey all, > > Just noticed this when some of our tests started to fail. SPARK-4072 added a > new method to the "SparkListener" trait, and even though it has a default > implementation, it doesn't seem like that applies retroactively. > > Namely, if you have an existing, compiled app that has an implementation of > SparkListener, that app won't work on 1.5 without a recompile. You'll get > something like this: > > java.lang.AbstractMethodError > at > org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:62) > at > org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31) > at > org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31) > at > org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:56) > at > org.apache.spark.util.AsynchronousListenerBus.postToAll(AsynchronousListenerBus.scala:37) > at > org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(AsynchronousListenerBus.scala:79) > at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1235) > at > org.apache.spark.util.AsynchronousListenerBus$$anon$1.run(AsynchronousListenerBus.scala:63) > > > Now I know that "SparkListener" is marked as @DeveloperApi, but is this > something we should care about? Seems like adding methods to traits is just > as backwards-incompatible as adding new methods to Java interfaces. > > > -- > Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
