Re: Slight API incompatibility caused by SPARK-4072

2015-07-15 Thread Patrick Wendell
One related note here is that we have a Java version of this that is an abstract class - in the doc it says that it exists more or less to allow for binary compatibility (it says it's for Java users, but really Scala could use this also):

Re: Slight API incompatibility caused by SPARK-4072

2015-07-15 Thread Patrick Wendell
Actually the java one is a concrete class. On Wed, Jul 15, 2015 at 12:14 PM, Patrick Wendell pwend...@gmail.com wrote: One related note here is that we have a Java version of this that is an abstract class - in the doc it says that it exists more or less to allow for binary compatibility (it

Re: Slight API incompatibility caused by SPARK-4072

2015-07-15 Thread Marcelo Vanzin
Or, alternatively, the bus could catch that error and ignore / log it, instead of stopping the context... On Wed, Jul 15, 2015 at 12:20 PM, Marcelo Vanzin van...@cloudera.com wrote: Hmm, the Java listener was added in 1.3, so I think it will work for my needs. Might be worth it to make it

Re: Slight API incompatibility caused by SPARK-4072

2015-07-15 Thread Reynold Xin
It's bad that expose a trait - even though we want to mixin stuff. We should really audit all of these and expose only abstract classes for anything beyond an extremely simple interface. That itself however would break binary compatibility. On Wed, Jul 15, 2015 at 12:15 PM, Patrick Wendell

Re: Slight API incompatibility caused by SPARK-4072

2015-07-15 Thread Marcelo Vanzin
Hmm, the Java listener was added in 1.3, so I think it will work for my needs. Might be worth it to make it clear in the SparkListener documentation that people should avoid using it directly. Or follow Reynold's suggestion. On Wed, Jul 15, 2015 at 12:14 PM, Patrick Wendell pwend...@gmail.com