Yea we are going to tighten a lot of class' visibility. A lot of APIs were made experimental, developer, or public for no good reason in the past. Many of them (not Logging in this case) are tied to the internal implementation of Spark at a specific time, and no longer make sense given the project's evolution.
On Tue, Mar 15, 2016 at 12:32 PM, Koert Kuipers <ko...@tresata.com> wrote: > oh i just noticed the big warning in spark 1.x Logging > > > * NOTE: DO NOT USE this class outside of Spark. It is intended as an > internal utility. > * This will likely be changed or removed in future releases. > > On Tue, Mar 15, 2016 at 3:29 PM, Koert Kuipers <ko...@tresata.com> wrote: > >> makes sense >> >> note that Logging was not private[spark] in 1.x, which is why i used it. >> >> On Tue, Mar 15, 2016 at 12:55 PM, Marcelo Vanzin <van...@cloudera.com> >> wrote: >> >>> Logging is a "private[spark]" class so binary compatibility is not >>> important at all, because code outside of Spark isn't supposed to use >>> it. Mixing Spark library versions is also not recommended, not just >>> because of this reason. >>> >>> There have been other binary changes in the Logging class in the past >>> too. >>> >>> On Tue, Mar 15, 2016 at 7:49 AM, Koert Kuipers <ko...@tresata.com> >>> wrote: >>> > i have been using spark 2.0 snapshots with some libraries build for >>> spark >>> > 1.0 so far (simply because it worked). in last few days i noticed this >>> new >>> > error: >>> > >>> > [error] Uncaught exception when running >>> > com.tresata.spark.sql.fieldsapi.FieldsApiSpec: >>> java.lang.AbstractMethodError >>> > sbt.ForkMain$ForkError: java.lang.AbstractMethodError: null >>> > at org.apache.spark.Logging$class.log(Logging.scala:46) >>> > at >>> > >>> com.tresata.spark.sorted.PairRDDFunctions.log(PairRDDFunctions.scala:13) >>> > >>> > so it seems spark made binary incompatible changes in logging. >>> > i do not think spark 2.0 is trying to have binary compatibility with >>> 1.0 so >>> > i assume this is a non-issue, but just in case the assumptions are >>> different >>> > (or incompatibilities are actively minimized) i wanted to point it out. >>> > >>> >>> >>> >>> -- >>> Marcelo >>> >> >> >
