So weird, as I said - I created a new empty project where Spark core was the only dependency...
[image: datastax_logo.png] <http://www.datastax.com/> JACEK LEWANDOWSKI DSE Software Engineer | +48.609.810.774 | jacek.lewandow...@datastax.com [image: linkedin.png] <https://www.linkedin.com/company/datastax> [image: facebook.png] <https://www.facebook.com/datastax> [image: twitter.png] <https://twitter.com/datastax> [image: g+.png] <https://plus.google.com/+Datastax/about> <http://feeds.feedburner.com/datastax> <https://github.com/datastax/> DataStax is the fastest, most scalable distributed database technology, delivering Apache Cassandra to the world’s most innovative enterprises. Datastax is built to be agile, always-on, and predictably scalable to any size. With more than 500 customers in 45 countries, DataStax is the database technology and transactional backbone of choice for the worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay. On Tue, Apr 7, 2015 at 7:08 PM, Marcelo Vanzin <van...@cloudera.com> wrote: > BTW, just out of curiosity, I checked both the 1.3.0 release assembly > and the spark-core_2.10 artifact downloaded from > http://mvnrepository.com/, and neither contain any references to > anything under "org.eclipse" (all referenced jetty classes are the > shaded ones under org.spark-project.jetty). > > On Mon, Apr 6, 2015 at 10:30 PM, Josh Rosen <rosenvi...@gmail.com> wrote: > > My hunch is that this behavior was introduced by a patch to start shading > > Jetty in Spark 1.3: https://issues.apache.org/jira/browse/SPARK-3996. > > > > Note that Spark's MetricsSystem class is marked as private[spark] and > thus > > isn't intended to be interacted with directly by users. It's not super > > likely that this API would break, but it's excluded from our MiMa checks > and > > thus is liable to change in incompatible ways across releases. > > > > If you add these Jetty classes as a compile-only dependency but don't add > > them to the runtime classpath, do you get runtime errors? If the metrics > > system is usable at runtime and we only have errors when attempting to > > compile user code against non-public APIs, then I'm not sure that this > is a > > high-priority issue to fix since. If the metrics system doesn't work at > > runtime, on the other hand, then that's definitely a bug that should be > > fixed. > > > > If you'd like to continue debugging this issue, I think we should move > this > > discussion over to JIRA so it's easier to track and reference. > > > > Hope this helps, > > Josh > > > > On Thu, Apr 2, 2015 at 7:34 AM, Jacek Lewandowski > > <jacek.lewandow...@datastax.com> wrote: > >> > >> A very simple example which works well with Spark 1.2, and fail to > compile > >> with Spark 1.3: > >> > >> build.sbt: > >> > >> name := "untitled" > >> version := "1.0" > >> scalaVersion := "2.10.4" > >> libraryDependencies += "org.apache.spark" %% "spark-core" % "1.3.0" > >> > >> Test.scala: > >> > >> package org.apache.spark.metrics > >> import org.apache.spark.SparkEnv > >> class Test { > >> SparkEnv.get.metricsSystem.report() > >> } > >> > >> Produces: > >> > >> Error:scalac: bad symbolic reference. A signature in MetricsSystem.class > >> refers to term eclipse > >> in package org which is not available. > >> It may be completely missing from the current classpath, or the version > on > >> the classpath might be incompatible with the version used when compiling > >> MetricsSystem.class. > >> > >> Error:scalac: bad symbolic reference. A signature in MetricsSystem.class > >> refers to term jetty > >> in value org.eclipse which is not available. > >> It may be completely missing from the current classpath, or the version > on > >> the classpath might be incompatible with the version used when compiling > >> MetricsSystem.class. > >> > >> This looks like something wrong with shading jetty. > >> MetricsSystem references MetricsServlet which references some classes > from > >> Jetty, in the original package instead of shaded one. I'm not sure, but > >> adding the following dependencies solves the problem: > >> > >> libraryDependencies += "org.eclipse.jetty" % "jetty-server" % > >> "8.1.14.v20131031" > >> libraryDependencies += "org.eclipse.jetty" % "jetty-servlet" % > >> "8.1.14.v20131031" > >> > >> Is it intended or is it a bug? > >> > >> > >> Thanks ! > >> > >> > >> Jacek > >> > >> > > > > > > -- > Marcelo >
