Yes, I used both. The discussion on this seems to be at github now: https://github.com/apache/spark/pull/4780
I am using more classes from a package from which spark uses HyperLogLog as well. So we are both including the jar file but Spark is excluding the dependent package that is required. On Thu, Feb 26, 2015 at 9:54 AM, Marcelo Vanzin <van...@cloudera.com> wrote: > On Wed, Feb 25, 2015 at 8:42 PM, Jim Kleckner <j...@cloudphysics.com> > wrote: > > So, should the userClassPathFirst flag work and there is a bug? > > Sorry for jumping in the middle of conversation (and probably missing > some of it), but note that this option applies only to executors. If > you're trying to use the class in your driver, there's a separate > option for that. > > Also to note is that if you're adding a class that doesn't exist > inside the Spark jars, which seems to be the case, this option should > be irrelevant, since the class loaders should all end up finding the > one copy of the class that you're adding with your app. > > -- > Marcelo > -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Re-Spark-excludes-fastutil-dependencies-we-need-tp21849.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
