Dependency hell... My fav problem :).
I had run into a similar issue with hbase and jetty. I cant remember thw
exact fix, but is are excerpts from my dependencies that may be relevant:
val hadoop2Common = "org.apache.hadoop" % "hadoop-common" % hadoop2Version
excludeAll(
ExclusionRule(organization = "javax.servlet"),
ExclusionRule(organization = "javax.servlet.jsp"),
ExclusionRule(organization = "org.mortbay.jetty")
)
val hadoop2MapRedClient = "org.apache.hadoop" %
"hadoop-mapreduce-client-core" % hadoop2Version
val hbase = "org.apache.hbase" % "hbase" % hbaseVersion excludeAll(
ExclusionRule(organization = "org.apache.maven.wagon"),
ExclusionRule(organization = "org.jboss.netty"),
ExclusionRule(organization = "org.mortbay.jetty"),
ExclusionRule(organization = "org.jruby") // Don't need
HBASE's jruby. It pulls in whole lot of other dependencies like joda-time.
)
val sparkCore = "org.apache.spark" %% "spark-core" % sparkVersion
val sparkStreaming = "org.apache.spark" %% "spark-streaming" %
sparkVersion
val sparkSQL = "org.apache.spark" %% "spark-sql" % sparkVersion
val sparkHive = "org.apache.spark" %% "spark-hive" % sparkVersion
val sparkRepl = "org.apache.spark" %% "spark-repl" % sparkVersion
val sparkAll = Seq (
sparkCore excludeAll(
ExclusionRule(organization = "org.apache.hadoop")), // We assume
hadoop 2 and hence omit hadoop 1 dependencies
sparkSQL,
sparkStreaming,
hadoop2MapRedClient,
hadoop2Common,
"org.mortbay.jetty" % "servlet-api" % "3.0.20100224"
)
On Sep 11, 2014 8:05 PM, <sp...@orbit-x.de> wrote:
> Hi guys,
>
> any luck with this issue, anyone?
>
> I aswell tried all the possible exclusion combos to a no avail.
>
> thanks for your ideas
> reinis
>
> -----Original-Nachricht-----
> > Von: "Stephen Boesch" <java...@gmail.com>
> > An: user <user@spark.apache.org>
> > Datum: 28-06-2014 15:12
> > Betreff: Re: HBase 0.96+ with Spark 1.0+
> >
> > Hi Siyuan,
> Thanks for the input. We are preferring to use the SparkBuild.scala
> instead of maven. I did not see any protobuf.version related settings in
> that file. But - as noted by Sean Owen - in any case the issue we are
> facing presently is about the duplicate incompatible javax.servlet entries
> - apparently from the org.mortbay artifacts.
>
>
> >
> > 2014-06-28 6:01 GMT-07:00 Siyuan he <hsy...@gmail.com>:
> > Hi Stephen,
> >
> I am using spark1.0+ HBase0.96.2. This is what I did:
> 1) rebuild spark using: mvn -Dhadoop.version=2.3.0
> -Dprotobuf.version=2.5.0 -DskipTests clean package
> 2) In spark-env.sh, set SPARK_CLASSPATH =
> /path-to/hbase-protocol-0.96.2-hadoop2.jar
>
> >
> Hopefully it can help.
> Siyuan
>
>
> >
> > On Sat, Jun 28, 2014 at 8:52 AM, Stephen Boesch <java...@gmail.com>
> wrote:
> >
> >
> Thanks Sean. I had actually already added exclusion rule for
> org.mortbay.jetty - and that had not resolved it.
> >
> Just in case I used your precise formulation:
>
> >
> val excludeMortbayJetty = ExclusionRule(organization = "org.mortbay.jetty")
> ..
>
> ,("org.apache.spark" % "spark-core_2.10" % sparkVersion
> withSources()).excludeAll(excludeMortbayJetty)
> ,("org.apache.spark" % "spark-sql_2.10" % sparkVersion
> withSources()).excludeAll(excludeMortbayJetty)
>
> >
> However the same error still recurs:
>
> >
> 14/06/28 05:48:35 INFO HttpServer: Starting HTTP Server
> [error] (run-main-0) java.lang.SecurityException: class
> "javax.servlet.FilterRegistration"'s signer information does not match
> signer information of other classes in the same package
> java.lang.SecurityException: class "javax.servlet.FilterRegistration"'s
> signer information does not match signer information of other classes in
> the same package
>
>
>
> >
>
> >
>
> >
> > 2014-06-28 4:22 GMT-07:00 Sean Owen <so...@cloudera.com>:
>
> > This sounds like an instance of roughly the same item as in
> > https://issues.apache.org/jira/browse/SPARK-1949 Have a look at
> > adding that exclude to see if it works.
> >
>
> > On Fri, Jun 27, 2014 at 10:21 PM, Stephen Boesch <java...@gmail.com>
> wrote:
> > > The present trunk is built and tested against HBase 0.94.
> > >
> > >
> > > I have tried various combinations of versions of HBase 0.96+ and Spark
> 1.0+
> > > and all end up with
> > >
> > > 14/06/27 20:11:15 INFO HttpServer: Starting HTTP Server
> > > [error] (run-main-0) java.lang.SecurityException: class
> > > "javax.servlet.FilterRegistration"'s signer information does not match
> > > signer information of other classes in the same package
> > > java.lang.SecurityException: class "javax.servlet.FilterRegistration"'s
> > > signer information does not match signer information of other classes
> in the
> > > same package
> > > at java.lang.ClassLoader.checkCerts(ClassLoader.java:952)
> > >
> > >
> > > I have tried a number of different ways to exclude javax.servlet
> related
> > > jars. But none have avoided this error.
> > >
> > > Anyone have a (small-ish) build.sbt that works with later versions of
> HBase?
> > >
> > >
> >
>
>
> >
>
>
> >
>
> >
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>
