Thanks for your input, Koert and DB. Rebuilding with 9.x didn't seem to work. For now we've downgraded dropwizard to 0.6.2 which uses a compatible version of jetty. Not optimal, but it works for now.
On Tue, Jul 8, 2014 at 7:04 PM, DB Tsai <dbt...@dbtsai.com> wrote: > We're doing similar thing to lunch spark job in tomcat, and I opened a > JIRA for this. There are couple technical discussions there. > > https://issues.apache.org/jira/browse/SPARK-2100 > > In this end, we realized that spark uses jetty not only for Spark > WebUI, but also for distributing the jars and tasks, so it really hard > to remove the web dependency in Spark. > > In the end, we lunch our spark job in yarn-cluster mode, and in the > runtime, the only dependency in our web application is spark-yarn > which doesn't contain any spark web stuff. > > PS, upgrading the spark jetty 8.x to 9.x in spark may not be > straightforward by just changing the version in spark build script. > Jetty 9.x required Java 7 since the servlet api (servlet 3.1) requires > Java 7. > > Sincerely, > > DB Tsai > ------------------------------------------------------- > My Blog: https://www.dbtsai.com > LinkedIn: https://www.linkedin.com/in/dbtsai > > > On Tue, Jul 8, 2014 at 8:43 AM, Koert Kuipers <ko...@tresata.com> wrote: >> do you control your cluster and spark deployment? if so, you can try to >> rebuild with jetty 9.x >> >> >> On Tue, Jul 8, 2014 at 9:39 AM, Martin Gammelsæter >> <martingammelsae...@gmail.com> wrote: >>> >>> Digging a bit more I see that there is yet another jetty instance that >>> is causing the problem, namely the BroadcastManager has one. I guess >>> this one isn't very wise to disable... It might very well be that the >>> WebUI is a problem as well, but I guess the code doesn't get far >>> enough. Any ideas on how to solve this? Spark seems to use jetty >>> 8.1.14, while dropwizard uses jetty 9.0.7, so that might be the source >>> of the problem. Any ideas? >>> >>> On Tue, Jul 8, 2014 at 2:58 PM, Martin Gammelsæter >>> <martingammelsae...@gmail.com> wrote: >>> > Hi! >>> > >>> > I am building a web frontend for a Spark app, allowing users to input >>> > sql/hql and get results back. When starting a SparkContext from within >>> > my server code (using jetty/dropwizard) I get the error >>> > >>> > java.lang.NoSuchMethodError: >>> > org.eclipse.jetty.server.AbstractConnector: method <init>()V not found >>> > >>> > when Spark tries to fire up its own jetty server. This does not happen >>> > when running the same code without my web server. This is probably >>> > fixable somehow(?) but I'd like to disable the webUI as I don't need >>> > it, and ideally I would like to access that information >>> > programatically instead, allowing me to embed it in my own web >>> > application. >>> > >>> > Is this possible? >>> > >>> > -- >>> > Best regards, >>> > Martin Gammelsæter >>> >>> >>> >>> -- >>> Mvh. >>> Martin Gammelsæter >>> 92209139 >> >> -- Mvh. Martin Gammelsæter 92209139
