To add to Sean and Reynold's point:
Please correct me if I'm wrong, but Spark depends on hadoop-common which
also uses jetty in the HttpServer2 code. So even if you remove jetty
from Spark by making it an optional dependency, it will be pulled in by
Hadoop.
So you'll still see that your prog
Sure, but you are not using Netty at all. It's invisible to you. It's
not as if you have to set up and maintain a Jetty container. I don't
think your single platform for your apps is relevant.
You can turn off the UI, but as Reynold said, the HTTP servers are
also part of the core data transport f
Hi Sean,
The issue we have here is that all our products are based on a single
platform and we try to make all our products coherent with our platform as
much as possible. so, having two web services in one instance would not be
a very elegant solution. That is why we were seeking a way to switch i
I do not think it makes sense to make the web server configurable.
Mostly because there's no real problem in running an HTTP service
internally based on Netty while you run your own HTTP service based on
something else like Tomcat. What's the problem?
On Wed, Feb 18, 2015 at 3:14 AM, Niranda Perer
Niranda,
I'm not sure if I'd say Spark's use of Jetty to expose its UI monitoring
layer constitutes a use of "two web servers in a single product". Hadoop
uses Jetty as well as do many other applications today that need embedded
http layers for serving up their monitoring UI to users. This is comp
Hey Niranda,
It seems to me a lot of effort to support multiple libraries inside of
Spark like this, so I'm not sure that's a great solution.
If you are building an application that embeds Spark, is it not
possible for you to continue to use Jetty for Spark's internal servers
and use tomcat for y
Hi Sean,
The main issue we have is, running two web servers in a single product. we
think it would not be an elegant solution.
Could you please point me to the main areas where jetty server is tightly
coupled or extension points where I could plug tomcat instead of jetty?
If successful I could con
There's no particular reason you have to remove the embedded Jetty
server, right? it doesn't prevent you from using it inside another app
that happens to run in Tomcat. You won't be able to switch it out
without rewriting a fair bit of code, no, but you don't need to.
On Mon, Feb 16, 2015 at 5:08
Mostly UI.
However, we are also using Jetty as a file server I believe (for
distributing files from the driver to workers).
On Sun, Feb 15, 2015 at 9:24 PM, Niranda Perera
wrote:
> Hi Reynold,
>
> Thank you for the response. Could you please clarify the need of Jetty
> server inside Spark? Is
Hi Reynold,
Thank you for the response. Could you please clarify the need of Jetty
server inside Spark? Is it used for Spark core functionality or is it there
for Spark jobs UI purposes?
cheers
On Mon, Feb 16, 2015 at 10:47 AM, Reynold Xin wrote:
> Most likely no. We are using the embedded mod
Most likely no. We are using the embedded mode of Jetty, rather than using
servlets.
Even if it is possible, you probably wouldn't want to embed Spark in your
application server ...
On Sun, Feb 15, 2015 at 9:08 PM, Niranda Perera
wrote:
> Hi,
>
> We are thinking of integrating Spark server ins
11 matches
Mail list logo
