Yes, the driver has to be able to accept incoming connections. All the executors connect back to the driver sending heartbeats, map status, metrics. It is critical and I don't know of a way around it. You could look into using something like the https://github.com/spark-jobserver/spark-jobserver that could run outside the firewall. Then from inside the firewall you can make REST calls to the server.
On Thu, Feb 5, 2015 at 5:03 PM, Kane Kim <kane.ist...@gmail.com> wrote: > I submit spark job from machine behind firewall, I can't open any incoming > connections to that box, does driver absolutely need to accept incoming > connections? Is there any workaround for that case? > > Thanks. >
