Hi, AFAIK, spark driver and executor share its environments including SPARK_HOME. I don't have tested them, but it won't work well because there are different scheme of each OS.
Hope this help, JL On Thu, Jun 9, 2016 at 6:15 AM, Pedro Reta <[email protected]> wrote: > Hi Moon, > > Thanks for the quick and kind response. > I've just tried the following: > > SPARK_HOME/bin/spark-shell.cmd --master *spark://<master>:7077* > > This works properly, the connection is done. Also, the remote spark > localhost:8081 page shows the remote connection under 'Running > Applications'. > After this, I changed my interpreter master configuration as this > master spark://<master>:7077 > Restarted the interpreter and tried to run a paragraph, it fails. This is > the stack trace in the zeppelin command line: > > java.io.IOException: Failed to connect to <master>:7077 at > org.apache.spark.network.client.TransportClientFactory.createClient(T > ransportClientFactory.java:216) at > org.apache.spark.network.client.TransportClientFactory.createClient(T > ransportClientFactory.java:167) at > org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala :200) > at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:187) at > org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:183) at > java.util.concurrent.FutureTask.run(FutureTask.java:266) at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. > java:1142) at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor > .java:617) at java.lang.Thread.run(Thread.java:745) Caused by: > java.nio.channels.UnresolvedAddressException at > sun.nio.ch.Net.checkAddress(Net.java:101) at > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622) at > io.netty.channel.socket.nio.NioSocketChannel.doConnect(NioSocketChann > el.java:209) at > io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(Abs > tractNioChannel.java:207) at > io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultCh > annelPipeline.java:1097) at > io.netty.channel.AbstractChannelHandlerContext.invokeConnect(Abstract > ChannelHandlerContext.java:471) at > io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChanne > lHandlerContext.java:456) at > io.netty.channel.ChannelOutboundHandlerAdapter.connect(ChannelOutboun > dHandlerAdapter.java:47) at > io.netty.channel.AbstractChannelHandlerContext.invokeConnect(Abstract > ChannelHandlerContext.java:471) at > io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChanne > lHandlerContext.java:456) at > io.netty.channel.ChannelDuplexHandler.connect(ChannelDuplexHandler.ja > va:50) at > io.netty.channel.AbstractChannelHandlerContext.invokeConnect(Abstract > ChannelHandlerContext.java:471) at > io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChanne > lHandlerContext.java:456) at > io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChanne > lHandlerContext.java:438) at > io.netty.channel.DefaultChannelPipeline.connect(DefaultChannelPipelin > e.java:908) at > io.netty.channel.AbstractChannel.connect(AbstractChannel.java:203) at > io.netty.bootstrap.Bootstrap$2.run(Bootstrap.java:166) at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(Single > ThreadEventExecutor.java:357) at > io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThread > EventExecutor.java:111) ... 1 more > > My *zeppelin-env.cmd* contains: > > *set SPARK_HOME=C:\SPARK\spark-1.6.1* > *set HADOOP_HOME=C:\hadoop* > *set PYTHONPATH=%SPARK_HOME%\python* > > Also tried adding > > *set MASTER=spark://<master>:7077* > > Still no luck. > Do you have any suggestions? I would appreciate them a lot :) > > Thanks in advance. > > > On Wed, Jun 8, 2016 at 3:32 PM, moon soo Lee <[email protected]> wrote: > >> Hi, >> >> Could you verify if SPARK_HOME/bin/spark-shell.cmd from your windows >> machine to connect to spark cluster running on a linux machine works? >> >> Then Zeppelin supposed to work with 'export SPARK_HOME' configuration in >> your conf/zeppelin-env.sh >> >> Thanks, >> moon >> >> >> On Wed, Jun 8, 2016 at 12:48 PM Pedro Reta <[email protected]> >> wrote: >> >>> Hi, >>> >>> I was wondering if you could help me with some insight about a problem I >>> am having. >>> It might sound silly to you but here it goes: >>> >>> I built a Zeppelin on Windows, and also a Spark standalone server in the >>> same machine. >>> Now, I am able to connect to my Windows Spark server using Zeppelin. >>> I have another Spark server running on a Linux machine, and I was trying >>> to connect to it from my Windows Zeppelin but I've had no luck. I'm kinda >>> lost of how to set the env variables defined in zeppelin-env.cmd. >>> So the real question is... Is it possible to do such connection? If so, >>> do you think you could give me some hints about how to do it? >>> >>> Thanks in advance >>> >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
