Zhan specifying port fixed the port issue.
Is it possible to specify the log directory while starting the spark
thriftserver?
Still getting this error even through the folder exists and everyone has
permission to use that directory.
drwxr-xr-x 2 root root 4096 Mar 24 19:04 spark-events
Exception in thread "main" java.lang.IllegalArgumentException: Log
directory /tmp/spark-events does not exist.
at
org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:99)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:399)
at
org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:49)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:58)
at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166)
at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
On Mon, Mar 23, 2015 at 6:51 PM, Zhan Zhang <zzh...@hortonworks.com> wrote:
> Probably the port is already used by others, e.g., hive. You can change
> the port similar to below
>
> ./sbin/start-thriftserver.sh --master yarn --executor-memory 512m
> --hiveconf hive.server2.thrift.port=10001
>
>
> Thanks.
>
> Zhan Zhang
>
> On Mar 23, 2015, at 12:01 PM, Neil Dev <neilk...@gmail.com> wrote:
>
> Hi,
>
> I am having issue starting spark-thriftserver. I'm running spark 1.3.with
> Hadoop 2.4.0. I would like to be able to change its port too so, I can hive
> hive-thriftserver as well as spark-thriftserver running at the same time.
>
> Starting sparkthrift server:-
> sudo ./start-thriftserver.sh --master spark://ip-172-31-10-124:7077
> --executor-memory 2G
>
> Error:-
> I created the folder manually but still getting the following error----
> Exception in thread "main" java.lang.IllegalArgumentException: Log
> directory /tmp/spark-events does not exist.
>
>
> I am getting the following error
> 15/03/23 15:07:02 ERROR thrift.ThriftCLIService: Error:
> org.apache.thrift.transport.TTransportException: Could not create
> ServerSocket on address0.0.0.0/0.0.0.0:10000.
> at
> org.apache.thrift.transport.TServerSocket.<init>(TServerSocket.java:93)
> at
> org.apache.thrift.transport.TServerSocket.<init>(TServerSocket.java:79)
> at
>
> org.apache.hive.service.auth.HiveAuthFactory.getServerSocket(HiveAuthFactory.java:236)
> at
>
> org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:69)
> at java.lang.Thread.run(Thread.java:745)
>
> Thanks
> Neil
>
>
>
