Hi,All
We run a spark cluster with three workers.
created a spark streaming application,
then run the spark project using below command:
shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo
we looked at the webui of workers, jobs failed without any error or info,
but FileNotFoundException occurred in workers' log file as below:
Is this an existent issue of spark?
-----------------------------in workers'
logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out-----
-----------------------
14/05/05 02:39:39 WARN AbstractHttpConnection:
/logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout
java.io.FileNotFoundException:
/test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or
directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.s
cala:52)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.s
cala:52)
at
org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java
:1040)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:
976)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135
)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:1
16)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpCo
nnection.java:483)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpC
onnection.java:920)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplet
e(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java
:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.
java:628)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.j
ava:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:
608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:5
43)
at java.lang.Thread.run(Thread.java:722)
14/05/05 02:39:41 WARN AbstractHttpConnection:
/logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr
java.io.FileNotFoundException:
/test/spark-0.9.1/work/app-20140505053550-0000/9/stderr (No such file or
directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.s
cala:52)
at
org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.s
cala:52)
at
org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java
:1040)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:
976)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135
)
at
org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:1
16)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpCo
nnection.java:483)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpC
onnection.java:920)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplet
e(AbstractHttpConnection.java:982)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java
:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.
java:628)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.j
ava:52)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:
608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:5
43)
at java.lang.Thread.run(Thread.java:722)
Thanks,
Francis.Hu
