[ https://issues.apache.org/jira/browse/SPARK-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Dai updated SPARK-4078: ----------------------------- Assignee: Jason Dai > New FsPermission instance w/o FsPermission.createImmutable in eventlog > ---------------------------------------------------------------------- > > Key: SPARK-4078 > URL: https://issues.apache.org/jira/browse/SPARK-4078 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.1.0 > Reporter: Jie Huang > Assignee: Jason Dai > > By default, Spark builds its package against Hadoop 1.0.4 version. In that > version, it has some FsPermission bug (see HADOOP-7629 by Todd Lipcon). This > bug got fixed since 1.1 version. By using that FsPermission.createImmutable() > API, end-user may see some RPC exception like below (if turn on eventlog over > HDFS). > {quote} > Exception in thread "main" java.io.IOException: Call to sr484/10.1.2.84:54310 > failed on local exception: java.io.EOFException > at org.apache.hadoop.ipc.Client.wrapException(Client.java:1150) > at org.apache.hadoop.ipc.Client.call(Client.java:1118) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229) > at $Proxy6.setPermission(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62) > at $Proxy6.setPermission(Unknown Source) > at org.apache.hadoop.hdfs.DFSClient.setPermission(DFSClient.java:1285) > at > org.apache.hadoop.hdfs.DistributedFileSystem.setPermission(DistributedFileSystem.java:572) > at org.apache.spark.util.FileLogger.createLogDir(FileLogger.scala:138) > at org.apache.spark.util.FileLogger.start(FileLogger.scala:115) > at > org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:74) > at org.apache.spark.SparkContext.<init>(SparkContext.scala:324) > {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org