[ https://issues.apache.org/jira/browse/SPARK-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ding Yuan updated SPARK-4863: ----------------------------- Description: Following up with the discussion in https://issues.apache.org/jira/browse/SPARK-1148, I am creating a new JIRA to report the suspicious exception handlers detected by our tool aspirator on spark-1.1.1. {noformat} ========================================== WARNING: TODO; in handler. Line: 129, File: "org/apache/thrift/transport/TNonblockingServerSocket.java" 122: public void registerSelector(Selector selector) { 123: try { 124: // Register the server socket channel, indicating an interest in 125: // accepting new connections 126: serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT); 127: } catch (ClosedChannelException e) { 128: // this shouldn't happen, ideally... 129: // TODO: decide what to do with this. 130: } 131: } ========================================== ========================================== WARNING: TODO; in handler. Line: 1583, File: "org/apache/spark/SparkContext.scala" 1578: val scheduler = try { 1579: val clazz = Class.forName("org.apache.spark.scheduler.cluster.YarnClusterScheduler") 1580: val cons = clazz.getConstructor(classOf[SparkContext]) 1581: cons.newInstance(sc).asInstanceOf[TaskSchedulerImpl] 1582: } catch { 1583: // TODO: Enumerate the exact reasons why it can fail 1584: // But irrespective of it, it means we cannot proceed ! 1585: case e: Exception => { 1586: throw new SparkException("YARN mode not available ?", e) 1587: } ========================================== ========================================== WARNING 1: empty handler for exception: java.lang.Exception THERE IS NO LOG MESSAGE!!! Line: 75, File: "org/apache/spark/repl/ExecutorClassLoader.scala" try { val pathInDirectory = name.replace('.', '/') + ".class" val inputStream = { if (fileSystem != null) { fileSystem.open(new Path(directory, pathInDirectory)) } else { if (SparkEnv.get.securityManager.isAuthenticationEnabled()) { val uri = new URI(classUri + "/" + urlEncode(pathInDirectory)) val newuri = Utils.constructURIForAuthentication(uri, SparkEnv.get.securityManager) newuri.toURL().openStream() } else { new URL(classUri + "/" + urlEncode(pathInDirectory)).openStream() } } } val bytes = readAndTransformClass(name, inputStream) inputStream.close() Some(defineClass(name, bytes, 0, bytes.length)) } catch { case e: Exception => None } ========================================== ========================================== WARNING 1: empty handler for exception: java.io.IOException THERE IS NO LOG MESSAGE!!! Line: 275, File: "org/apache/spark/util/Utils.scala" try { dir = new File(root, "spark-" + UUID.randomUUID.toString) if (dir.exists() || !dir.mkdirs()) { dir = null } } catch { case e: IOException => ; } ========================================== ========================================== WARNING 1: empty handler for exception: java.lang.InterruptedException THERE IS NO LOG MESSAGE!!! Line: 172, File: "parquet/org/apache/thrift/server/TNonblockingServer.java" protected void joinSelector() { // wait until the selector thread exits try { selectThread_.join(); } catch (InterruptedException e) { // for now, just silently ignore. technically this means we'll have less of // a graceful shutdown as a result. } } ========================================== ========================================== WARNING 2: empty handler for exception: java.net.SocketException There are log messages.. Line: 111, File: "parquet/org/apache/thrift/transport/TNonblockingSocket.java" public void setTimeout(int timeout) { try { socketChannel_.socket().setSoTimeout(timeout); } catch (SocketException sx) { LOGGER.warn("Could not set socket timeout.", sx); } } ========================================== ========================================== WARNING 3: empty handler for exception: java.net.SocketException There are log messages.. Line: 103, File: "parquet/org/apache/thrift/transport/TServerSocket.java" public void listen() throws TTransportException { // Make sure not to block on accept if (serverSocket_ != null) { try { serverSocket_.setSoTimeout(0); } catch (SocketException sx) { LOGGER.error("Could not set socket timeout.", sx); } } } ========================================== ========================================== WARNING 4: empty handler for exception: java.net.SocketException There are log messages.. Line: 70, File: "parquet/org/apache/thrift/transport/TSocket.java" private void initSocket() { socket_ = new Socket(); try { socket_.setSoLinger(false, 0); socket_.setTcpNoDelay(true); socket_.setSoTimeout(timeout_); } catch (SocketException sx) { LOGGER.error("Could not configure socket.", sx); } } ========================================== ========================================== WARNING 5: empty handler for exception: java.net.SocketException There are log messages.. Line: 134, File: "parquet/org/apache/thrift/transport/TSocket.java" public void setTimeout(int timeout) { timeout_ = timeout; try { socket_.setSoTimeout(timeout); } catch (SocketException sx) { LOGGER.warn("Could not set socket timeout.", sx); } } ========================================== ========================================== WARNING 1: empty handler for exception: java.lang.Throwable There are log messages.. Line: 484, File: "tachyon/org/apache/thrift/server/AbstractNonblockingServer.java" try { processorFactory_.getProcessor(inTrans).process(inProt, outProt); responseReady(); return; } catch (TException te) { LOGGER.warn("Exception while invoking!", te); } catch (Throwable t) { LOGGER.error("Unexpected throwable while invoking!", t); } ========================================== {noformat} was: Following up with the discussion in https://issues.apache.org/jira/browse/SPARK-1148, I am creating a new JIRA to report the suspicious exception handlers detected by our tool aspirator on spark-1.1.1. ========================================== WARNING: TODO; in handler. Line: 129, File: "org/apache/thrift/transport/TNonblockingServerSocket.java" 122: public void registerSelector(Selector selector) { 123: try { 124: // Register the server socket channel, indicating an interest in 125: // accepting new connections 126: serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT); 127: } catch (ClosedChannelException e) { 128: // this shouldn't happen, ideally... 129: // TODO: decide what to do with this. 130: } 131: } ========================================== ========================================== WARNING: TODO; in handler. Line: 1583, File: "org/apache/spark/SparkContext.scala" 1578: val scheduler = try { 1579: val clazz = Class.forName("org.apache.spark.scheduler.cluster.YarnClusterScheduler") 1580: val cons = clazz.getConstructor(classOf[SparkContext]) 1581: cons.newInstance(sc).asInstanceOf[TaskSchedulerImpl] 1582: } catch { 1583: // TODO: Enumerate the exact reasons why it can fail 1584: // But irrespective of it, it means we cannot proceed ! 1585: case e: Exception => { 1586: throw new SparkException("YARN mode not available ?", e) 1587: } ========================================== ========================================== WARNING 1: empty handler for exception: java.lang.Exception THERE IS NO LOG MESSAGE!!! Line: 75, File: "org/apache/spark/repl/ExecutorClassLoader.scala" try { val pathInDirectory = name.replace('.', '/') + ".class" val inputStream = { if (fileSystem != null) { fileSystem.open(new Path(directory, pathInDirectory)) } else { if (SparkEnv.get.securityManager.isAuthenticationEnabled()) { val uri = new URI(classUri + "/" + urlEncode(pathInDirectory)) val newuri = Utils.constructURIForAuthentication(uri, SparkEnv.get.securityManager) newuri.toURL().openStream() } else { new URL(classUri + "/" + urlEncode(pathInDirectory)).openStream() } } } val bytes = readAndTransformClass(name, inputStream) inputStream.close() Some(defineClass(name, bytes, 0, bytes.length)) } catch { case e: Exception => None } ========================================== ========================================== WARNING 1: empty handler for exception: java.io.IOException THERE IS NO LOG MESSAGE!!! Line: 275, File: "org/apache/spark/util/Utils.scala" try { dir = new File(root, "spark-" + UUID.randomUUID.toString) if (dir.exists() || !dir.mkdirs()) { dir = null } } catch { case e: IOException => ; } ========================================== ========================================== WARNING 1: empty handler for exception: java.lang.InterruptedException THERE IS NO LOG MESSAGE!!! Line: 172, File: "parquet/org/apache/thrift/server/TNonblockingServer.java" protected void joinSelector() { // wait until the selector thread exits try { selectThread_.join(); } catch (InterruptedException e) { // for now, just silently ignore. technically this means we'll have less of // a graceful shutdown as a result. } } ========================================== ========================================== WARNING 2: empty handler for exception: java.net.SocketException There are log messages.. Line: 111, File: "parquet/org/apache/thrift/transport/TNonblockingSocket.java" public void setTimeout(int timeout) { try { socketChannel_.socket().setSoTimeout(timeout); } catch (SocketException sx) { LOGGER.warn("Could not set socket timeout.", sx); } } ========================================== ========================================== WARNING 3: empty handler for exception: java.net.SocketException There are log messages.. Line: 103, File: "parquet/org/apache/thrift/transport/TServerSocket.java" public void listen() throws TTransportException { // Make sure not to block on accept if (serverSocket_ != null) { try { serverSocket_.setSoTimeout(0); } catch (SocketException sx) { LOGGER.error("Could not set socket timeout.", sx); } } } ========================================== ========================================== WARNING 4: empty handler for exception: java.net.SocketException There are log messages.. Line: 70, File: "parquet/org/apache/thrift/transport/TSocket.java" private void initSocket() { socket_ = new Socket(); try { socket_.setSoLinger(false, 0); socket_.setTcpNoDelay(true); socket_.setSoTimeout(timeout_); } catch (SocketException sx) { LOGGER.error("Could not configure socket.", sx); } } ========================================== ========================================== WARNING 5: empty handler for exception: java.net.SocketException There are log messages.. Line: 134, File: "parquet/org/apache/thrift/transport/TSocket.java" public void setTimeout(int timeout) { timeout_ = timeout; try { socket_.setSoTimeout(timeout); } catch (SocketException sx) { LOGGER.warn("Could not set socket timeout.", sx); } } ========================================== ========================================== WARNING 1: empty handler for exception: java.lang.Throwable There are log messages.. Line: 484, File: "tachyon/org/apache/thrift/server/AbstractNonblockingServer.java" try { processorFactory_.getProcessor(inTrans).process(inProt, outProt); responseReady(); return; } catch (TException te) { LOGGER.warn("Exception while invoking!", te); } catch (Throwable t) { LOGGER.error("Unexpected throwable while invoking!", t); } ========================================== > Suspicious exception handlers > ----------------------------- > > Key: SPARK-4863 > URL: https://issues.apache.org/jira/browse/SPARK-4863 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.1.1 > Reporter: Ding Yuan > > Following up with the discussion in > https://issues.apache.org/jira/browse/SPARK-1148, I am creating a new JIRA to > report the suspicious exception handlers detected by our tool aspirator on > spark-1.1.1. > {noformat} > ========================================== > WARNING: TODO; in handler. > Line: 129, File: "org/apache/thrift/transport/TNonblockingServerSocket.java" > 122: public void registerSelector(Selector selector) { > 123: try { > 124: // Register the server socket channel, indicating an interest in > 125: // accepting new connections > 126: serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT); > 127: } catch (ClosedChannelException e) { > 128: // this shouldn't happen, ideally... > 129: // TODO: decide what to do with this. > 130: } > 131: } > ========================================== > ========================================== > WARNING: TODO; in handler. > Line: 1583, File: "org/apache/spark/SparkContext.scala" > 1578: val scheduler = try { > 1579: val clazz = > Class.forName("org.apache.spark.scheduler.cluster.YarnClusterScheduler") > 1580: val cons = clazz.getConstructor(classOf[SparkContext]) > 1581: cons.newInstance(sc).asInstanceOf[TaskSchedulerImpl] > 1582: } catch { > 1583: // TODO: Enumerate the exact reasons why it can fail > 1584: // But irrespective of it, it means we cannot proceed ! > 1585: case e: Exception => { > 1586: throw new SparkException("YARN mode not available ?", e) > 1587: } > ========================================== > ========================================== > WARNING 1: empty handler for exception: java.lang.Exception > THERE IS NO LOG MESSAGE!!! > Line: 75, File: "org/apache/spark/repl/ExecutorClassLoader.scala" > try { > val pathInDirectory = name.replace('.', '/') + ".class" > val inputStream = { > if (fileSystem != null) { > fileSystem.open(new Path(directory, pathInDirectory)) > } else { > if (SparkEnv.get.securityManager.isAuthenticationEnabled()) { > val uri = new URI(classUri + "/" + urlEncode(pathInDirectory)) > val newuri = Utils.constructURIForAuthentication(uri, > SparkEnv.get.securityManager) > newuri.toURL().openStream() > } else { > new URL(classUri + "/" + urlEncode(pathInDirectory)).openStream() > } > } > } > val bytes = readAndTransformClass(name, inputStream) > inputStream.close() > Some(defineClass(name, bytes, 0, bytes.length)) > } catch { > case e: Exception => None > } > ========================================== > ========================================== > WARNING 1: empty handler for exception: java.io.IOException > THERE IS NO LOG MESSAGE!!! > Line: 275, File: "org/apache/spark/util/Utils.scala" > try { > dir = new File(root, "spark-" + UUID.randomUUID.toString) > if (dir.exists() || !dir.mkdirs()) { > dir = null > } > } catch { case e: IOException => ; } > ========================================== > ========================================== > WARNING 1: empty handler for exception: java.lang.InterruptedException > THERE IS NO LOG MESSAGE!!! > Line: 172, File: "parquet/org/apache/thrift/server/TNonblockingServer.java" > protected void joinSelector() { > // wait until the selector thread exits > try { > selectThread_.join(); > } catch (InterruptedException e) { > // for now, just silently ignore. technically this means we'll have > less of > // a graceful shutdown as a result. > } > } > ========================================== > ========================================== > WARNING 2: empty handler for exception: java.net.SocketException > There are log messages.. > Line: 111, File: > "parquet/org/apache/thrift/transport/TNonblockingSocket.java" > public void setTimeout(int timeout) { > try { > socketChannel_.socket().setSoTimeout(timeout); > } catch (SocketException sx) { > LOGGER.warn("Could not set socket timeout.", sx); > } > } > ========================================== > ========================================== > WARNING 3: empty handler for exception: java.net.SocketException > There are log messages.. > Line: 103, File: "parquet/org/apache/thrift/transport/TServerSocket.java" > public void listen() throws TTransportException { > // Make sure not to block on accept > if (serverSocket_ != null) { > try { > serverSocket_.setSoTimeout(0); > } catch (SocketException sx) { > LOGGER.error("Could not set socket timeout.", sx); > } > } > } > ========================================== > ========================================== > WARNING 4: empty handler for exception: java.net.SocketException > There are log messages.. > Line: 70, File: "parquet/org/apache/thrift/transport/TSocket.java" > private void initSocket() { > socket_ = new Socket(); > try { > socket_.setSoLinger(false, 0); > socket_.setTcpNoDelay(true); > socket_.setSoTimeout(timeout_); > } catch (SocketException sx) { > LOGGER.error("Could not configure socket.", sx); > } > } > ========================================== > ========================================== > WARNING 5: empty handler for exception: java.net.SocketException > There are log messages.. > Line: 134, File: "parquet/org/apache/thrift/transport/TSocket.java" > public void setTimeout(int timeout) { > timeout_ = timeout; > try { > socket_.setSoTimeout(timeout); > } catch (SocketException sx) { > LOGGER.warn("Could not set socket timeout.", sx); > } > } > ========================================== > ========================================== > WARNING 1: empty handler for exception: java.lang.Throwable > There are log messages.. > Line: 484, File: > "tachyon/org/apache/thrift/server/AbstractNonblockingServer.java" > try { > processorFactory_.getProcessor(inTrans).process(inProt, outProt); > responseReady(); > return; > } catch (TException te) { > LOGGER.warn("Exception while invoking!", te); > } catch (Throwable t) { > LOGGER.error("Unexpected throwable while invoking!", t); > } > ========================================== > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org