[GitHub] spark issue #16614: [SPARK-19260] Spaces or "%20" in path parameter are not ...

2017-02-07 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/16614 Could you please merge to master and close this PR? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #16614: [SPARK-19260] Spaces or "%20" in path parameter are not ...

2017-02-05 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/16614 @srowen Happy Chinese New Year ! Sorry for the late reply. It sounds more sensible to test this behavior everywhere since all the test cases used the directory, right? Thank you for your

[GitHub] spark issue #16614: [SPARK-19260] Spaces or "%20" in path parameter are not ...

2017-02-09 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/16614 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #16614: [SPARK-19260] Spaces in path parameter are not co...

2017-01-17 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/16614 [SPARK-19260] Spaces in path parameter are not correctly handled with… JIRA Issue: https://issues.apache.org/jira/browse/SPARK-19260 ## What changes were proposed in this pull

[GitHub] spark pull request #16614: [SPARK-19260] Spaces in path parameter are not co...

2017-01-20 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/16614#discussion_r97032463 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -93,10 +94,12 @@ private[history] class FsHistoryProvider

[GitHub] spark pull request #16439: [SPARK-19026]SPARK_LOCAL_DIRS(multiple directorie...

2017-01-06 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/16439#discussion_r95052089 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala --- @@ -445,12 +445,24 @@ private[deploy] class Worker

[GitHub] spark pull request #16439: [SPARK-19026]SPARK_LOCAL_DIRS(multiple directorie...

2017-01-06 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/16439#discussion_r95052088 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala --- @@ -445,12 +445,24 @@ private[deploy] class Worker

[GitHub] spark issue #16439: [SPARK-19026]SPARK_LOCAL_DIRS(multiple directories on di...

2017-01-04 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/16439 @srowen Thanks for your advise, as @vanzin said I did not configure an unwritable directory, but it can become unwritable for other reasons (e.g. disk fills up or disk damaged suddenly etc

[GitHub] spark issue #16439: [SPARK-19026]SPARK_LOCAL_DIRS(multiple directories on di...

2017-01-03 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/16439 @srowen i do not think there should be a fatal error since some of SPARK_LOCAL_DIRS can be written successfully, even there is only one of SPARK_LOCAL_DIRS can be written, the application

[GitHub] spark pull request #16439: [SPARK-19026]SPARK_LOCAL_DIRS(multiple directorie...

2016-12-29 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/16439 [SPARK-19026]SPARK_LOCAL_DIRS(multiple directories on different disks) cannot be deleted JIRA Issue: https://issues.apache.org/jira/browse/SPARK-19026 SPARK_LOCAL_DIRS (Standalone

[GitHub] spark pull request #16439: [SPARK-19026]SPARK_LOCAL_DIRS(multiple directorie...

2017-01-02 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/16439#discussion_r94352892 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala --- @@ -445,12 +445,24 @@ private[deploy] class Worker

[GitHub] spark pull request #16439: [SPARK-19026]SPARK_LOCAL_DIRS(multiple directorie...

2017-01-02 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/16439#discussion_r94352890 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala --- @@ -445,12 +445,24 @@ private[deploy] class Worker

[GitHub] spark pull request #16484: [SPARK-19083]sbin/start-history-server.sh script ...

2017-01-06 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/16484#discussion_r94911586 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala --- @@ -445,12 +445,24 @@ private[deploy] class Worker

[GitHub] spark pull request #16484: Sh

2017-01-05 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/16484 Sh JIRA Issue: https://issues.apache.org/jira/browse/SPARK-19083# sbin/start-history-server.sh script use of $@ without quotes, this will affect the length of args which used

[GitHub] spark issue #16484: [SPARK-19083]sbin/start-history-server.sh script use of ...

2017-01-06 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/16484 @jerryshao Thank you for your advice. I checked off all the scripts under bin and sbin. It seems ok. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #17452: [SPARK-20123]$SPARK_HOME variable might have spac...

2017-03-28 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/17452 [SPARK-20123]$SPARK_HOME variable might have spaces in it(e.g. $SPARK… JIRA Issue: https://issues.apache.org/jira/browse/SPARK-20123 ## What changes were proposed in this pull

[GitHub] spark pull request #17452: [SPARK-20123][build]$SPARK_HOME variable might ha...

2017-03-28 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17452#discussion_r108400132 --- Diff: R/check-cran.sh --- @@ -20,14 +20,14 @@ set -o pipefail set -e -FWDIR="$(cd `dirname "${BASH_SOURCE[

[GitHub] spark pull request #17452: [SPARK-20123][build]$SPARK_HOME variable might ha...

2017-03-30 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17452#discussion_r108898299 --- Diff: dev/make-distribution.sh --- @@ -217,43 +217,43 @@ fi # Make R package - this is used for both CRAN release and packing R layout

[GitHub] spark pull request #17452: [SPARK-20123][build]$SPARK_HOME variable might ha...

2017-03-30 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17452#discussion_r108969951 --- Diff: dev/make-distribution.sh --- @@ -217,43 +217,43 @@ fi # Make R package - this is used for both CRAN release and packing R layout

[GitHub] spark pull request #17452: [SPARK-20123][build]$SPARK_HOME variable might ha...

2017-03-30 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17452#discussion_r108978377 --- Diff: dev/make-distribution.sh --- @@ -217,43 +217,43 @@ fi # Make R package - this is used for both CRAN release and packing R layout

[GitHub] spark issue #17452: [SPARK-20123][build]$SPARK_HOME variable might have spac...

2017-03-29 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/17452 OK,will do. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17452: [SPARK-20123][build]$SPARK_HOME variable might ha...

2017-03-29 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17452#discussion_r108623175 --- Diff: R/check-cran.sh --- @@ -40,7 +40,7 @@ fi if [ -d "$SPARK_JARS_DIR" ]; then # Build a zip file containing the sour

[GitHub] spark pull request #17493: [SPARK-20173][SQL] Throw NullPointerException whe...

2017-03-31 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/17493 [SPARK-20173][SQL] Throw NullPointerException when HiveThriftServer2 is shutdown ## What changes were proposed in this pull request? Throw NullPointerException when HiveThriftServer2

[GitHub] spark pull request #17452: [SPARK-20123][build]$SPARK_HOME variable might ha...

2017-03-30 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17452#discussion_r109072645 --- Diff: dev/make-distribution.sh --- @@ -217,43 +217,43 @@ fi # Make R package - this is used for both CRAN release and packing R layout

[GitHub] spark issue #17493: [SPARK-20173][SQL] Throw NullPointerException when HiveT...

2017-03-31 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/17493 @srowen yes, i will fix it right now. Thanks! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #17496: [SPARK-20173][SQL][hive-thriftserver] Throw NullP...

2017-03-31 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17496#discussion_r109137993 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala --- @@ -82,7 +82,9 @@ object

[GitHub] spark pull request #17493: [SPARK-20173][SQL] Throw NullPointerException whe...

2017-03-31 Thread zuotingbing
Github user zuotingbing closed the pull request at: https://github.com/apache/spark/pull/17493 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #17496: [SPARK-20173][SQL][hive-thriftserver] Throw NullP...

2017-03-31 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/17496 [SPARK-20173][SQL][hive-thriftserver] Throw NullPointerException when HiveThriftServer2 is shutdown ## What changes were proposed in this pull request? Throw NullPointerException

[GitHub] spark pull request #17638: [SPARK-20338][CORE]Spaces in spark.eventLog.dir a...

2017-04-20 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17638#discussion_r112595663 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -50,22 +49,22 @@ import org.apache.spark.util.{JsonProtocol

[GitHub] spark pull request #17638: [SPARK-20338][CORE]Spaces in spark.eventLog.dir a...

2017-04-20 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17638#discussion_r112593597 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -405,9 +405,7 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark pull request #17638: [SPARK-20338][CORE]Spaces in spark.eventLog.dir a...

2017-04-20 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17638#discussion_r112596553 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -50,22 +49,22 @@ import org.apache.spark.util.{JsonProtocol

[GitHub] spark pull request #17638: [SPARK-20338][CORE]Spaces in spark.eventLog.dir a...

2017-04-20 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17638#discussion_r112601515 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -50,22 +49,22 @@ import org.apache.spark.util.{JsonProtocol

[GitHub] spark pull request #17638: [SPARK-20338][CORE]Spaces in spark.eventLog.dir a...

2017-04-20 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17638#discussion_r112594056 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -50,22 +49,22 @@ import org.apache.spark.util.{JsonProtocol

[GitHub] spark pull request #17638: [SPARK-20338][CORE]Spaces in spark.eventLog.dir a...

2017-04-20 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17638#discussion_r112597030 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -405,9 +405,7 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark pull request #17638: [SPARK-20338][CORE]Spaces in spark.eventLog.dir a...

2017-04-14 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/17638 [SPARK-20338][CORE]Spaces in spark.eventLog.dir are not correctly handled ## What changes were proposed in this pull request? Spaces in spark.eventLog.dir are not correctly handled

[GitHub] spark issue #17638: [SPARK-20338][CORE]Spaces in spark.eventLog.dir are not ...

2017-04-19 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/17638 cc @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #18812: [SPARK-21606][SQL]HiveThriftServer2 catches OOMs ...

2017-08-02 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/18812 [SPARK-21606][SQL]HiveThriftServer2 catches OOMs on request threads ## What changes were proposed in this pull request? Refer to Hive, ThriftCLIService methods such as ExecuteStatement

[GitHub] spark pull request #18811: [Spark-21604][SQL]Error class name for log, and i...

2017-08-01 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/18811 [Spark-21604][SQL]Error class name for log, and if the object extends Logging, i suggest to remove the var LOG which is useless. ## What changes were proposed in this pull request

[GitHub] spark pull request #18619: [SPARK-21397][BUILD]Maven shade plugin adding dep...

2017-08-02 Thread zuotingbing
Github user zuotingbing closed the pull request at: https://github.com/apache/spark/pull/18619 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #18816: [SPARK-21611][SQL]Error class name for log in sev...

2017-08-02 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/18816 [SPARK-21611][SQL]Error class name for log in several classes. ## What changes were proposed in this pull request? Error class name for log in several classes. such as: `2017-08-02

[GitHub] spark issue #18811: [SPARK-21604][SQL] if the object extends Logging, i sugg...

2017-08-02 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/18811 Thanks @srowen , will do. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18816: [SPARK-21611][SQL]Error class name for log in sev...

2017-08-02 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/18816#discussion_r131037188 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala --- @@ -272,7 +272,7 @@ private[hive

[GitHub] spark issue #18811: [SPARK-21604][SQL] if the object extends Logging, i sugg...

2017-08-03 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/18811 ok, have done. Thanks @srowen . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18619: [SPARK-21397][BUILD]Maven shade plugin adding dependency...

2017-07-13 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/18619 I just use `mvn checkstyle:check` ,which contains output files with `basedir`, to check whether the `basedir` is correct with this PR. Could you please specify the affect exactly as you

[GitHub] spark issue #18619: [SPARK-21397][BUILD]Maven shade plugin adding dependency...

2017-07-13 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/18619 Yes this PR is a trivial Improvement for me. i will closed it if this has side-effects exactly. Thanks. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #18619: [SPARK-21397][BUILD]Maven shade plugin adding dep...

2017-07-13 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/18619 [SPARK-21397][BUILD]Maven shade plugin adding dependency-reduced-pom.xml to … …base directory ## What changes were proposed in this pull request? 1. run `./build/mvn

[GitHub] spark issue #18619: [SPARK-21397][BUILD]Maven shade plugin adding dependency...

2017-07-13 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/18619 @srowen Thanks for your time. I am not sure the open issue, which is mentioned in the note, whether has bean resolved. as the note said > setting a value for this parame

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-11 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r116137309 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -494,4 +494,34 @@ class InsertIntoHiveTableSuite

[GitHub] spark issue #17858: [SPARK-20594][SQL]The staging directory should be a chil...

2017-05-14 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/17858 Thank you all. Delete the branch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-10 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115893632 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -494,4 +494,34 @@ class InsertIntoHiveTableSuite

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-11 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115974195 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -494,4 +494,45 @@ class InsertIntoHiveTableSuite

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-11 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115973153 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -494,4 +494,34 @@ class InsertIntoHiveTableSuite

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-11 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115974006 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -494,4 +494,45 @@ class InsertIntoHiveTableSuite

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-09 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115415586 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -97,12 +97,23 @@ case class InsertIntoHiveTable

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-09 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115640830 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -97,12 +97,23 @@ case class InsertIntoHiveTable

[GitHub] spark pull request #18012: [SPARK-20779][Examples]The ASF header placed in a...

2017-05-16 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/18012 [SPARK-20779][Examples]The ASF header placed in an incorrect location in some files. ## What changes were proposed in this pull request? The license is not at the top in some files

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-09 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115638060 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -97,12 +97,23 @@ case class InsertIntoHiveTable

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-09 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115637682 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -97,12 +97,23 @@ case class InsertIntoHiveTable

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-10 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115672879 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -97,12 +97,23 @@ case class InsertIntoHiveTable

[GitHub] spark pull request #17858: [SPARK-20594][SQL]The staging directory should be...

2017-05-10 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/17858#discussion_r115674151 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -494,4 +494,45 @@ class InsertIntoHiveTableSuite

[GitHub] spark pull request #18158: Lack of the most important case about the test of...

2017-05-31 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/18158 Lack of the most important case about the test of resolveURI in UtilsSuite, and add it as needed. ## What changes were proposed in this pull request? add the most important case

[GitHub] spark pull request #18158: Lack of the most important case about the test of...

2017-05-31 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/18158#discussion_r119302580 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -473,7 +474,7 @@ class UtilsSuite extends SparkFunSuite

[GitHub] spark pull request #18158: Lack of an important case about the test of resol...

2017-05-31 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/18158#discussion_r119511454 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -473,7 +474,7 @@ class UtilsSuite extends SparkFunSuite

[GitHub] spark pull request #18158: [SPARK-20936][CORE]Lack of an important case abou...

2017-06-01 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/18158#discussion_r119534690 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -473,7 +474,7 @@ class UtilsSuite extends SparkFunSuite

[GitHub] spark pull request #18158: [SPARK-20936][CORE]Lack of an important case abou...

2017-06-01 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/18158#discussion_r119533271 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -473,7 +474,7 @@ class UtilsSuite extends SparkFunSuite

[GitHub] spark pull request #18158: [SPARK-20936][CORE]Lack of an important case abou...

2017-06-01 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/18158#discussion_r119535585 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -473,7 +474,7 @@ class UtilsSuite extends SparkFunSuite

[GitHub] spark issue #18285: [SPARK-20338][CORE]Spaces in spark.eventLog.dir are not ...

2017-06-16 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/18285 Thanks @vanzin ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18285: [SPARK-20338]Spaces in spark.eventLog.dir are not...

2017-06-13 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/18285 [SPARK-20338]Spaces in spark.eventLog.dir are not correctly handled ## What changes were proposed in this pull request? “spark.eventLog.dir” supports with space characters

[GitHub] spark pull request #17858: [SPARK-20594]The staging directory should be appe...

2017-05-04 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/17858 [SPARK-20594]The staging directory should be appended with ".hive-staging" to avoid being deleted if we set hive.exec.stagingdir under the table directory without start with ".&q

[GitHub] spark issue #17858: [SPARK-20594][SQL]The staging directory should be append...

2017-05-04 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/17858 In this case, Hive will create the staging directory under the table directory, and when moving staging directory to table directory, Hive will still empty the table directory, but will exclude

[GitHub] spark issue #17858: [SPARK-20594][SQL]The staging directory should be append...

2017-05-08 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/17858 @gatorsmile My production environment is spark 2.0.2 and test successful. Is there something be changed since 2.0.2 for this case? Thanks! --- If your project is set up for it, you can reply

[GitHub] spark issue #17858: [SPARK-20594][SQL]The staging directory should be append...

2017-05-08 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/17858 yes i tried the same thing in Hive, got the same error: `2017-05-08T13:48:04,634 ERROR exec.Task (:()) - Failed with exception Unable to move source hdfs://nameservice/hive/test_table1

[GitHub] spark issue #17858: [SPARK-20594][SQL]The staging directory should be append...

2017-05-08 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/17858 @gatorsmile it seems my mistake, i will try to fix this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #19277: [SPARK-22058][COREthe BufferedInputStream will no...

2017-09-19 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/19277 [SPARK-22058][COREthe BufferedInputStream will not be closed if an exception occurs. ## What changes were proposed in this pull request? EventLoggingListener use `val in = new

[GitHub] spark pull request #19275: [SPARK-22057][CORE]Provide to update the applicat...

2017-09-19 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/19275 [SPARK-22057][CORE]Provide to update the applications in the HistoryPage which the event log files have not been existed. ## What changes were proposed in this pull request

[GitHub] spark pull request #19277: [SPARK-22058][COREthe BufferedInputStream will no...

2017-09-19 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/19277#discussion_r139665504 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -347,15 +347,14 @@ private[spark] object

[GitHub] spark pull request #19277: [SPARK-22058][CORE]the BufferedInputStream will n...

2017-09-19 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/19277#discussion_r139878136 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -351,11 +351,11 @@ private[spark] object

[GitHub] spark issue #19277: [SPARK-22058][CORE]the BufferedInputStream will not be c...

2017-09-20 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19277 I am not sure. If we move this line `val in = new BufferedInputStream(fs.open(log))` into try~catch, we should define `var in: BufferedInputStream = null` before, and use `catch { case e

[GitHub] spark pull request #19277: [SPARK-22058][CORE]the BufferedInputStream will n...

2017-09-20 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/19277#discussion_r139902219 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -351,11 +351,11 @@ private[spark] object

[GitHub] spark issue #19277: [SPARK-22058][CORE]the BufferedInputStream will not be c...

2017-09-20 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19277 @jerryshao if this line `new BufferedInputStream(fs.open(log))` throws exception, it mean we do not need to close the object of BufferedInputStream because it new failed

[GitHub] spark issue #19300: [SPARK-22082][SparkR]Spelling mistake: "choosen" in API ...

2017-09-22 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19300 if we find some mistakes likes this , shouldn't we to report it at first? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #19300: [SPARK-22082][SparkR]Spelling mistake: "choosen" in API ...

2017-09-22 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19300 @cloud-fan @gatorsmile Could you please help to review this and merge it to master? Thanks. --- - To unsubscribe, e-mail

[GitHub] spark pull request #19300: [SPARK-22082][SparkR]Spelling mistake: choosen in...

2017-09-20 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/19300 [SPARK-22082][SparkR]Spelling mistake: choosen in API doc of R. ## What changes were proposed in this pull request? "choosen" should be "chosen" in AP

[GitHub] spark pull request #19277: [SPARK-22058][CORE]the BufferedInputStream will n...

2017-09-20 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/19277#discussion_r140137420 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -351,14 +351,14 @@ private[spark] object

[GitHub] spark pull request #19275: [SPARK-22057][CORE]Provide to update the applicat...

2017-09-21 Thread zuotingbing
Github user zuotingbing closed the pull request at: https://github.com/apache/spark/pull/19275 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19300: [SPARK-22082][SparkR]Spelling mistake: "choosen" in API ...

2017-09-21 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19300 > or don't bother and close this I have not found other typos and sorry i'm not agree with you. Since this spelling mistake affects the API doc in official website, we should

[GitHub] spark issue #19398: [SPARK-22174][CORE]Support to automatically create the d...

2017-09-29 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19398 Do you know what is the reason to choice to manually create? What about changing it to create it automatically? Thanks

[GitHub] spark pull request #19398: [SPARK-22174][CORE]Support to automatically creat...

2017-09-29 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/19398 [SPARK-22174][CORE]Support to automatically create the directory where the event logs go (`spark.eventLog.dir`) ## What changes were proposed in this pull request? `2017-09-30 09

[GitHub] spark issue #19398: [SPARK-22174][CORE]Support to automatically create the d...

2017-09-29 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19398 Got it, thanks all. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #19398: [SPARK-22174][CORE]Support to automatically creat...

2017-09-29 Thread zuotingbing
Github user zuotingbing closed the pull request at: https://github.com/apache/spark/pull/19398 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #19034: [SPARK-21821][CORE]Support to force kill the Coar...

2017-08-24 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/19034 [SPARK-21821][CORE]Support to force kill the CoarseGrainedExecutorBackend process which is likely be orphaned. ## What changes were proposed in this pull request? In out

[GitHub] spark issue #19034: [SPARK-21821][CORE]Support to force kill the CoarseGrain...

2017-08-24 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19034 Hmm, this problem is common , i think some helpful tool scripts is to be expected to make spark more robust, and are useful for users. :) Similarly In Hive, there is a tool script

[GitHub] spark issue #19034: [SPARK-21821][CORE]Support to force kill the CoarseGrain...

2017-08-24 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19034 @srowen @vanzin could you please take a look at this? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #19034: [SPARK-21821][CORE]Support to force kill the Coar...

2017-08-29 Thread zuotingbing
Github user zuotingbing closed the pull request at: https://github.com/apache/spark/pull/19034 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #19841: [SPARK-22642][SQL] the createdTempDir will not be...

2017-11-28 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/19841 [SPARK-22642][SQL] the createdTempDir will not be deleted if an exception occurs ## What changes were proposed in this pull request? We found staging directories

[GitHub] spark issue #19841: [SPARK-22642][SQL] the createdTempDir will not be delete...

2017-11-30 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19841 Yes i am also confused why the diff is so big but it reported by git originally. --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-17 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19989 we can find the cache size of FileSystem `deleteOnExit` will keep increasing. [ ![mshot](https://user-images.githubusercontent.com/24823338/34095036-d1fcc408-e40a-11e7-9599-2acdd96da2d9

[GitHub] spark issue #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Server

2017-12-17 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/19989 as i debug, every time when i connect to thrift server through beeline, the `SessionState.start(state)` will be called two times. one is in `HiveSessionImpl:open` , **another

[GitHub] spark pull request #19989: [SPARK-22793][SQL]Memory leak in Spark Thrift Ser...

2017-12-15 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/19989 [SPARK-22793][SQL]Memory leak in Spark Thrift Server ## What changes were proposed in this pull request? 1. Start HiveThriftServer2. 2. Connect to thriftserver through beeline

[GitHub] spark pull request #20025: [SPARK-22837][SQL]Session timeout checker does no...

2017-12-19 Thread zuotingbing
GitHub user zuotingbing opened a pull request: https://github.com/apache/spark/pull/20025 [SPARK-22837][SQL]Session timeout checker does not work in SessionManager. ## What changes were proposed in this pull request? Currently we do not call the `super.init(hiveConf

  1   2   >