[GitHub] spark pull request #19074: [SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-upl...

2017-08-28 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/19074 [SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-uploading remote resources in yarn client mode ## What changes were proposed in this pull request? This is a backport PR to fix issue of

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17357 **[Test build #81200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81200/testReport)** for PR 17357 at commit

[GitHub] spark pull request #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, whil...

2017-08-28 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/17357#discussion_r135699350 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/StandaloneRestServer.scala --- @@ -139,7 +139,9 @@ private[rest] class

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19048 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19048 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81195/ Test FAILed. ---

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19048 **[Test build #81195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81195/testReport)** for PR 19048 at commit

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135694280 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,317 @@ +/* + * Licensed under the Apache License, Version

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135694119 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,317 @@ +/* + * Licensed under the Apache License, Version

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135695472 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License, Version

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135695625 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License, Version

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135693340 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,317 @@ +/* + * Licensed under the Apache License, Version

[GitHub] spark pull request #17014: [SPARK-18608][ML] Fix double-caching in ML algori...

2017-08-28 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/17014#discussion_r135695930 --- Diff: mllib/src/main/scala/org/apache/spark/ml/Predictor.scala --- @@ -85,6 +86,10 @@ abstract class Predictor[ M <:

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135692852 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License, Version

[GitHub] spark issue #19055: [SPARK-21839][SQL] Support SQL config for ORC compressio...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19055 **[Test build #81199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81199/testReport)** for PR 19055 at commit

[GitHub] spark issue #19072: [SPARK-17133][ML][FOLLOW-UP] Add convenient method `asBi...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19072 **[Test build #81198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81198/testReport)** for PR 19072 at commit

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-08-28 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/18902 @yanboliang Although dispointed by DF's performance, I also approve the choice of DF just for less code. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #19055: [SPARK-21839][SQL] Support SQL config for ORC com...

2017-08-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19055#discussion_r135692633 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -322,6 +322,15 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135692546 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License, Version

[GitHub] spark pull request #17014: [SPARK-18608][ML] Fix double-caching in ML algori...

2017-08-28 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/17014#discussion_r135692470 --- Diff: mllib/src/main/scala/org/apache/spark/ml/Predictor.scala --- @@ -85,6 +86,10 @@ abstract class Predictor[ M <:

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135689398 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135689367 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135689239 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135689225 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135689194 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135689169 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135689113 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135689067 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135688838 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License,

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r135688946 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,292 @@ +/* + * Licensed under the Apache License,

[GitHub] spark issue #19073: [SPARK-21855][DEPLOY] Should print error when upload sam...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19073 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18317: [SPARK-21113][CORE] Read ahead input stream to amortize ...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18317 **[Test build #81197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81197/testReport)** for PR 18317 at commit

[GitHub] spark pull request #19073: [SPARK][DEPLOY] Should print error when upload sa...

2017-08-28 Thread caneGuy
GitHub user caneGuy opened a pull request: https://github.com/apache/spark/pull/19073 [SPARK][DEPLOY] Should print error when upload same file multiple tim… …es to yarn ## What changes were proposed in this pull request? Now when submit job with yarn,and upload

[GitHub] spark issue #19072: [SPARK-17133][ML][FOLLOW-UP] Add convenient method `asBi...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19072 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81196/ Test FAILed. ---

[GitHub] spark issue #19072: [SPARK-17133][ML][FOLLOW-UP] Add convenient method `asBi...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19072 **[Test build #81196 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81196/testReport)** for PR 19072 at commit

[GitHub] spark issue #19072: [SPARK-17133][ML][FOLLOW-UP] Add convenient method `asBi...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19072 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19064: [SPARK-21848][SQL] Add trait UDFType to identify user-de...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19064 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81192/ Test PASSed. ---

[GitHub] spark issue #19064: [SPARK-21848][SQL] Add trait UDFType to identify user-de...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19064 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19064: [SPARK-21848][SQL] Add trait UDFType to identify user-de...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19064 **[Test build #81192 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81192/testReport)** for PR 19064 at commit

[GitHub] spark pull request #19055: [SPARK-21839][SQL] Support SQL config for ORC com...

2017-08-28 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19055#discussion_r135686725 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -322,6 +322,15 @@ object SQLConf { .booleanConf

[GitHub] spark issue #19072: [SPARK-17133][ML][FOLLOW-UP] Add convenient method `asBi...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19072 **[Test build #81196 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81196/testReport)** for PR 19072 at commit

[GitHub] spark pull request #19072: [SPARK-17133][ML][FOLLOW-UP] Add convenient metho...

2017-08-28 Thread WeichenXu123
GitHub user WeichenXu123 opened a pull request: https://github.com/apache/spark/pull/19072 [SPARK-17133][ML][FOLLOW-UP] Add convenient method `asBinary` for casting to BinaryLogisticRegressionSummary ## What changes were proposed in this pull request? add an "asBinary"

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-08-28 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/17862 Catching up here... To make sure I caught the decisions made in the discussion above, is it correct that this PR will: * Add support for squared hinge loss, and use that as the default (which

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19048 **[Test build #81195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81195/testReport)** for PR 19048 at commit

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19048 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19048 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81194/ Test FAILed. ---

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19048 **[Test build #81194 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81194/testReport)** for PR 19048 at commit

[GitHub] spark pull request #19049: [WEB-UI]Add the 'master' column to identify the t...

2017-08-28 Thread guoxiaolongzte
Github user guoxiaolongzte closed the pull request at: https://github.com/apache/spark/pull/19049 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #19049: [WEB-UI]Add the 'master' column to identify the type of ...

2017-08-28 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/19049 Ok, Thank you for your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18917: [SPARK-21704][DOC] Add the description of 'sbin/s...

2017-08-28 Thread guoxiaolongzte
Github user guoxiaolongzte closed the pull request at: https://github.com/apache/spark/pull/18917 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19048 **[Test build #81194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81194/testReport)** for PR 19048 at commit

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19048 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19048 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81193/ Test FAILed. ---

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19048 **[Test build #81193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81193/testReport)** for PR 19048 at commit

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19048 **[Test build #81193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81193/testReport)** for PR 19048 at commit

[GitHub] spark issue #19048: [SPARK-21834] Incorrect executor request in case of dyna...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/19048 @jiangxb1987 - I agree with you. I do not have the context or history to comment on that. Unfortunately, the api has been designed that way and book keeping of target number of executors is done

[GitHub] spark issue #19069: [MINOR][SQL][TEST]Test shuffle hash join while is not ex...

2017-08-28 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/19069 @sameeragarwal,Thank you for your suggest, modify it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #19026: [SPARK-21681][ML] fix bug of MLOR do not work cor...

2017-08-28 Thread WeichenXu123
Github user WeichenXu123 closed the pull request at: https://github.com/apache/spark/pull/19026 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #19018: [SPARK-21801][SPARKR][TEST] unit test randomly fail with...

2017-08-28 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19018 @felixcheung In Jenkins Log I only found Random forest and Decision Tree failed, random forest failed more frequently. thanks! --- If your project is set up for it, you can reply to this

[GitHub] spark pull request #19068: [SPARK-21428][SQL][FOLLOWUP] Reused state should ...

2017-08-28 Thread yaooqinn
Github user yaooqinn commented on a diff in the pull request: https://github.com/apache/spark/pull/19068#discussion_r135676677 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala --- @@ -81,11 +81,7 @@ private[hive] object

[GitHub] spark issue #19061: [SPARK-21568][CORE] ConsoleProgressBar should only be en...

2017-08-28 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19061 >If such a thing as a non-Spark repl-like application exists, it wouldn't be getting the progress bar by default, for example, because its default log level is "INFO" in Spark, something that

[GitHub] spark issue #18962: [SPARK-21714][CORE][YARN] Avoiding re-uploading remote r...

2017-08-28 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18962 Sorry I missed the comments, I will file another PR against branch 2.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #18111: [SPARK-20886][CORE] HadoopMapReduceCommitProtocol to han...

2017-08-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18111 OK. LGTM. I think this anyway should issue warnings: ``` WARN org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Output Path is null in setupJob() WARN

[GitHub] spark issue #19064: [SPARK-21848][SQL] Add trait UDFType to identify user-de...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19064 **[Test build #81192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81192/testReport)** for PR 19064 at commit

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-08-28 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/17357 LGTM, cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17357 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17357 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81191/ Test PASSed. ---

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17357 **[Test build #81191 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81191/testReport)** for PR 17357 at commit

[GitHub] spark issue #19061: [SPARK-21568][CORE] ConsoleProgressBar should only be en...

2017-08-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19061 I see. Thank you for the guide! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19061: [SPARK-21568][CORE] ConsoleProgressBar should only be en...

2017-08-28 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19061 > Is it okay we change this behavior like this? It's not optimal, but at the same time, the existing behavior wasn't really correctly advertised anyway. If such a thing as a non-Spark

[GitHub] spark issue #16422: [SPARK-17642] [SQL] support DESC EXTENDED/FORMATTED tabl...

2017-08-28 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/16422 @gatorsmile Will update in this week. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18421: [SPARK-21213][SQL] Support collecting partition-level st...

2017-08-28 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/18421 @mbasmanova Great work! I was really busy in the past two months so I didn't have time to look at this. Thank @gatorsmile for reviewing and merging this PR! --- If your project is set up for

[GitHub] spark issue #19061: [SPARK-21568][CORE] ConsoleProgressBar should only be en...

2017-08-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19061 BTW, @vanzin . I was waiting your comment because @jerryshao is worried about 3rd party relations. > other repl-like projects may actually require this Is it okay we change this

[GitHub] spark issue #19061: [SPARK-21568][CORE] ConsoleProgressBar should only be en...

2017-08-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19061 Thank you for reiview, @vanzin . Yep. @jerryshao also advised me the missing point. I'll try to make it more general. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #19061: [SPARK-21568][CORE] ConsoleProgressBar should onl...

2017-08-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19061#discussion_r135663178 --- Diff: docs/configuration.md --- @@ -804,7 +804,7 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #19061: [SPARK-21568][CORE] ConsoleProgressBar should onl...

2017-08-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19061#discussion_r135663076 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -537,6 +537,11 @@ object SparkSubmit extends CommandLineUtils { }

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-08-28 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19050 also cc @cloud-fan for review. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19050: [SPARK-21835][SQL] RewritePredicateSubquery should not p...

2017-08-28 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19050 ping @hvanhovell Does the current change look good for you? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17980 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81190/ Test PASSed. ---

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17980 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17980 **[Test build #81190 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81190/testReport)** for PR 17980 at commit

[GitHub] spark pull request #19065: [SPARK-21729][ML][TEST] Generic test for Probabil...

2017-08-28 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19065#discussion_r135655220 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/ProbabilisticClassifierSuite.scala --- @@ -91,4 +94,54 @@ object

[GitHub] spark pull request #19065: [SPARK-21729][ML][TEST] Generic test for Probabil...

2017-08-28 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19065#discussion_r135653663 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/ProbabilisticClassifierSuite.scala --- @@ -91,4 +94,54 @@ object

[GitHub] spark pull request #19065: [SPARK-21729][ML][TEST] Generic test for Probabil...

2017-08-28 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19065#discussion_r135656421 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/ProbabilisticClassifierSuite.scala --- @@ -91,4 +94,54 @@ object

[GitHub] spark pull request #19065: [SPARK-21729][ML][TEST] Generic test for Probabil...

2017-08-28 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19065#discussion_r135653421 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/ProbabilisticClassifierSuite.scala --- @@ -18,7 +18,10 @@ package

[GitHub] spark pull request #19065: [SPARK-21729][ML][TEST] Generic test for Probabil...

2017-08-28 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19065#discussion_r135653044 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/DecisionTreeClassifierSuite.scala --- @@ -262,6 +262,9 @@ class

[GitHub] spark pull request #19065: [SPARK-21729][ML][TEST] Generic test for Probabil...

2017-08-28 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19065#discussion_r135653479 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/ProbabilisticClassifierSuite.scala --- @@ -18,7 +18,10 @@ package

[GitHub] spark issue #19049: [WEB-UI]Add the 'master' column to identify the type of ...

2017-08-28 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19049 > For the 'Spark User' column. It would be identical in almost all use cases. That is absolutely not true in a true multi-user environment. --- If your project is set up for it, you can

[GitHub] spark issue #19064: [SPARK-21848][SQL] Add trait UDFType to identify user-de...

2017-08-28 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19064 I'm +1 on this, it should be useful to pattern match the trait instead of handling ScalaUDF/ScalaUDAF/PythonUDF... in future optimize rules. --- If your project is set up for it, you can reply

[GitHub] spark issue #18731: [SPARK-20990][SQL] Read all JSON documents in files when...

2017-08-28 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/18731 The CI error actually is a timeout. I don't know the reason of the timeout or how to retrigger it. If I should do anything please just tell me. If there is anything I can do for this PR just

[GitHub] spark issue #19071: [MINOR][ML] Document treatment of instance weights in lo...

2017-08-28 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19071 CC @WeichenXu123 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #19069: [MINOR][SQL][TEST]Test shuffle hash join while is...

2017-08-28 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/19069#discussion_r135645027 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/JoinBenchmark.scala --- @@ -212,18 +212,17 @@ class JoinBenchmark extends

[GitHub] spark issue #19064: [SPARK-21848][SQL] Add trait UDFType to identify user-de...

2017-08-28 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19064 I'm +1 on this, currently we are pattern matching `ScalaUDF` in optimize rules which is suboptional. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #19071: [MINOR][ML] Document treatment of instance weights in lo...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19071 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81189/ Test PASSed. ---

[GitHub] spark issue #19071: [MINOR][ML] Document treatment of instance weights in lo...

2017-08-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19071 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #19071: [MINOR][ML] Document treatment of instance weights in lo...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19071 **[Test build #81189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81189/testReport)** for PR 19071 at commit

[GitHub] spark pull request #18837: [Spark-20812][Mesos] Add secrets support to the d...

2017-08-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18837#discussion_r135640958 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -383,15 +385,57 @@

[GitHub] spark pull request #18837: [Spark-20812][Mesos] Add secrets support to the d...

2017-08-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18837#discussion_r135641205 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/deploy/mesos/config.scala --- @@ -58,12 +58,43 @@ package object config {

[GitHub] spark issue #19056: [SPARK-21765] Check that optimization doesn't affect isS...

2017-08-28 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/19056 (Dummy comment to test JIRA linkage) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17357: [SPARK-20025][CORE] Ignore SPARK_LOCAL* env, while deplo...

2017-08-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17357 **[Test build #81191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81191/testReport)** for PR 17357 at commit

[GitHub] spark issue #18317: [SPARK-21113][CORE] Read ahead input stream to amortize ...

2017-08-28 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/18317 Sure, I will address @mridulm comment in next few days. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

  1   2   3   >