[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17078 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73503/ Test PASSed. ---

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17078 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17076: [SPARK-19745][ML] SVCAggregator captures coefficients in...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17076 **[Test build #73505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73505/testReport)** for PR 17076 at commit

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17078 **[Test build #73503 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73503/testReport)** for PR 17078 at commit

[GitHub] spark issue #12896: [SPARK-14489][ML][PYSPARK] ALS unknown user/item predict...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12896 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12896: [SPARK-14489][ML][PYSPARK] ALS unknown user/item predict...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12896 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73521/ Test PASSed. ---

[GitHub] spark issue #12896: [SPARK-14489][ML][PYSPARK] ALS unknown user/item predict...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12896 **[Test build #73521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73521/testReport)** for PR 12896 at commit

[GitHub] spark issue #16965: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16965 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16965: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16965 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73512/ Test PASSed. ---

[GitHub] spark issue #16965: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16965 **[Test build #73512 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73512/testReport)** for PR 16965 at commit

[GitHub] spark issue #16793: [SPARK-19454][PYTHON][SQL] DataFrame.replace improvement...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16793 **[Test build #73524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73524/testReport)** for PR 16793 at commit

[GitHub] spark issue #17031: [SPARK-19702][MESOS] Add suppress/revive support to the ...

2017-02-27 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/17031 "The only way to fix this generally is to implement some periodic timer that calls reviveOffers() if there are queued/pending drivers to be scheduled. This can be chatty and complicates the code, so

[GitHub] spark issue #17032: [SPARK-19460][SparkR]:Update dataset used in R documenta...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17032 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17077 **[Test build #73523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73523/testReport)** for PR 17077 at commit

[GitHub] spark issue #17032: [SPARK-19460][SparkR]:Update dataset used in R documenta...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17032 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73509/ Test PASSed. ---

[GitHub] spark issue #17032: [SPARK-19460][SparkR]:Update dataset used in R documenta...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17032 **[Test build #73509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73509/testReport)** for PR 17032 at commit

[GitHub] spark pull request #16929: [SPARK-19595][SQL] Support json array in from_jso...

2017-02-27 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/16929#discussion_r103262990 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2969,11 +2969,27 @@ object functions { } /** - *

[GitHub] spark pull request #17031: [SPARK-19702][MESOS] Add suppress/revive support ...

2017-02-27 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/17031#discussion_r103261951 --- Diff: resource-managers/mesos/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterSchedulerSuite.scala --- @@ -48,45 +48,50 @@ class

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17077 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73522/ Test FAILed. ---

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17077 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17077 **[Test build #73504 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73504/testReport)** for PR 17077 at commit

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17077 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17077 **[Test build #73522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73522/testReport)** for PR 17077 at commit

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17077 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73504/ Test FAILed. ---

[GitHub] spark issue #16793: [SPARK-19454][PYTHON][SQL] DataFrame.replace improvement...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16793 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16793: [SPARK-19454][PYTHON][SQL] DataFrame.replace improvement...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16793 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73517/ Test FAILed. ---

[GitHub] spark issue #16793: [SPARK-19454][PYTHON][SQL] DataFrame.replace improvement...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16793 **[Test build #73517 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73517/testReport)** for PR 16793 at commit

[GitHub] spark pull request #16867: [SPARK-16929] Improve performance when check spec...

2017-02-27 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16867#discussion_r103258743 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -909,16 +921,17 @@ private[spark] class TaskSetManager( var

[GitHub] spark issue #14299: Ensure broadcasted variables are destroyed even in case ...

2017-02-27 Thread AnthonyTruchet
Github user AnthonyTruchet commented on the issue: https://github.com/apache/spark/pull/14299 @thunterdb Copy that, working on it and sorry for the acknowledge delay. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-02-27 Thread datumbox
Github user datumbox commented on the issue: https://github.com/apache/spark/pull/17059 Yeah, Scala Long matches. Here is the "stand-alone" script that I used to confirm that everything works ok (tested on Spark 2.1): ```scala import org.apache.spark.sql.types._ import

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17077 **[Test build #73522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73522/testReport)** for PR 17077 at commit

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistices to improve...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistices to improve...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16677 **[Test build #73518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73518/testReport)** for PR 16677 at commit

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistices to improve...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73518/ Test FAILed. ---

[GitHub] spark issue #17031: [SPARK-19702][MESOS] Add suppress/revive support to the ...

2017-02-27 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/17031 @mgummelt Yes they should look at the logs but how do they know this is something that requires action from their side and not a cluster issue or anything else. It should be documented since it is

[GitHub] spark pull request #17076: [SPARK-19745][ML] SVCAggregator captures coeffici...

2017-02-27 Thread AnthonyTruchet
Github user AnthonyTruchet commented on a diff in the pull request: https://github.com/apache/spark/pull/17076#discussion_r103254026 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala --- @@ -440,19 +440,9 @@ private class LinearSVCAggregator(

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17081 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73500/ Test FAILed. ---

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17081 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17081 **[Test build #73500 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73500/testReport)** for PR 17081 at commit

[GitHub] spark issue #16809: [SPARK-19463][SQL]refresh cache after the InsertIntoHado...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16809 **[Test build #73516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73516/testReport)** for PR 16809 at commit

[GitHub] spark issue #12896: [SPARK-14489][ML][PYSPARK] ALS unknown user/item predict...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12896 **[Test build #73521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73521/testReport)** for PR 12896 at commit

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistices to improve...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16677 **[Test build #73518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73518/testReport)** for PR 16677 at commit

[GitHub] spark issue #16793: [SPARK-19454][PYTHON][SQL] DataFrame.replace improvement...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16793 **[Test build #73517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73517/testReport)** for PR 16793 at commit

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16478 **[Test build #73519 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73519/testReport)** for PR 16478 at commit

[GitHub] spark issue #16910: [SPARK-19575][SQL]Reading from or writing to a hive serd...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16910 **[Test build #73513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73513/testReport)** for PR 16910 at commit

[GitHub] spark issue #15505: [SPARK-18890][CORE] Move task serialization from the Tas...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15505 **[Test build #73520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73520/testReport)** for PR 15505 at commit

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16867 **[Test build #73514 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73514/testReport)** for PR 16867 at commit

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17078 **[Test build #73503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73503/testReport)** for PR 17078 at commit

[GitHub] spark issue #16965: [SPARK-18450][ML] Scala API Change for LSH AND-amplifica...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16965 **[Test build #73512 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73512/testReport)** for PR 16965 at commit

[GitHub] spark issue #17071: [SPARK-15615][SQL][BUILD][FOLLOW-UP] Replace deprecated ...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17071 **[Test build #73506 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73506/testReport)** for PR 17071 at commit

[GitHub] spark issue #16819: [SPARK-16441][YARN] Set maxNumExecutor depends on yarn c...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16819 **[Test build #73515 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73515/testReport)** for PR 16819 at commit

[GitHub] spark issue #17032: [SPARK-19460][SparkR]:Update dataset used in R documenta...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17032 **[Test build #73509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73509/testReport)** for PR 17032 at commit

[GitHub] spark issue #17079: [SPARK-19748][SQL]refresh function has a wrong order to ...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17079 **[Test build #73502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73502/testReport)** for PR 17079 at commit

[GitHub] spark issue #17080: [SPARK-19739][CORE] propagate S3 session token to cluser

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17080 **[Test build #73501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73501/testReport)** for PR 17080 at commit

[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16990 **[Test build #73511 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73511/testReport)** for PR 16990 at commit

[GitHub] spark issue #17077: [SPARK-16931][PYTHON][SQL] Add Python wrapper for bucket...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17077 **[Test build #73504 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73504/testReport)** for PR 17077 at commit

[GitHub] spark issue #17039: [SPARK-19710][SQL][TESTS] Fix ordering of rows in query ...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17039 **[Test build #73508 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73508/testReport)** for PR 17039 at commit

[GitHub] spark issue #17076: [SPARK-19745][ML] SVCAggregator captures coefficients in...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17076 **[Test build #73505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73505/testReport)** for PR 17076 at commit

[GitHub] spark issue #17001: [SPARK-19667][SQL]create table with hiveenabled in defau...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17001 **[Test build #73510 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73510/testReport)** for PR 17001 at commit

[GitHub] spark issue #17052: [SPARK-19690][SS] Join a streaming DataFrame with a batc...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17052 **[Test build #73507 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73507/testReport)** for PR 17052 at commit

[GitHub] spark issue #17082: [SPARK-19749][SS] Name socket source with a meaningful n...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17082 **[Test build #73499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73499/testReport)** for PR 17082 at commit

[GitHub] spark issue #17083: [SPARK-19750][UI][branch-2.1] Fix redirect issue from ht...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17083 **[Test build #73498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73498/testReport)** for PR 17083 at commit

[GitHub] spark pull request #17076: [SPARK-19745][ML] SVCAggregator captures coeffici...

2017-02-27 Thread AnthonyTruchet
Github user AnthonyTruchet commented on a diff in the pull request: https://github.com/apache/spark/pull/17076#discussion_r103249915 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala --- @@ -440,19 +440,9 @@ private class LinearSVCAggregator(

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-02-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17081 **[Test build #73500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73500/testReport)** for PR 17081 at commit

[GitHub] spark issue #17076: [SPARK-19745][ML] SVCAggregator captures coefficients in...

2017-02-27 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17076 Jenkins test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17078: [SPARK-19746][ML] Faster indexing for logistic aggregato...

2017-02-27 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/17078 Jenkins test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17076: [SPARK-19745][ML] SVCAggregator captures coeffici...

2017-02-27 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17076#discussion_r103248918 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala --- @@ -440,19 +440,9 @@ private class LinearSVCAggregator(

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-02-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17059 This one would be good for @mlnick to look at. The Scala Long will match java.lang.Number right? if so it works even if there's another conversion there, but maybe that one is trivial, and

[GitHub] spark issue #16793: [SPARK-19454][PYTHON][SQL] DataFrame.replace improvement...

2017-02-27 Thread zero323
Github user zero323 commented on the issue: https://github.com/apache/spark/pull/16793 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...

2017-02-27 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/16990 OK. I have reverted `set hive.mapreduce.job.reduces.speculative.execution=false` to `set hive.mapred.reduce.tasks.speculative.execution=false`. --- If your project is set up for it, you can reply

[GitHub] spark pull request #17078: [SPARK-19746][ML] Faster indexing for logistic ag...

2017-02-27 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17078#discussion_r103239934 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1447,7 +1447,7 @@ private class LogisticAggregator(

[GitHub] spark pull request #17078: [SPARK-19746][ML] Faster indexing for logistic ag...

2017-02-27 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17078#discussion_r103236865 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -1447,7 +1447,7 @@ private class LogisticAggregator(

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-02-27 Thread datumbox
Github user datumbox commented on the issue: https://github.com/apache/spark/pull/17059 @srowen The following snippet handles explicitly Longs. It can be rewritten to remove duplicate code by introducing bools for overflow detection but I don't think it is worth it. In theory you can

[GitHub] spark pull request #17012: [SPARK-19677][SS] Renaming a file atop an existin...

2017-02-27 Thread vitillo
Github user vitillo commented on a diff in the pull request: https://github.com/apache/spark/pull/17012#discussion_r103219157 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -274,6 +274,11 @@

[GitHub] spark pull request #17068: [SPARK-19709][SQL] Read empty file with CSV data ...

2017-02-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17068#discussion_r103214603 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVInferSchema.scala --- @@ -40,7 +41,19 @@ private[csv] object

[GitHub] spark issue #15505: [SPARK-18890][CORE] Move task serialization from the Tas...

2017-02-27 Thread witgo
Github user witgo commented on the issue: https://github.com/apache/spark/pull/15505 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16867: [WIP][SPARK-16929] Improve performance when check specul...

2017-02-27 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16867 @squito Thanks a lot for your comments : ) >When check speculatable tasks in TaskSetManager, current code scan all task infos and sort durations of successful tasks in O(NlogN) time

[GitHub] spark issue #17071: [SPARK-15615][SQL][BUILD][FOLLOW-UP] Replace deprecated ...

2017-02-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17071 (I put a test here - https://github.com/apache/spark/pull/17071/files#diff-7e47859dbd409cc39f2908615fbd07ffR419) --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #16990: [SPARK-19660][CORE][SQL] Replace the configuratio...

2017-02-27 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/16990#discussion_r103200223 --- Diff: python/pyspark/tests.py --- @@ -1515,12 +1515,12 @@ def test_oldhadoop(self): conf = {

[GitHub] spark issue #17039: [SPARK-19710][SQL][TESTS] Fix ordering of rows in query ...

2017-02-27 Thread robbinspg
Github user robbinspg commented on the issue: https://github.com/apache/spark/pull/17039 Jenkins retest please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16971: [SPARK-19573][SQL] Make NaN/null handling consistent in ...

2017-02-27 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/16971 ping @MLnick @gatorsmile @thunterdb --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17039: [SPARK-19710][SQL][TESTS] Fix ordering of rows in query ...

2017-02-27 Thread robbinspg
Github user robbinspg commented on the issue: https://github.com/apache/spark/pull/17039 @gatorsmile I'm glad it wasn't just me that found it complex ;-) I've modified the patch to remove an unnecessary change as that query was not ordered and the test suite code handles

[GitHub] spark issue #17083: [SPARK-19750][UI][branch-2.1] Fix redirect issue from ht...

2017-02-27 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17083 Not sure why Jenkins test cannot be started automatically. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #17076: [SPARK-19745][ML] SVCAggregator captures coeffici...

2017-02-27 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/17076#discussion_r103187723 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala --- @@ -440,19 +440,9 @@ private class LinearSVCAggregator(

[GitHub] spark pull request #14731: [SPARK-17159] [streaming]: optimise check for new...

2017-02-27 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/14731#discussion_r103187577 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala --- @@ -140,7 +137,7 @@ class FileInputDStream[K, V, F <:

[GitHub] spark issue #17080: [SPARK-19739][CORE] propagate S3 session token to cluser

2017-02-27 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/17080 LGTM. Verified option name in `org.apache.hadoop.fs.s3a.Constants` file; env var name in `com.amazonaws.SDKGlobalConfiguration' --- If your project is set up for it, you can reply to

[GitHub] spark issue #17083: [SPARK-19750][UI][branch-2.1] Fix redirect issue from ht...

2017-02-27 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17083 Due to the change of (https://github.com/apache/spark/pull/16625), the issue is obsolete. So it effects spark 2.1 and 2.0. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #16990: [SPARK-19660][CORE][SQL] Replace the configuratio...

2017-02-27 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/16990#discussion_r103185158 --- Diff: sql/hive/src/test/resources/ql/src/test/queries/clientpositive/smb_mapjoin_25.q --- @@ -19,7 +19,7 @@ select * from (select a.key from

[GitHub] spark pull request #16809: [SPARK-19463][SQL]refresh cache after the InsertI...

2017-02-27 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/16809#discussion_r103185139 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -132,6 +132,9 @@ case class

[GitHub] spark pull request #14731: [SPARK-17159] [streaming]: optimise check for new...

2017-02-27 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/14731#discussion_r103184528 --- Diff: docs/streaming-programming-guide.md --- @@ -615,35 +615,114 @@ which creates a DStream from text data received over a TCP socket

[GitHub] spark issue #17071: [SPARK-15615][SQL][BUILD][FOLLOW-UP] Replace deprecated ...

2017-02-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17071 Sure, sounds better and I can't find a reason to not follow. Let me maybe add single small Java one somewhere because the deprecated Java one calls the deprecated Scala one. --- If your

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-02-27 Thread datumbox
Github user datumbox commented on the issue: https://github.com/apache/spark/pull/17059 Ignore my comment about duplicate code. It can be written to avoid it. I will investigate handling the SQL decimal types as you recommended and I will update the code tonight. --- If your

[GitHub] spark pull request #16990: [SPARK-19660][CORE][SQL] Replace the configuratio...

2017-02-27 Thread jcamachor
Github user jcamachor commented on a diff in the pull request: https://github.com/apache/spark/pull/16990#discussion_r103184073 --- Diff: sql/hive/src/test/resources/ql/src/test/queries/clientpositive/smb_mapjoin_25.q --- @@ -19,7 +19,7 @@ select * from (select a.key from

[GitHub] spark pull request #14731: [SPARK-17159] [streaming]: optimise check for new...

2017-02-27 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/14731#discussion_r103183646 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala --- @@ -140,7 +137,7 @@ class FileInputDStream[K, V, F

[GitHub] spark pull request #16990: [SPARK-19660][CORE][SQL] Replace the configuratio...

2017-02-27 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/16990#discussion_r103183030 --- Diff: sql/hive/src/test/resources/ql/src/test/queries/clientpositive/smb_mapjoin_25.q --- @@ -19,7 +19,7 @@ select * from (select a.key from

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-02-27 Thread datumbox
Github user datumbox commented on the issue: https://github.com/apache/spark/pull/17059 @srowen: Thanks for the comments. We are getting there. :) I will handle the Long case as you suggest. If you think people use SQL decimal types, I can include them at the end of

[GitHub] spark issue #17083: [SPARK-19750][UI][branch-2.1] Fix redirect issue from ht...

2017-02-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17083 Was this fixed otherwise in master, or did some other change make it obsolete? just trying to link this to whatever reason it's only a problem in 2.1, for the record. --- If your project is set up

[GitHub] spark pull request #16990: [SPARK-19660][CORE][SQL] Replace the configuratio...

2017-02-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16990#discussion_r103179898 --- Diff: python/pyspark/tests.py --- @@ -1515,12 +1515,12 @@ def test_oldhadoop(self): conf = {

[GitHub] spark pull request #16990: [SPARK-19660][CORE][SQL] Replace the configuratio...

2017-02-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16990#discussion_r103180859 --- Diff: sql/hive/src/test/resources/ql/src/test/queries/clientpositive/smb_mapjoin_25.q --- @@ -19,7 +19,7 @@ select * from (select a.key from

[GitHub] spark pull request #17083: [SPARK-19750][UI][branch-2.1] Fix redirect issue ...

2017-02-27 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/17083 [SPARK-19750][UI][branch-2.1] Fix redirect issue from http to https ## What changes were proposed in this pull request? If spark ui port (4040) is not set, it will choose port number 0,

<    1   2   3   4   5   6   7   >