[GitHub] spark issue #17564: [MINOR][DOCS] Fix spacings in Structured Streaming Progr...

2017-04-11 Thread dongjinleekr
Github user dongjinleekr commented on the issue: https://github.com/apache/spark/pull/17564 @srowen I just completed the review. I could not find any other typos from this document. :) --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #17265: [SPARK-19924] [SQL] Handle InvocationTargetExcept...

2017-04-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17265#discussion_r111030752 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -94,7 +96,13 @@ private[spark] class

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-04-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r111025724 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/ValidatorParams.scala --- @@ -67,6 +71,39 @@ private[ml] trait ValidatorParams extends

[GitHub] spark issue #17610: [SPARK-20131][Core]Use a separate lock for StandaloneSch...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17610 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17610: [SPARK-20131][Core]Use a separate lock for StandaloneSch...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17610 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75712/ Test PASSed. ---

[GitHub] spark issue #17610: [SPARK-20131][Core]Use a separate lock for StandaloneSch...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17610 **[Test build #75712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75712/testReport)** for PR 17610 at commit

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17546 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17546 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75713/ Test PASSed. ---

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17546 **[Test build #75713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75713/testReport)** for PR 17546 at commit

[GitHub] spark pull request #13932: [SPARK-15354] [CORE] Topology aware block replica...

2017-04-11 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/13932#discussion_r111015013 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockReplicationPolicySuite.scala --- @@ -68,7 +68,60 @@ class BlockReplicationPolicySuite

[GitHub] spark issue #17609: [SPARK-20296][TRIVIAL][DOCS] Count distinct error messag...

2017-04-11 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/17609 jenkins test this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17609: [SPARK-20296][TRIVIAL][DOCS] Count distinct error messag...

2017-04-11 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/17609 That's correct. LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17130 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75714/ Test PASSed. ---

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17130 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17130 **[Test build #75714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75714/testReport)** for PR 17130 at commit

[GitHub] spark issue #17280: [SPARK-19939] [ML] Add support for association rules in ...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17280 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17280: [SPARK-19939] [ML] Add support for association rules in ...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17280 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75715/ Test FAILed. ---

[GitHub] spark issue #17280: [SPARK-19939] [ML] Add support for association rules in ...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17280 **[Test build #75715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75715/testReport)** for PR 17280 at commit

[GitHub] spark issue #17606: [SPARK-20291][SQL] NaNvl(FloatType, NullType) should not...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17606 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75711/ Test PASSed. ---

[GitHub] spark issue #17606: [SPARK-20291][SQL] NaNvl(FloatType, NullType) should not...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17606 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17606: [SPARK-20291][SQL] NaNvl(FloatType, NullType) should not...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17606 **[Test build #75711 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75711/testReport)** for PR 17606 at commit

[GitHub] spark issue #17330: [SPARK-19993][SQL] Caching logical plans containing subq...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17330 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17330: [SPARK-19993][SQL] Caching logical plans containing subq...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17330 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75710/ Test PASSed. ---

[GitHub] spark issue #17330: [SPARK-19993][SQL] Caching logical plans containing subq...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17330 **[Test build #75710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75710/testReport)** for PR 17330 at commit

[GitHub] spark issue #17611: [SPARK-20298][SparkR][MINOR] fixed spelling mistake "cha...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17611 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17280: [SPARK-19939] [ML] Add support for association rules in ...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17280 **[Test build #75715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75715/testReport)** for PR 17280 at commit

[GitHub] spark pull request #17611: [SPARK-20298][SparkR][MINOR] fixed spelling mista...

2017-04-11 Thread bdwyer2
GitHub user bdwyer2 opened a pull request: https://github.com/apache/spark/pull/17611 [SPARK-20298][SparkR][MINOR] fixed spelling mistake "charactor" ## What changes were proposed in this pull request? Fixed spelling of "charactor" ## How was this patch tested?

[GitHub] spark issue #17109: [SPARK-19740][MESOS]Add support in Spark to pass arbitra...

2017-04-11 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/17109 @srowen We do appreciate your help with Mesos commits, and generally find you responsive. I have a habit of pinging you for merges because you seemed to have stepped in once @andrewor14 stepped

[GitHub] spark issue #17609: [SPARK-20296][TRIVIAL][DOCS] Count distinct error messag...

2017-04-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17609 CC @tdas to confirm that's correct --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17130 **[Test build #75714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75714/testReport)** for PR 17130 at commit

[GitHub] spark issue #17109: [SPARK-19740][MESOS]Add support in Spark to pass arbitra...

2017-04-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17109 @yanji84 I have no experience with Mesos and am not, in general, a reviewer for this code, and don't follow Mesos changes. That's why. I pitch in to help merge but would appreciate others stepping

[GitHub] spark issue #17610: [SPARK-20131][Core]Use a separate lock for StandaloneSch...

2017-04-11 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/17610 > Isn't it not depending on this being locked in super class methods invoked in the invocation subtree ? I don't get it. But I think the stack trace shows why this dead-lock happens. ---

[GitHub] spark issue #17610: [SPARK-20131][Core]Use a separate lock for StandaloneSch...

2017-04-11 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/17610 @mridulm yeah, I was thinking to just change `stopping` to a AtomicBoolean flag. However, it changes the semantics a little, e.g., the second `stop` will return at once when the first `stop` is

[GitHub] spark issue #17469: [SPARK-20132][Docs] Add documentation for column string ...

2017-04-11 Thread map222
Github user map222 commented on the issue: https://github.com/apache/spark/pull/17469 @HyukjinKwon I have updated the documentation to pass the tests (I didn't realize the tests actually executed the documented code!). I was able to build Spark locally, and run `./python/run-tests.py

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-11 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17222 Thanks for working on this, one thing is the return's from the `UDFRegistration` functions seem like they won't do much since the functions they call on the sqlContext don't have returns (and they

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-04-11 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16845 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #17606: [SPARK-20291][SQL] NaNvl(FloatType, NullType) sho...

2017-04-11 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17606#discussion_r110991443 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercionSuite.scala --- @@ -656,14 +656,20 @@ class TypeCoercionSuite

[GitHub] spark pull request #16845: [SPARK-19505][Python] AttributeError on Exception...

2017-04-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16845 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17610: [SPARK-20131][Core]Use a separate lock for StandaloneSch...

2017-04-11 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/17610 Isn't it not depending on 'this' being locked in super class method ? If it is not, then why not simply restrict lock to changing of `stopping` flag (if already set, return, else set and proceed

[GitHub] spark pull request #17130: [SPARK-19791] [ML] Add doc and example for fpgrow...

2017-04-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17130#discussion_r110990099 --- Diff: examples/src/main/python/ml/fpgrowth_example.py --- @@ -0,0 +1,48 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK][DOCS] import sort and autopep8 on...

2017-04-11 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14830 Sure, if you have a chance to rebase & check if any other changes are needed that would be useful. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-11 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110989528 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,426 @@ +/*

[GitHub] spark issue #17546: [SPARK-20233] [SQL] Apply star-join filter heuristics to...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17546 **[Test build #75713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75713/testReport)** for PR 17546 at commit

[GitHub] spark issue #17605: [SPARK-20290][MINOR][PYTHON][SQL] Add PySpark wrapper fo...

2017-04-11 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17605 LGTM thanks for adding this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-11 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110988847 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/StarSchemaDetection.scala --- @@ -76,7 +76,7 @@ case class

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-11 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110987936 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -218,28 +220,48 @@ object

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-11 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110987408 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +349,110 @@ object

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-11 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110987294 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,428 @@ +/*

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-11 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110987218 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -218,28 +220,48 @@ object

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-04-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r110986801 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/ValidatorParams.scala --- @@ -67,6 +71,39 @@ private[ml] trait ValidatorParams extends

[GitHub] spark issue #17463: [SPARK-20131][DStream][Test] Flaky Test: org.apache.spar...

2017-04-11 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/17463 Could you close this one, please? I submitted #17610 to fix the root issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #17610: [SPARK-20131][Core]Use a separate lock for Standa...

2017-04-11 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/17610 [SPARK-20131][Core]Use a separate lock for StandaloneSchedulerBackend.stop ## What changes were proposed in this pull request? `o.a.s.streaming.StreamingContextSuite.SPARK-18560 Receiver

[GitHub] spark pull request #17265: [SPARK-19924] [SQL] Handle InvocationTargetExcept...

2017-04-11 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/17265#discussion_r110985107 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -94,7 +96,13 @@ private[spark] class

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-11 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110984951 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -54,8 +54,6 @@ case class

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-04-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r110983267 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tuning/CrossValidatorSuite.scala --- @@ -121,6 +121,33 @@ class CrossValidatorSuite }

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-04-11 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15125 @felixcheung ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #17586: [SPARK-20249][ML][PYSPARK] Add summary for Linear...

2017-04-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17586#discussion_r110982571 --- Diff: python/pyspark/ml/classification.py --- @@ -172,6 +172,47 @@ def intercept(self): """ return self._call_java("intercept")

[GitHub] spark pull request #17586: [SPARK-20249][ML][PYSPARK] Add summary for Linear...

2017-04-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17586#discussion_r110982055 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala --- @@ -355,6 +368,19 @@ object LinearSVCModel extends

[GitHub] spark pull request #17586: [SPARK-20249][ML][PYSPARK] Add summary for Linear...

2017-04-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17586#discussion_r110982433 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala --- @@ -287,6 +290,16 @@ class LinearSVCModel private[classification] (

[GitHub] spark pull request #17586: [SPARK-20249][ML][PYSPARK] Add summary for Linear...

2017-04-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17586#discussion_r110981812 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala --- @@ -355,6 +368,19 @@ object LinearSVCModel extends

[GitHub] spark pull request #17586: [SPARK-20249][ML][PYSPARK] Add summary for Linear...

2017-04-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17586#discussion_r110978675 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala --- @@ -287,6 +290,16 @@ class LinearSVCModel private[classification] (

[GitHub] spark pull request #17586: [SPARK-20249][ML][PYSPARK] Add summary for Linear...

2017-04-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17586#discussion_r110980991 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/LinearSVCExample.scala --- @@ -44,6 +44,12 @@ object LinearSVCExample { //

[GitHub] spark pull request #17586: [SPARK-20249][ML][PYSPARK] Add summary for Linear...

2017-04-11 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/17586#discussion_r110981511 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala --- @@ -355,6 +368,19 @@ object LinearSVCModel extends

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-04-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r110981439 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala --- @@ -100,31 +108,60 @@ class CrossValidator @Since("1.2.0")

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-04-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r110980824 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/ValidatorParams.scala --- @@ -67,6 +71,39 @@ private[ml] trait ValidatorParams extends

[GitHub] spark issue #17109: [SPARK-19740][MESOS]Add support in Spark to pass arbitra...

2017-04-11 Thread yanji84
Github user yanji84 commented on the issue: https://github.com/apache/spark/pull/17109 @srowen is there anything else holding this up? why does it take so long? thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #17606: [SPARK-20291][SQL] NaNvl(FloatType, NullType) should not...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17606 **[Test build #75711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75711/testReport)** for PR 17606 at commit

[GitHub] spark issue #17606: [SPARK-20291][SQL] NaNvl(FloatType, NullType) should not...

2017-04-11 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/17606 +cc @cloud-fan @gatorsmile @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #16774: [SPARK-19357][ML] Adding parallel model evaluatio...

2017-04-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/16774#discussion_r110979877 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/ValidatorParams.scala --- @@ -67,6 +71,39 @@ private[ml] trait ValidatorParams extends

[GitHub] spark pull request #17606: [SPARK-20291][SQL] NaNvl(FloatType, NullType) sho...

2017-04-11 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/17606#discussion_r110979361 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -571,6 +571,7 @@ object TypeCoercion {

[GitHub] spark issue #17330: [SPARK-19993][SQL] Caching logical plans containing subq...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17330 **[Test build #75710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75710/testReport)** for PR 17330 at commit

[GitHub] spark pull request #17330: [SPARK-19993][SQL] Caching logical plans containi...

2017-04-11 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/17330#discussion_r110977254 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala --- @@ -670,4 +677,139 @@ class CachedTableSuite extends QueryTest with

[GitHub] spark pull request #17330: [SPARK-19993][SQL] Caching logical plans containi...

2017-04-11 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/17330#discussion_r110977330 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala --- @@ -670,4 +677,139 @@ class CachedTableSuite extends QueryTest with

[GitHub] spark issue #17609: [SPARK-20296][TRIVIAL][DOCS] Count distinct error messag...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17609 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17609: [SPARK-20296][TRIVIAL][DOCS] Count distinct error...

2017-04-11 Thread jtoka
GitHub user jtoka opened a pull request: https://github.com/apache/spark/pull/17609 [SPARK-20296][TRIVIAL][DOCS] Count distinct error message for streaming ## What changes were proposed in this pull request? Update count distinct error message for streaming datasets/dataframes

[GitHub] spark pull request #17330: [SPARK-19993][SQL] Caching logical plans containi...

2017-04-11 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/17330#discussion_r110976069 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala --- @@ -76,6 +76,13 @@ class CachedTableSuite extends QueryTest with

[GitHub] spark pull request #17604: [SPARK-20289][SQL] Use StaticInvoke to box primit...

2017-04-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17604 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17604: [SPARK-20289][SQL] Use StaticInvoke to box primitive typ...

2017-04-11 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17604 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17295: [SPARK-19556][core] Do not encrypt block manager data in...

2017-04-11 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/17295 > LGTM, cc @mallman to check the unmap part LGTM, too. Sorry for the late reply... I've been away the past two weeks. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17436 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75709/ Test PASSed. ---

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17436 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17436 **[Test build #75709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75709/testReport)** for PR 17436 at commit

[GitHub] spark issue #7652: [SPARK-9312] [ML] Added max confidence factor to OneVsRes...

2017-04-11 Thread AxenGitHub
Github user AxenGitHub commented on the issue: https://github.com/apache/spark/pull/7652 Is there any news on this branch? we would benefit a lot from this feature. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java String t...

2017-04-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17527 The general idea is to leave any lower-casing that affects strings in the user program alone, to use the locale-sensitive `toLowerCase()`. This is more conservative. All of the changes should only

[GitHub] spark issue #17455: [Spark-20044][Web UI] Support Spark UI behind front-end ...

2017-04-11 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/17455 It seems it didn't take @holdenk ok, @vanzin mind okaying this to test? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17608: [SPARK-20293][WEB UI][History]In the page of 'jobs' or '...

2017-04-11 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/17608 @guoxiaolongzte This seems familiar, are you using the latest version of Knox with your Spark UI? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java String t...

2017-04-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17527 I am wondering what is the reason some of `toLowerCase` is changed, but the others remain unchanged? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #17598: [SPARK-20284][CORE] Make {Des,S}erializationStrea...

2017-04-11 Thread superbobry
Github user superbobry commented on a diff in the pull request: https://github.com/apache/spark/pull/17598#discussion_r110950456 --- Diff: core/src/main/scala/org/apache/spark/serializer/Serializer.scala --- @@ -125,7 +125,7 @@ abstract class SerializerInstance { * A stream

[GitHub] spark issue #17459: [SPARK-20109][MLlib] Rewrote toBlockMatrix method on Ind...

2017-04-11 Thread johnc1231
Github user johnc1231 commented on the issue: https://github.com/apache/spark/pull/17459 @viirya Do you have any more comments on this, or are you happy with it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #9571: [SPARK-11373] [CORE] Add metrics to the History Server an...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9571 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75708/ Test PASSed. ---

[GitHub] spark issue #9571: [SPARK-11373] [CORE] Add metrics to the History Server an...

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9571 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #9571: [SPARK-11373] [CORE] Add metrics to the History Server an...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9571 **[Test build #75708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75708/testReport)** for PR 9571 at commit

[GitHub] spark issue #17491: [SPARK-20175][SQL] Exists should not be evaluated in Joi...

2017-04-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17491 I think the current approach will have a LeftSemi join for this Exists subquery. Is it far from the optimal access plan you said? --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17436 **[Test build #75709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75709/testReport)** for PR 17436 at commit

[GitHub] spark issue #17150: [SPARK-19810][BUILD][CORE] Remove support for Scala 2.10

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17150 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75707/ Test PASSed. ---

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-04-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r110923167 --- Diff: core/src/main/java/org/apache/spark/memory/MemoryConsumer.java --- @@ -41,7 +41,7 @@ protected MemoryConsumer(TaskMemoryManager taskMemoryManager,

[GitHub] spark issue #17150: [SPARK-19810][BUILD][CORE] Remove support for Scala 2.10

2017-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17150 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17150: [SPARK-19810][BUILD][CORE] Remove support for Scala 2.10

2017-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17150 **[Test build #75707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75707/testReport)** for PR 17150 at commit

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-11 Thread nsyca
Github user nsyca commented on the issue: https://github.com/apache/spark/pull/17520 @cloud-fan: would you be interested in reviewing this PR since I have not heard from @hvanhovell for a while? Note this is a WIP and I want to hear your feedback on the issues I put in the comments

[GitHub] spark issue #17491: [SPARK-20175][SQL] Exists should not be evaluated in Joi...

2017-04-11 Thread nsyca
Github user nsyca commented on the issue: https://github.com/apache/spark/pull/17491 @cloud-fan wrote: "How useful is this optimization? It only works when Exists has no condition, is that a common case?" One of the common cases of this usage is an application of ACL where

<    1   2   3   4   >