[GitHub] spark issue #16826: Fork SparkSession with option to inherit a copy of the S...

2017-02-08 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16826 @kunalkhamar by the way, please add the JIRA number to the title. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100192972 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -38,16 +38,31 @@ import

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100197297 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -213,6 +218,24 @@ class SparkSession private( new

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100195293 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -213,6 +218,24 @@ class SparkSession private( new

[GitHub] spark issue #16715: [Spark-18080][ML] Python API & Examples for Locality Sen...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16715 **[Test build #72612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72612/testReport)** for PR 16715 at commit

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100190232 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ExperimentalMethods.scala --- @@ -46,4 +48,23 @@ class ExperimentalMethods private[sql]() {

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100195235 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -213,6 +218,24 @@ class SparkSession private( new

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100193350 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -38,16 +38,31 @@ import

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100195109 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -213,6 +218,24 @@ class SparkSession private( new

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100195324 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -213,6 +218,24 @@ class SparkSession private( new

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100194991 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/test/TestHive.scala --- @@ -151,7 +155,7 @@ private[hive] class TestHiveSparkSession(

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100198503 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SparkSessionBuilderSuite.scala --- @@ -123,4 +125,70 @@ class SparkSessionBuilderSuite extends

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100194351 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -65,12 +80,29 @@ private[sql] class SessionState(sparkSession:

[GitHub] spark pull request #16826: Fork SparkSession with option to inherit a copy o...

2017-02-08 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r100194640 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/test/TestHive.scala --- @@ -115,16 +113,22 @@ class TestHiveContext( private[hive] class

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100199037 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,200 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark issue #16859: [SPARK-17714][Core][maven][test-hadoop2.6]Avoid using Ex...

2017-02-08 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16859 cc @vanzin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100198559 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,200 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark issue #16859: [SPARK-17714][Core][maven][test-hadoop2.6]Avoid using Ex...

2017-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16859 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16859: [SPARK-17714][Core][maven][test-hadoop2.6]Avoid using Ex...

2017-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16859 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72599/ Test PASSed. ---

[GitHub] spark issue #16859: [SPARK-17714][Core][maven][test-hadoop2.6]Avoid using Ex...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16859 **[Test build #72599 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72599/testReport)** for PR 16859 at commit

[GitHub] spark issue #15415: [SPARK-14503][ML] spark.ml API for FPGrowth

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15415 **[Test build #72611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72611/testReport)** for PR 15415 at commit

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16858 **[Test build #3567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3567/testReport)** for PR 16858 at commit

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #72610 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72610/testReport)** for PR 15009 at commit

[GitHub] spark issue #15604: [SPARK-18066] [CORE] [TESTS] Add Pool usage policies tes...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/15604 Ok thanks for the update! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15604: [SPARK-18066] [CORE] [TESTS] Add Pool usage policies tes...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/15604 Ok thanks for the update! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #16813: [SPARK-19466][CORE][SCHEDULER] Improve Fair Sched...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16813#discussion_r100193866 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala --- @@ -69,19 +70,31 @@ private[spark] class

[GitHub] spark pull request #16813: [SPARK-19466][CORE][SCHEDULER] Improve Fair Sched...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16813#discussion_r100194712 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala --- @@ -140,14 +157,15 @@ private[spark] class

[GitHub] spark issue #16860: [SPARK-18613][ML] make spark.mllib LDA dependencies in s...

2017-02-08 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16860 Try running dev/mima locally. I bet you'll have to add stuff to the MiMaExcludes.scala file b/c of false positives. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #16813: [SPARK-19466][CORE][SCHEDULER] Improve Fair Sched...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16813#discussion_r100194109 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala --- @@ -69,19 +70,31 @@ private[spark] class

[GitHub] spark issue #15604: [SPARK-18066] [CORE] [TESTS] Add Pool usage policies tes...

2017-02-08 Thread erenavsarogullari
Github user erenavsarogullari commented on the issue: https://github.com/apache/spark/pull/15604 @kayousterhout thanks for query this PR. I will be submitting this and the other one( #15326) once #16813 is merged. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #16860: [SPARK-18613][ML] make spark.mllib LDA dependencies in s...

2017-02-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16860 Oh, the class is private already, right on. That seems OK then. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192059 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,200 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark issue #16715: [Spark-18080][ML] Python API & Examples for Locality Sen...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16715 **[Test build #72609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72609/testReport)** for PR 16715 at commit

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100193058 --- Diff: python/pyspark/ml/feature.py --- @@ -755,6 +951,102 @@ def maxAbs(self): @inherit_doc +class MinHashLSH(JavaEstimator,

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100193020 --- Diff: python/pyspark/ml/feature.py --- @@ -755,6 +951,102 @@ def maxAbs(self): @inherit_doc +class MinHashLSH(JavaEstimator,

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100193043 --- Diff: python/pyspark/ml/feature.py --- @@ -755,6 +951,102 @@ def maxAbs(self): @inherit_doc +class MinHashLSH(JavaEstimator,

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192985 --- Diff: examples/src/main/python/ml/bucketed_random_projection_lsh.py --- @@ -0,0 +1,76 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192933 --- Diff: examples/src/main/python/ml/bucketed_random_projection_lsh.py --- @@ -0,0 +1,76 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192881 --- Diff: examples/src/main/python/ml/min_hash_lsh.py --- @@ -0,0 +1,75 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #13077: [SPARK-10748] [Mesos] Log error instead of crashing Spar...

2017-02-08 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/13077 @srowen I think it is still needed, @tnachen mentioned that it will be closed as 'it's no longer being updated' if the conflicts cannot be resolved. I have updated the PR and there are no

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192685 --- Diff: examples/src/main/python/ml/min_hash_lsh.py --- @@ -0,0 +1,75 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #16813: [SPARK-19466][CORE][SCHEDULER] Improve Fair Scheduler Lo...

2017-02-08 Thread erenavsarogullari
Github user erenavsarogullari commented on the issue: https://github.com/apache/spark/pull/16813 Hi @kayousterhout and @markhamstra, All comments have been addressed via latest patch(790097e) --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192402 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,200 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192347 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,200 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark issue #16860: [SPARK-18613][ML] make spark.mllib LDA dependencies in s...

2017-02-08 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16860 @srowen This actually isn't a breaking change. This is making protected methods private within classes which cannot be extended outside of Spark. The classes are not final, but they only

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192333 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,200 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192298 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,200 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192314 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,200 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192074 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,200 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark pull request #16715: [Spark-18080][ML] Python API & Examples for Local...

2017-02-08 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r100192026 --- Diff: python/pyspark/ml/feature.py --- @@ -120,6 +122,200 @@ def getThreshold(self): return self.getOrDefault(self.threshold)

[GitHub] spark issue #16800: [SPARK-19456][SparkR]:Add LinearSVC R API

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16800 **[Test build #72608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72608/testReport)** for PR 16800 at commit

[GitHub] spark issue #16860: [SPARK-18613][ML] make spark.mllib LDA dependencies in s...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16860 **[Test build #72607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72607/testReport)** for PR 16860 at commit

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100187460 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,55 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100190769 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,55 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100188875 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,55 @@ class TaskSetManagerSuite extends

[GitHub] spark issue #16860: [SPARK-18613][ML] make spark.mllib LDA dependencies in s...

2017-02-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16860 I get it, but making something less visible after it's released is also a breaking change. Without a good reason for that we shouldn't make changes like this. --- If your project is set up for it,

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100186410 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -874,7 +874,8 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100189090 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,55 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100187114 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -874,7 +874,8 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100190631 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,55 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16855: [SPARK-13931] Resolve stage hanging up problem in...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/16855#discussion_r100189015 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -664,6 +665,55 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #16860: [SPARK-18613][ML] make spark.mllib LDA dependenci...

2017-02-08 Thread sueann
GitHub user sueann opened a pull request: https://github.com/apache/spark/pull/16860 [SPARK-18613][ML] make spark.mllib LDA dependencies in spark.ml LDA private ## What changes were proposed in this pull request? spark.ml.*LDAModel classes were exposing spark.mllib LDA models

[GitHub] spark issue #16625: [SPARK-17874][core] Add SSL port configuration.

2017-02-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16625 Unless I hear back I'll push this tomorrow morning. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16826: Fork SparkSession with option to inherit a copy of the S...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16826 **[Test build #72606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72606/testReport)** for PR 16826 at commit

[GitHub] spark issue #13072: [SPARK-15288] [Mesos] Mesos dispatcher should handle gra...

2017-02-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/13072 I can't really evaluate the change; is anyone else familiar with Mesos around to comment? that'd be best. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #13077: [SPARK-10748] [Mesos] Log error instead of crashing Spar...

2017-02-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/13077 Looks like @tnachen suggested closing this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13326: [SPARK-15560] [Mesos] Queued/Supervise drivers waiting f...

2017-02-08 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/13326 @tnachen Can you check this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13077: [SPARK-10748] [Mesos] Log error instead of crashing Spar...

2017-02-08 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/13077 @srowen /@tnachen Can you check this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16826: Fork SparkSession with option to inherit a copy of the S...

2017-02-08 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16826 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #13072: [SPARK-15288] [Mesos] Mesos dispatcher should handle gra...

2017-02-08 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/13072 @srowen Can you check this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13143: [SPARK-15359] [Mesos] Mesos dispatcher should handle DRI...

2017-02-08 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/13143 @tnachen Can you check this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r100186876 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -218,7 +247,14 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #16846: [SPARK-19506][ML][PYTHON] Import warnings in pyspark.ml....

2017-02-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16846 This looks good, normally I'd say we should have tests but since we are going to remove this in 2.2 that is probably ok as is. I'll merge this to master & the 2.1 branch if no one objects by EOD.

[GitHub] spark issue #16855: [SPARK-13931] Resolve stage hanging up problem in a part...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/16855 Can you make the PR and JIRA description something more specific? Maybe "[SPARK-13931] Stage can hang if an executor fails while speculated tasks are running" --- If your project is set up

[GitHub] spark issue #16825: [SPARK-19481][REPL][maven]Avoid to leak SparkContext in ...

2017-02-08 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16825 This PR doesn't fix all leaks though. I noticed that there are many Finalizers and it slows down GC. Most of them are `JarFile` and `Inflater`. --- If your project is set up for it, you can reply

[GitHub] spark issue #16855: [SPARK-13931] Resolve stage hanging up problem in a part...

2017-02-08 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/16855 Jenkins, this is OK to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r100184896 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -218,7 +247,14 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #16736: [SPARK-19265][SQL][Follow-up] Configurable `tableRelatio...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16736 **[Test build #72605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72605/testReport)** for PR 16736 at commit

[GitHub] spark issue #16804: [SPARK-19459][SQL] Add Hive datatype (char/varchar) to S...

2017-02-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16804 LGTM pending test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16736: [SPARK-19265][SQL][Follow-up] Configurable `tableRelatio...

2017-02-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16736 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16664: [SPARK-18120 ][SQL] Call QueryExecutionListener c...

2017-02-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16664#discussion_r100182956 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -218,7 +247,14 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #16804: [SPARK-19459][SQL] Add Hive datatype (char/varchar) to S...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16804 **[Test build #72604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72604/testReport)** for PR 16804 at commit

[GitHub] spark issue #16804: [SPARK-19459][SQL] Add Hive datatype (char/varchar) to S...

2017-02-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16804 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16386: [SPARK-18352][SQL] Support parsing multiline json files

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16386 **[Test build #72603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72603/testReport)** for PR 16386 at commit

[GitHub] spark issue #16837: [SPARK-19359][SQL] renaming partition should not leave u...

2017-02-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16837 LGTM except one comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16386: [SPARK-18352][SQL] Support parsing multiline json files

2017-02-08 Thread NathanHowell
Github user NathanHowell commented on the issue: https://github.com/apache/spark/pull/16386 Rebased again to pickup the build break hotfix in c618ccdbe9ac103dfa3182346e2a14a1e7fca91a --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15666: [SPARK-11421] [Core][Python][R] Added ability for...

2017-02-08 Thread mariusvniekerk
Github user mariusvniekerk commented on a diff in the pull request: https://github.com/apache/spark/pull/15666#discussion_r100177099 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1802,19 +1802,34 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark pull request #15666: [SPARK-11421] [Core][Python][R] Added ability for...

2017-02-08 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15666#discussion_r100176735 --- Diff: python/pyspark/context.py --- @@ -836,6 +836,17 @@ def addPyFile(self, path): import importlib

[GitHub] spark issue #16830: [MINOR][CORE] Fix incorrect documentation of WritableCon...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16830 **[Test build #3569 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3569/testReport)** for PR 16830 at commit

[GitHub] spark pull request #15666: [SPARK-11421] [Core][Python][R] Added ability for...

2017-02-08 Thread mariusvniekerk
Github user mariusvniekerk commented on a diff in the pull request: https://github.com/apache/spark/pull/15666#discussion_r100176188 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1802,19 +1802,34 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark issue #16852: [SPARK-19512][SQL] codegen for compare structs fails

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16852 **[Test build #3568 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3568/testReport)** for PR 16852 at commit

[GitHub] spark pull request #16845: [SPARK-19505][Python] AttributeError on Exception...

2017-02-08 Thread dgingrich
Github user dgingrich commented on a diff in the pull request: https://github.com/apache/spark/pull/16845#discussion_r100175968 --- Diff: python/pyspark/broadcast.py --- @@ -82,7 +82,7 @@ def dump(self, value, f): except pickle.PickleError: raise

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16858 Thank you for review and merging, @srowen ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use...

2017-02-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16858 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16837: [SPARK-19359][SQL] renaming partition should not ...

2017-02-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16837#discussion_r100174684 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -892,21 +892,58 @@ private[spark] class

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16858 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16858 Finally, it passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-08 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/16228 cc @cloud-fan can you please take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16858 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72597/ Test PASSed. ---

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16858 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

<    1   2   3   4   5   6   7   >