[GitHub] spark pull request: [SPARK-11579] [ML] avoid creating new optimize...

2016-01-05 Thread hhbyyh
Github user hhbyyh closed the pull request at: https://github.com/apache/spark/pull/9614 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12573][SPARK-12574][SQL] Move SQL Parse...

2016-01-05 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10583#discussion_r48919450 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -587,6 +586,13 @@ class Analyzer( case

[GitHub] spark pull request: [SPARK-12573][SPARK-12574][SQL] Move SQL Parse...

2016-01-05 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10583#discussion_r48919389 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/parser/ParseUtils.java --- @@ -0,0 +1,163 @@ +/** + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-12573][SPARK-12574][SQL] Move SQL Parse...

2016-01-05 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10583#discussion_r48919495 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -129,23 +129,39 @@ object UnresolvedAttribute {

[GitHub] spark pull request: [SPARK-12659] fix NPE in UnsafeExternalSorter ...

2016-01-05 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10606#discussion_r48919509 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeInMemorySorter.java --- @@ -223,14 +227,9 @@ public void loadNext() {

[GitHub] spark pull request: [SPARK-12591][Streaming]Register OpenHashMapBa...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10609#issuecomment-169191880 **[Test build #48798 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48798/consoleFull)** for PR 10609 at commit

[GitHub] spark pull request: [SPARK-12591][Streaming]Register OpenHashMapBa...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10609#issuecomment-169192001 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12591][Streaming]Register OpenHashMapBa...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10609#issuecomment-169191999 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-7675][ML][PYSpark] sparkml params type ...

2016-01-05 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9581#issuecomment-169173662 It'd be nice to have a test or two which failed before your PR but work now. They don't need to touch all (or any) algorithms; you should hopefully be able to use a

[GitHub] spark pull request: [SPARK-3873] [core] Import ordering fixes.

2016-01-05 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/10578#issuecomment-169174148 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3873] [tests] Import ordering fixes.

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10582#issuecomment-169174742 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3873] [tests] Import ordering fixes.

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10582#issuecomment-169174666 **[Test build #48784 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48784/consoleFull)** for PR 10582 at commit

[GitHub] spark pull request: [SPARK-3873] [tests] Import ordering fixes.

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10582#issuecomment-169174744 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3873] [core] Import ordering fixes.

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10578#issuecomment-169175178 **[Test build #48799 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48799/consoleFull)** for PR 10578 at commit

[GitHub] spark pull request: [SPARK-12591][Streaming]Register OpenHashMapBa...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10609#issuecomment-169175648 **[Test build #48798 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48798/consoleFull)** for PR 10609 at commit

[GitHub] spark pull request: [SPARK-12558][SQL][WIP] AnalysisException when...

2016-01-05 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/10520#issuecomment-169175945 @yhuai Hi Yin, Here is what i had while answering the question from Nong Li. Please let me know if you have more questions that i need to find the answers for.

[GitHub] spark pull request: [SPARK-12450][MLLib] Un-persist broadcasted va...

2016-01-05 Thread rnowling
Github user rnowling commented on the pull request: https://github.com/apache/spark/pull/10415#issuecomment-169176203 Thanks everyone! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3873] [tests] Import ordering fixes.

2016-01-05 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/10582#issuecomment-169176325 Man tests are flaky today... retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12558][SQL][WIP] AnalysisException when...

2016-01-05 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10520#issuecomment-169177927 so, two `HiveFunctionWrapper`s equality check failed because they have different `instance`s? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-4036][MLlib]Add Conditional Random Fiel...

2016-01-05 Thread HuJiayin
Github user HuJiayin closed the pull request at: https://github.com/apache/spark/pull/9794 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12558][SQL][WIP] AnalysisException when...

2016-01-05 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/10520#issuecomment-169178261 @yhuai Yes Yin. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-12637 Print stage info of finished stage...

2016-01-05 Thread navis
Github user navis commented on a diff in the pull request: https://github.com/apache/spark/pull/10585#discussion_r48915612 --- Diff: core/src/main/scala/org/apache/spark/scheduler/StageInfo.scala --- @@ -61,6 +61,21 @@ class StageInfo( "running" } } +

[GitHub] spark pull request: [SPARK-11696] [ML, MLlib] Optimization: Extend...

2016-01-05 Thread NarineK
Github user NarineK commented on the pull request: https://github.com/apache/spark/pull/9667#issuecomment-169178623 Originally, I needed that information for the neuronal network. I extended it so that the ANN can expose it too. I think in general it would be good to have the

[GitHub] spark pull request: [SPARK-12582][Test] IndexShuffleBlockResolverS...

2016-01-05 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10526#issuecomment-169178713 Jenkins, this is ok to test? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-3873] [tests] Import ordering fixes.

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10582#issuecomment-169179009 **[Test build #48800 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48800/consoleFull)** for PR 10582 at commit

[GitHub] spark pull request: SPARK-12637 Print stage info of finished stage...

2016-01-05 Thread navis
Github user navis commented on a diff in the pull request: https://github.com/apache/spark/pull/10585#discussion_r48916174 --- Diff: core/src/main/scala/org/apache/spark/scheduler/StageInfo.scala --- @@ -61,6 +61,21 @@ class StageInfo( "running" } } +

[GitHub] spark pull request: [SPARK-12631] [PYSPARK] [DOC] PySpark clusteri...

2016-01-05 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/10610 [SPARK-12631] [PYSPARK] [DOC] PySpark clustering parameter desc to consistent format Part of task for [SPARK-11219](https://issues.apache.org/jira/browse/SPARK-11219) to make PySpark MLlib

[GitHub] spark pull request: [SPARK-5682][Core] Add encrypted shuffle in sp...

2016-01-05 Thread winningsix
Github user winningsix commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-169181174 In fact, native optimization(using Openssl cipher) with AES-NI is very necessary. Regards the performance, I have some rough data about the performance impacts

[GitHub] spark pull request: [SPARK-12582][Test] IndexShuffleBlockResolverS...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10526#issuecomment-169181081 **[Test build #48801 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48801/consoleFull)** for PR 10526 at commit

[GitHub] spark pull request: [SPARK-12647][SQL] Fix o.a.s.sqlexecution.Exch...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10599#issuecomment-169112038 **[Test build #2327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2327/consoleFull)** for PR 10599 at commit

[GitHub] spark pull request: [SPARK-12539][SQL] support writing bucketed ta...

2016-01-05 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10498#discussion_r48889876 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/bucket.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-529] [core] [yarn] Add type-safe config...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10205#issuecomment-169119691 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-529] [core] [yarn] Add type-safe config...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10205#issuecomment-169119625 **[Test build #48785 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48785/consoleFull)** for PR 10205 at commit

[GitHub] spark pull request: [SPARK-529] [core] [yarn] Add type-safe config...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10205#issuecomment-169119685 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11696] [ML, MLlib] Optimization: Extend...

2016-01-05 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/9667#issuecomment-169126422 @jkbradley Could you suggest why do you think that it does not worth having the output from `mlllib` optimizers? Are planning to deprecate all optimizers from `mllib`

[GitHub] spark pull request: [SPARK-529] [core] [yarn] Add type-safe config...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10205#issuecomment-169130352 **[Test build #48788 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48788/consoleFull)** for PR 10205 at commit

[GitHub] spark pull request: [SPARK-12379][ML][MLLIB] Copy GBT implementati...

2016-01-05 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/10607#discussion_r48908602 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/GradientBoostedTrees.scala --- @@ -0,0 +1,272 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-529] [core] [yarn] Add type-safe config...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10205#issuecomment-169161754 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12295] [SQL] external spilling for wind...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10605#issuecomment-169163892 **[Test build #48790 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48790/consoleFull)** for PR 10605 at commit

[GitHub] spark pull request: [SPARK-12295] [SQL] external spilling for wind...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10605#issuecomment-169164112 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12295] [SQL] external spilling for wind...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10605#issuecomment-169164110 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12453][Streaming] Remove explicit depen...

2016-01-05 Thread BrianLondon
Github user BrianLondon commented on the pull request: https://github.com/apache/spark/pull/10492#issuecomment-169164791 Sounds good to me. Once done, I'll rebase 12269 and reopen that discussion. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-12560][SQL] SqlTestUtils.stripSparkFilt...

2016-01-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/10510#issuecomment-169164600 @squito Ah, sorry I misunderstood. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-12379][ML][MLLIB] Copy GBT implementati...

2016-01-05 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/10607#discussion_r48910260 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/DecisionTreeRegressor.scala --- @@ -87,6 +87,14 @@ final class DecisionTreeRegressor

[GitHub] spark pull request: [SPARK-12453][Streaming] Remove explicit depen...

2016-01-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10492 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12573][SPARK-12574][SQL] Move SQL Parse...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10583#issuecomment-169165991 **[Test build #48789 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48789/consoleFull)** for PR 10583 at commit

[GitHub] spark pull request: [SPARK-12453][Streaming] Remove explicit depen...

2016-01-05 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10492#issuecomment-169166570 Merged to master/1.6/1.5. I'll keep an eye on the 1.5 build to make sure that cherry pick still worked OK. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-6363][BUILD] Make Scala 2.11 the defaul...

2016-01-05 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/10608 [SPARK-6363][BUILD] Make Scala 2.11 the default Scala version This patch changes Spark's build to make Scala 2.11 the default Scala version. To be clear, this does not mean that Spark will stop

[GitHub] spark pull request: [SPARK-12504][SQL] Masking credentials in the ...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10452#issuecomment-169167587 **[Test build #48792 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48792/consoleFull)** for PR 10452 at commit

[GitHub] spark pull request: [SPARK-12504][SQL] Masking credentials in the ...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10452#issuecomment-169167869 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10775][UI]add search keywords in histor...

2016-01-05 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/8885#issuecomment-169167691 so in https://issues.apache.org/jira/browse/SPARK-10873 we are looking at changing to use the jquery datatables. This would give us a bunch of things including a

[GitHub] spark pull request: [SPARK-12659] fix NPE in UnsafeExternalSorter ...

2016-01-05 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10606#discussion_r48911600 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeInMemorySorter.java --- @@ -99,7 +99,11 @@ public UnsafeInMemorySorter(

[GitHub] spark pull request: [SPARK-12581][SQL] Support case-sensitive tabl...

2016-01-05 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10523#issuecomment-169169290 @maropu Thank you for the PR. So, when you quote a table name in postgres, the name is case-sensitive, right? Looks like we are always quoting the table name? If so,

[GitHub] spark pull request: [SPARK-7675][ML][PYSpark] sparkml params type ...

2016-01-05 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9581#issuecomment-169170474 @holdenk Thanks! Just one more item I forgot: could you please add a unit test for this in ml/tests.py? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-12558][SQL][WIP] AnalysisException when...

2016-01-05 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10520#issuecomment-169170856 Can you explain more why the semanticEquals at

[GitHub] spark pull request: [SPARK-3873] [sql] Import ordering fixes.

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10573#issuecomment-169171217 **[Test build #48791 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48791/consoleFull)** for PR 10573 at commit

[GitHub] spark pull request: [SPARK-12580][SQL] Remove string concatenation...

2016-01-05 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/10524#discussion_r48912564 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/MonotonicallyIncreasingID.scala --- @@ -33,12 +33,12 @@ import

[GitHub] spark pull request: [SPARK-12580][SQL] Remove string concatenation...

2016-01-05 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/10524#discussion_r48912607 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -57,10 +57,9 @@ case class Md5(child: Expression) extends

[GitHub] spark pull request: [SPARK-3873] [core] Import ordering fixes.

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10578#issuecomment-169172280 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12630][DOC] Update param descriptions

2016-01-05 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/10598#discussion_r48912976 --- Diff: python/pyspark/mllib/classification.py --- @@ -323,38 +330,45 @@ def train(cls, data, iterations=100, initialWeights=None, regParam=0.01,

[GitHub] spark pull request: [SPARK-3873] [core] Import ordering fixes.

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10578#issuecomment-169172115 **[Test build #48782 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48782/consoleFull)** for PR 10578 at commit

[GitHub] spark pull request: [SPARK-12591][Streaming]Register OpenHashMapBa...

2016-01-05 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/10609 [SPARK-12591][Streaming]Register OpenHashMapBasedStateMap for Kryo The default serializer in Kryo is FieldSerializer and it ignores transient fields and never calls `writeObject` or `readObject`.

[GitHub] spark pull request: [SPARK-12317][SQL]Support configurate value fo...

2016-01-05 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/10314#issuecomment-169173265 Hello @srowen @marmbrus @viirya : I have made the code changes, and change the title based on the comments. Can you help review the codes? Thanks. --- If your

[GitHub] spark pull request: [SPARK-529] [core] [yarn] Add type-safe config...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10205#issuecomment-169161234 **[Test build #48788 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48788/consoleFull)** for PR 10205 at commit

[GitHub] spark pull request: [SPARK-529] [core] [yarn] Add type-safe config...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10205#issuecomment-169161759 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12450][MLLib] Un-persist broadcasted va...

2016-01-05 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/10415#issuecomment-169163055 LGTM too I'll merge this with master, branch-1.6 and branch-1.5 --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-12450][MLLib] Un-persist broadcasted va...

2016-01-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10415 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10086] [PySpark] [MLlib] [Streaming] [T...

2016-01-05 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/9670#issuecomment-169163501 @jkbradley, I haven't had much time to look into the root cause, but I'm still planning to get to it. I'll close this and open another once I have something. ---

[GitHub] spark pull request: [SPARK-10086] [PySpark] [MLlib] [Streaming] [T...

2016-01-05 Thread BryanCutler
Github user BryanCutler closed the pull request at: https://github.com/apache/spark/pull/9670 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-12041] [ML] [PySpark] Add columnSimilar...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10158#issuecomment-169163584 **[Test build #2331 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2331/consoleFull)** for PR 10158 at commit

[GitHub] spark pull request: [SPARK-11945] [ML] [PySpark] Add computeCost t...

2016-01-05 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9931#issuecomment-169164398 I'll review now --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11945] [ML] [PySpark] Add computeCost t...

2016-01-05 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9931#issuecomment-169164506 Just a few comments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11945] [ML] [PySpark] Add computeCost t...

2016-01-05 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/9931#discussion_r48909691 --- Diff: python/pyspark/ml/clustering.py --- @@ -36,6 +36,14 @@ def clusterCenters(self): """Get the cluster centers, represented as a list of

[GitHub] spark pull request: [SPARK-11945] [ML] [PySpark] Add computeCost t...

2016-01-05 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/9931#discussion_r48909689 --- Diff: python/pyspark/ml/clustering.py --- @@ -36,6 +36,14 @@ def clusterCenters(self): """Get the cluster centers, represented as a list of

[GitHub] spark pull request: [SPARK-7675][ML][PYSpark] sparkml params type ...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9581#issuecomment-169165400 **[Test build #48794 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48794/consoleFull)** for PR 9581 at commit

[GitHub] spark pull request: [SPARK-7675][ML][PYSpark] sparkml params type ...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9581#issuecomment-169165624 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-7675][ML][PYSpark] sparkml params type ...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9581#issuecomment-169165625 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12573][SPARK-12574][SQL] Move SQL Parse...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10583#issuecomment-169166339 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12573][SPARK-12574][SQL] Move SQL Parse...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10583#issuecomment-169166337 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-6363][BUILD] Make Scala 2.11 the defaul...

2016-01-05 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10608#issuecomment-169166671 Note: I'm happy to defer merging this pull request for a while. I just happened to have the changes ready locally and figured that it would be nice to test them on

[GitHub] spark pull request: [SPARK-12504][SQL] Masking credentials in the ...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10452#issuecomment-169167867 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11696] [ML, MLlib] Optimization: Extend...

2016-01-05 Thread NarineK
Github user NarineK commented on the pull request: https://github.com/apache/spark/pull/9667#issuecomment-169167949 Thanks for your feedback! @jkbradley : Which package do you exactly mean with DataFrame's spark.ml ? I can see

[GitHub] spark pull request: [SPARK-12041] [ML] [PySpark] Add columnSimilar...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10158#issuecomment-169168054 **[Test build #2331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2331/consoleFull)** for PR 10158 at commit

[GitHub] spark pull request: [SPARK-12603] [MLlib] PySpark MLlib GaussianMi...

2016-01-05 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/10552#issuecomment-169168114 These changes look fine to me, but could you please test predictSoft in Python (also to add this to the example in the doc)? --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-6363][BUILD] Make Scala 2.11 the defaul...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10608#issuecomment-169168673 **[Test build #48796 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48796/consoleFull)** for PR 10608 at commit

[GitHub] spark pull request: [SPARK-12041] [ML] [PySpark] Add columnSimilar...

2016-01-05 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/10158#issuecomment-169168681 Merging with master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12041] [ML] [PySpark] Add columnSimilar...

2016-01-05 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/10158#issuecomment-169168695 Thanks for the PR! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12659] fix NPE in UnsafeExternalSorter ...

2016-01-05 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10606#discussion_r48911664 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeInMemorySorter.java --- @@ -223,14 +227,9 @@ public void loadNext() {

[GitHub] spark pull request: [SPARK-12041] [ML] [PySpark] Add columnSimilar...

2016-01-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10158 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11696] [ML, MLlib] Optimization: Extend...

2016-01-05 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/9667#issuecomment-169169854 Maybe I misunderstood the need here. Do you want to use the optimizers for your own custom algorithms? Or do you want to get info back from MLlib algorithms like

[GitHub] spark pull request: [SPARK-12573][SPARK-12574][SQL] Move SQL Parse...

2016-01-05 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10583#issuecomment-169169980 Can you update the pull request description? It still says WIP. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-12573][SPARK-12574][SQL] Move SQL Parse...

2016-01-05 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10583#issuecomment-169171058 Done. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-3873] [tests] Import ordering fixes.

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10582#issuecomment-169171464 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3873] [sql] Import ordering fixes.

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10573#issuecomment-169171534 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3873] [sql] Import ordering fixes.

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10573#issuecomment-169171537 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12379][ML][MLLIB] Copy GBT implementati...

2016-01-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10607#issuecomment-169171629 **[Test build #48795 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48795/consoleFull)** for PR 10607 at commit

[GitHub] spark pull request: [SPARK-3873] [tests] Import ordering fixes.

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10582#issuecomment-169171466 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3873] [sql] Import ordering fixes.

2016-01-05 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/10573#issuecomment-169171431 If there's no concern I'll merge this soon since I don't want to track conflicts forever. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-12379][ML][MLLIB] Copy GBT implementati...

2016-01-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10607#issuecomment-169171783 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-7675][ML][PYSpark] sparkml params type ...

2016-01-05 Thread holdenk
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/9581#issuecomment-169171921 Sure - I figured it was already pretty well covered by the existing tests which use the params but I'll add some explicit unit tests for this. --- If your project is

<    3   4   5   6   7   8   9   >