[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10440 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12490] Don't use Javascript for web UI'...

2015-12-28 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10441#discussion_r48511839 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -1232,24 +1243,16 @@ private[ui] class TaskPagedTable( override d

[GitHub] spark pull request: [SPARK-12490] Don't use Javascript for web UI'...

2015-12-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10441#issuecomment-167675703 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12490] Don't use Javascript for web UI'...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10441#issuecomment-167675690 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12490] Don't use Javascript for web UI'...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10441#issuecomment-167675687 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-28 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/10440#issuecomment-167675496 Thanks. Merging to master and 1.6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-28 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10440#issuecomment-167675193 sql changes look good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10479 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10368#issuecomment-167673871 As @zsxwing pointed out, the existing unit tests are not sufficient because they didn't catch the fact that `ShuffleReadMetrics` et al must also be registered. This

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167673766 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167673765 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167673716 **[Test build #48376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48376/consoleFull)** for PR 10479 at commit [`422ef49`](https://g

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10368#discussion_r48511217 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -109,6 +111,9 @@ class KryoSerializer(conf: SparkConf) kryo

[GitHub] spark pull request: [SPARK-12490] Don't use Javascript for web UI'...

2015-12-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10441#issuecomment-167672868 @zsxwing, I've pushed a new commit which aims to preserve the old behavior when increasing the number of items displayed per page while pageNumber > 1; see fd2d1f2a49

[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10440#issuecomment-167672081 Maybe @yhuai or @liancheng should take a quick look at the SQL changes. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10440#issuecomment-167671951 Looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10440#discussion_r48510554 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -386,9 +386,9 @@ private[tree] object LearningNode { var levelsToGo = in

[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10440#discussion_r48510515 --- Diff: launcher/src/main/java/org/apache/spark/launcher/Main.java --- @@ -151,7 +151,7 @@ private static String prepareWindowsCommand(List cmd, Map

[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167670013 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167670012 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167669938 **[Test build #48373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48373/consoleFull)** for PR 10499 at commit [`7884e87`](https://g

[GitHub] spark pull request: [SPARKR] [SPARK-11199] Improve R context manag...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9185#issuecomment-167668475 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARKR] [SPARK-11199] Improve R context manag...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9185#issuecomment-167668477 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167668461 **[Test build #48376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48376/consoleFull)** for PR 10479 at commit [`422ef49`](https://gi

[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-28 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/10440#issuecomment-167668145 @andrewor14 could you take a look at this pr? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()

2015-12-28 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10499#discussion_r48508973 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -119,7 +119,7 @@ final class DataFrameWriter private[sql](df: DataFrame

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-28 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/10368#discussion_r48508806 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -109,6 +111,9 @@ class KryoSerializer(conf: SparkConf) kryo.re

[GitHub] spark pull request: [SPARKR] [SPARK-11199] Improve R context manag...

2015-12-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9185#issuecomment-167665605 This seems fine to me as a first step. Eventually we will probably want to make the RBackend multi-session aware. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167665578 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12222] [Core] Deserialize RoaringBitmap...

2015-12-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10253 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12222] [Core] Deserialize RoaringBitmap...

2015-12-28 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/10253#issuecomment-167665206 LGTM. Merging this into `master` and `branch-1.6`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[GitHub] spark pull request: [SPARKR] [SPARK-11199] Improve R context manag...

2015-12-28 Thread falaki
Github user falaki commented on the pull request: https://github.com/apache/spark/pull/9185#issuecomment-167664825 ping @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-12486] Worker should kill the executors...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10438#issuecomment-167664384 **[Test build #48374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48374/consoleFull)** for PR 10438 at commit [`67611ac`](https://gi

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10498#issuecomment-167664157 BTW in github you can use square brackets to create a checklist, e.g. ``` - [] item a - [] item b ``` becomes - [] item a - [] ite

[GitHub] spark pull request: [SPARK-12522] [SQL] [MINOR] Add the missing do...

2015-12-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10471#discussion_r48507936 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/commands.scala --- @@ -102,6 +102,7 @@ case class SetCommand(kv: Option[(String, Option[String

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-28 Thread tedyu
Github user tedyu commented on a diff in the pull request: https://github.com/apache/spark/pull/10368#discussion_r48507734 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -109,6 +111,9 @@ class KryoSerializer(conf: SparkConf) kryo.regi

[GitHub] spark pull request: [SPARK-12415] Do not use closure serializer to...

2015-12-28 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/10368#discussion_r48507496 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -109,6 +111,9 @@ class KryoSerializer(conf: SparkConf) kryo.re

[GitHub] spark pull request: [SPARK-12513] [Streaming] SocketReceiver hang ...

2015-12-28 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/10464#issuecomment-167661660 Looks a race condition in `restart` and `finally { ... socket.stop() ...}`. `restart` will start a new thread and call `receiver.onStart`. So `receiver.onStart` may run

[GitHub] spark pull request: [SPARK-7995][SPARK-6280][Core]Remove AkkaRpcEn...

2015-12-28 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/10459#discussion_r48505634 --- Diff: core/src/test/scala/org/apache/spark/util/AkkaUtilsSuite.scala --- @@ -61,9 +55,14 @@ class AkkaUtilsSuite extends SparkFunSuite with LocalSparkCon

[GitHub] spark pull request: [SPARK-7995][SPARK-6280][Core]Remove AkkaRpcEn...

2015-12-28 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/10459#discussion_r48505487 --- Diff: core/src/test/scala/org/apache/spark/util/AkkaUtilsSuite.scala --- @@ -61,9 +55,14 @@ class AkkaUtilsSuite extends SparkFunSuite with LocalSparkCon

[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-28 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/10440#issuecomment-167656781 ML changes look good to me. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10499#issuecomment-167655997 **[Test build #48373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48373/consoleFull)** for PR 10499 at commit [`7884e87`](https://gi

[GitHub] spark pull request: [SPARK-12316] Wait a minutes to avoid cycle ca...

2015-12-28 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/10475#issuecomment-167655015 CC: @harishreedharan @SaintBacchus could you add test cases for this change? --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-7995][SPARK-6280][Core]Remove AkkaRpcEn...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10459#issuecomment-167654637 **[Test build #48372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48372/consoleFull)** for PR 10459 at commit [`1f5a523`](https://gi

[GitHub] spark pull request: [SPARK-12340][SQL]fix Int overflow in the Spar...

2015-12-28 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/10487#issuecomment-167653173 @QiangCai LGTM but could you add test cases? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10393 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [DOC] Adjust coverage for partitionBy()

2015-12-28 Thread ted-yu
GitHub user ted-yu opened a pull request: https://github.com/apache/spark/pull/10499 [DOC] Adjust coverage for partitionBy() This is the related thread: http://search-hadoop.com/m/q3RTtO3ReeJ1iF02&subj=Re+partitioning+json+data+in+spark Michael suggested fixing the doc.

[GitHub] spark pull request: [SPARK-12441] [SQL] Fixing missingInput in Gen...

2015-12-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/10393#issuecomment-167651518 Thanks, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-7727] [SQL] Avoid inner classes in Rule...

2015-12-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10174 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-7727] [SQL] Avoid inner classes in Rule...

2015-12-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/10174#issuecomment-167650817 Thanks, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-12539][SQL][WIP] support writing bucket...

2015-12-28 Thread nongli
Github user nongli commented on the pull request: https://github.com/apache/spark/pull/10498#issuecomment-167649759 @cloud-fan > currently we don't shuffle before writing partitioned data, which means we will have same partition data in different RDD blocks, and that's why we hav

[GitHub] spark pull request: [SPARK-12526][SPARKR]`ifelse`, `when`, `otherw...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10481#issuecomment-167649006 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12526][SPARKR]`ifelse`, `when`, `otherw...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10481#issuecomment-167649007 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9608#issuecomment-167648846 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9608#issuecomment-167648836 **[Test build #48370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48370/consoleFull)** for PR 9608 at commit [`946ec59`](https://git

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9608#issuecomment-167648843 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6624][WIP] Draft of another alternative...

2015-12-28 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10444#discussion_r48503899 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -47,6 +48,34 @@ trait Predicate extends Expression {

[GitHub] spark pull request: [SPARK-12424][ML] The implementation of ParamM...

2015-12-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10381 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...

2015-12-28 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/10488#discussion_r48503757 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -57,9 +57,10 @@ case class Md5(child: Expression) extends U

[GitHub] spark pull request: [SPARK-12424][ML] The implementation of ParamM...

2015-12-28 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/10381#issuecomment-167648268 Thanks @srowen , @BenFradet and @yanboliang for the review. I'm merging this into `master` and `branch-1.6`. --- If your project is set up for it, you can reply to th

[GitHub] spark pull request: [SPARK-12438][SQL] Add SQLUserDefinedType supp...

2015-12-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/10390#issuecomment-167648261 Okay, I'm fine with merging this now to unblock other progress. However, I think we should move the test to be with the other encoder tests. --- If your project is s

[GitHub] spark pull request: [SPARK-12349] [ML] Make spark.ml PCAModel load...

2015-12-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/10327#discussion_r48503471 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/PCA.scala --- @@ -167,14 +167,37 @@ object PCAModel extends MLReadable[PCAModel] { pr

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10398 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-6624][SQL]Add CNF Normalization as part...

2015-12-28 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/8200#discussion_r48503308 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/BooleanSimplificationSuite.scala --- @@ -116,13 +110,4 @@ class BooleanSimplific

[GitHub] spark pull request: [SPARK-12287] [SQL] Support UnsafeRow in MapPa...

2015-12-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/10398#issuecomment-167645851 Thanks, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

2015-12-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9608#issuecomment-167645599 **[Test build #48370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48370/consoleFull)** for PR 9608 at commit [`946ec59`](https://gith

[GitHub] spark pull request: [SPARK-12517] add default RDD name for one cre...

2015-12-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10456 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12517] add default RDD name for one cre...

2015-12-28 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/10456#issuecomment-167645225 O.K, I'll merge this into `master` and `branch-1.6`. Thanks @wyaron ! --- If your project is set up for it, you can reply to this email and have your reply appear on Gi

[GitHub] spark pull request: [SPARK-12349] [ML] Make spark.ml PCAModel load...

2015-12-28 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/10327#issuecomment-167645124 My only comment is that, in general, I'd like us to add unit tests for backwards compatibility (as was done for spark.mllib NaiveBayesModel). But I'm OK with skippin

[GitHub] spark pull request: [SPARK-12349] [ML] Make spark.ml PCAModel load...

2015-12-28 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/10327#discussion_r48502942 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/PCA.scala --- @@ -167,14 +167,37 @@ object PCAModel extends MLReadable[PCAModel] {

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167644604 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167644603 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-11354] [Web UI] Expose custom log4j fil...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9321#issuecomment-167644002 > On the other hand, I cannot see serious issues by allowing user to potentially read any files under workerDir, it will display the bytecode if reading a jar file bu

[GitHub] spark pull request: [SPARK-6624][SQL]Add CNF Normalization as part...

2015-12-28 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/8200#discussion_r48502484 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -464,6 +465,24 @@ object OptimizeIn extends Rule[LogicalP

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167643902 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167643900 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9608#issuecomment-167643628 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-11638] [Mesos + Docker Bridge networkin...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9608#issuecomment-167643636 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2827][GraphX] Add collectDegreeDist to ...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4399#issuecomment-167643533 Weird, something is wrong with this PR. Maybe re-open it and see if the problem goes away? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10435#discussion_r48502287 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -176,3 +178,221 @@ case class Crc32(child: Expression) extend

[GitHub] spark pull request: [SPARK-12517] add default RDD name for one cre...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10456#issuecomment-167643456 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-5095][MESOS] Support capping cores and ...

2015-12-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4027#issuecomment-167643240 > It's much easier (for me) to think about the number of cores and amount of memory per NODE You did not understand my suggestion. That's exactly what I'm sug

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10388 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12231][SQL]create a combineFilters' pro...

2015-12-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/10388#issuecomment-167642446 Thanks, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167641066 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167640841 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request: [SPARK-12495][SQL] use true as default value f...

2015-12-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10443#discussion_r48501675 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects.scala --- @@ -241,7 +241,7 @@ case class NewInstance(

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48501690 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala --- @@ -448,7 +448,21 @@ case class Pivot(

[GitHub] spark pull request: [SPARK-12460][SQL]Add ExpressionDescription to...

2015-12-28 Thread wilson888888888
Github user wilson8 commented on the pull request: https://github.com/apache/spark/pull/10486#issuecomment-167640827 @vectorijk Thanks. I changed the commit message. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10391#discussion_r48501635 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects.scala --- @@ -439,6 +439,13 @@ case class MapObjects( s"

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10479#issuecomment-167639232 I'm only planning to merge this into 2.0.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-12503] [SQL] Pushing Limit Through Unio...

2015-12-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10451#discussion_r48501449 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala --- @@ -448,7 +448,21 @@ case class Pivot( }

[GitHub] spark pull request: [SPARK-12536] [SQL] Added "Empty Seq" in Expla...

2015-12-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10494#discussion_r48501359 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LocalRelation.scala --- @@ -62,6 +62,10 @@ case class LocalRelation(out

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10479#discussion_r48501316 --- Diff: extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala --- @@ -30,7 +30,7 @@ import org.apache.spark.st

[GitHub] spark pull request: [HOT-FIX] bypass hive test when parse logical ...

2015-12-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10430 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10479#discussion_r48501332 --- Diff: extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala --- @@ -70,12 +70,12 @@ class KinesisBackedBloc

[GitHub] spark pull request: [SPARK-12525] Fix fatal compiler warnings in K...

2015-12-28 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10479#discussion_r48501272 --- Diff: extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala --- @@ -70,12 +70,12 @@ class KinesisBackedBloc

[GitHub] spark pull request: [HOT-FIX] bypass hive test when parse logical ...

2015-12-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/10430#issuecomment-167638369 Thanks, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/10435#discussion_r48500881 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -176,3 +178,221 @@ case class Crc32(child: Expression) ex

[GitHub] spark pull request: [SPARK-12530][Build] Fix build break at Spark-...

2015-12-28 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/10488#discussion_r48500799 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -57,9 +57,10 @@ case class Md5(child: Expression) extends U

<    1   2   3   4   5   >