[GitHub] spark issue #13130: [SPARK-15340][SQL]Limit the size of the map used to cach...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13130 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13130: [SPARK-15340][SQL]Limit the size of the map used to cach...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13130 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61142/ Test FAILed. ---

[GitHub] spark issue #13130: [SPARK-15340][SQL]Limit the size of the map used to cach...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13130 **[Test build #61142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61142/consoleFull)** for PR 13130 at commit [`82d78a3`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13778: [SPARK-16062][SPARK-15989][SQL] Fix two bugs of Python-o...

2016-06-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13778 ping @vlad17 @davies @liancheng Any thing else? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark issue #13130: [SPARK-15340][SQL]Limit the size of the map used to cach...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13130 **[Test build #61142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61142/consoleFull)** for PR 13130 at commit [`82d78a3`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #13130: [SPARK-15340][SQL]Limit the size of the map used to cach...

2016-06-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13130 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #13786: [SPARK-15294][R] Add `pivot` to SparkR

2016-06-23 Thread Div333
Github user Div333 commented on the issue: https://github.com/apache/spark/pull/13786 Hello everyone, Thanks a lot for implementing the pivot functionality. I have started using SparkR recently and would like to know if the pivot method is included in the library as I don't find it i

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-06-23 Thread JustinPihony
Github user JustinPihony commented on the issue: https://github.com/apache/spark/pull/12601 Bump @HyukjinKwon I have some comments to your comments. Could you please review them and I can push my changes. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #13882: Branch 1.6

2016-06-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/13882 Please close this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #13720: [SPARK-16004] [SQL] improve the display of Catalo...

2016-06-23 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/13720#discussion_r68343766 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -522,7 +523,7 @@ case class DescribeTableCommand(table: TableIdent

[GitHub] spark issue #13882: Branch 1.6

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13882 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #13865: [SPARK-13709][SQL] Initialize deserializer with b...

2016-06-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13865#discussion_r68343609 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala --- @@ -65,4 +68,77 @@ class QueryPartitionSuite extends QueryTest wi

[GitHub] spark pull request #13882: Branch 1.6

2016-06-23 Thread liu549676915
GitHub user liu549676915 opened a pull request: https://github.com/apache/spark/pull/13882 Branch 1.6 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how th

[GitHub] spark issue #13720: [SPARK-16004] [SQL] improve the display of CatalogTable ...

2016-06-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13720 For the test, currently we only have one `desc table` test in `HiveDDLSuite`, It will be good if we can have an individual test suite for it. --- If your project is set up for it, you can reply t

[GitHub] spark pull request #13720: [SPARK-16004] [SQL] improve the display of Catalo...

2016-06-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13720#discussion_r68342801 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -522,7 +523,7 @@ case class DescribeTableCommand(table: TableI

[GitHub] spark issue #13874: [SQL][minor] ParserUtils.operationNotAllowed should thro...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13874 **[Test build #61141 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61141/consoleFull)** for PR 13874 at commit [`ec0506f`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #13880: SPARK-16178: Remove unnecessary Hive partition check.

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13880 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61139/ Test PASSed. ---

[GitHub] spark issue #13880: SPARK-16178: Remove unnecessary Hive partition check.

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13880 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13880: SPARK-16178: Remove unnecessary Hive partition check.

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13880 **[Test build #61139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61139/consoleFull)** for PR 13880 at commit [`919f520`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce additonal implementation wi...

2016-06-23 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13680 @kiszk we should definitely put zero into the corresponding field when set null. It will be a little harder than `UnsafeRow`, as we need `setNullBoolean`, `setNullInt`, etc. but it's still doable.

[GitHub] spark pull request #13832: [SPARK-16123] Avoid NegativeArraySizeException wh...

2016-06-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13832 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #13832: [SPARK-16123] Avoid NegativeArraySizeException while res...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13832 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61138/ Test PASSed. ---

[GitHub] spark issue #13832: [SPARK-16123] Avoid NegativeArraySizeException while res...

2016-06-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13832 Merging to master/2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark issue #13832: [SPARK-16123] Avoid NegativeArraySizeException while res...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13832 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13832: [SPARK-16123] Avoid NegativeArraySizeException while res...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13832 **[Test build #61138 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61138/consoleFull)** for PR 13832 at commit [`e2a1e1e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61137/ Test PASSed. ---

[GitHub] spark issue #13860: [SPARK-16157] [SQL] Add New Methods for comments in Stru...

2016-06-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13860 @hvanhovell Sure, will do it! It sounds like you also like the suggestion by @cloud-fan . Let me do it too. Thanks! --- If your project is set up for it, you can reply to this email and have you

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61137 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61137/consoleFull)** for PR 13876 at commit [`5a9f4ec`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #13837: [SPARK-16126] [SQL] Better Error Message When usi...

2016-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13837#discussion_r68341391 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetOptions.scala --- @@ -40,7 +40,7 @@ private[sql] class Parque

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61135/ Test PASSed. ---

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61135/consoleFull)** for PR 13876 at commit [`6180daf`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61136/ Test PASSed. ---

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61136 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61136/consoleFull)** for PR 13876 at commit [`e0239a0`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13699: [SPARK-15958] Make initial buffer size for the Sorter co...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13699 **[Test build #61140 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61140/consoleFull)** for PR 13699 at commit [`cf464a3`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #13699: [SPARK-15958] Make initial buffer size for the So...

2016-06-23 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/13699#discussion_r68340566 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java --- @@ -122,6 +123,8 @@ public UnsafeShuffleWriter( this.task

[GitHub] spark issue #13881: [SPARK-3723] [MLlib] Adding instrumentation to random fo...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13881 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #13881: [SPARK-3723] [MLlib] Adding instrumentation to ra...

2016-06-23 Thread smurching
GitHub user smurching opened a pull request: https://github.com/apache/spark/pull/13881 [SPARK-3723] [MLlib] Adding instrumentation to random forests ## What changes were proposed in this pull request? In RandomForest.run(), added instrumentation for the number of node group

[GitHub] spark pull request #13877: [SPARK-16142] [R] group naiveBayes method docs in...

2016-06-23 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/13877#discussion_r68337983 --- Diff: R/pkg/R/mllib.R --- @@ -390,23 +376,41 @@ setMethod("predict", signature(object = "KMeansModel"), return(dataFrame(callJMethod(obje

[GitHub] spark issue #13794: [SPARK-15574][WIP][ML][PySpark] Python transformer wrapp...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13794 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61133/ Test FAILed. ---

[GitHub] spark issue #13794: [SPARK-15574][WIP][ML][PySpark] Python transformer wrapp...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13794 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13794: [SPARK-15574][WIP][ML][PySpark] Python transformer wrapp...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13794 **[Test build #61133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61133/consoleFull)** for PR 13794 at commit [`b8ddcdb`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13880: SPARK-16178: Remove unnecessary Hive partition check.

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13880 **[Test build #61139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61139/consoleFull)** for PR 13880 at commit [`919f520`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #13880: SPARK-16178: Remove unnecessary Hive partition ch...

2016-06-23 Thread rdblue
GitHub user rdblue opened a pull request: https://github.com/apache/spark/pull/13880 SPARK-16178: Remove unnecessary Hive partition check. ## What changes were proposed in this pull request? This removes a check that partition names match from the Hive write path, which was

[GitHub] spark issue #13879: [SPARK-16177] [ML] model loading backward compatibility ...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13879 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61134/ Test PASSed. ---

[GitHub] spark issue #13879: [SPARK-16177] [ML] model loading backward compatibility ...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13879 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13879: [SPARK-16177] [ML] model loading backward compatibility ...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13879 **[Test build #61134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61134/consoleFull)** for PR 13879 at commit [`63d280f`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #13865: [SPARK-13709][SQL] Initialize deserializer with b...

2016-06-23 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13865#discussion_r68335825 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala --- @@ -230,10 +234,21 @@ class HadoopTableReader( // Fill all par

[GitHub] spark issue #13865: [SPARK-13709][SQL] Initialize deserializer with both tab...

2016-06-23 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13865 This is ready for review. cc @yhuai @cloud-fan @clockfly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark issue #13860: [SPARK-16157] [SQL] Add New Methods for comments in Stru...

2016-06-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13860 @gatorsmile could you also remove the current uses of `new MetadataBuilder().putString("comment", "test").build()` (and friends) from the code base? --- If your project is set up for it, you ca

[GitHub] spark issue #13832: [SPARK-16123] Avoid NegativeArraySizeException while res...

2016-06-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13832 LGTM - pending jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark issue #13832: [SPARK-16123] Avoid NegativeArraySizeException while res...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13832 **[Test build #61138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61138/consoleFull)** for PR 13832 at commit [`e2a1e1e`](https://github.com/apache/spark/commit/e

[GitHub] spark pull request #13832: [SPARK-16123] Avoid NegativeArraySizeException wh...

2016-06-23 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/13832#discussion_r68334002 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java --- @@ -847,6 +862,11 @@ public final int appendStruct(bo

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61137/consoleFull)** for PR 13876 at commit [`5a9f4ec`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #13870: [SPARK-16165][SQL] Fix the update logic for InMemoryTabl...

2016-06-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13870 Thank you for merging, @liancheng and @davies . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark issue #13870: [SPARK-16165][SQL] Fix the update logic for InMemoryTabl...

2016-06-23 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13870 LGTM, merging to master and branch-2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request #13870: [SPARK-16165][SQL] Fix the update logic for InMem...

2016-06-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13870 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61136/consoleFull)** for PR 13876 at commit [`e0239a0`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStreams to ...

2016-06-23 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/11863 *1+2. Architecture* I had to double check the code to understand the use of sorted list of executors to get consistent topic-to-preferred locations. PLEASE WRITE MORE SCALA DOCS on how the logic in

[GitHub] spark issue #13876: [SPARK-16174][SQL] Improve OptimizeIn optimizer to remov...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61135/consoleFull)** for PR 13876 at commit [`6180daf`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #13879: [SPARK-16177] [ML] model loading backward compatibility ...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13879 **[Test build #61134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61134/consoleFull)** for PR 13879 at commit [`63d280f`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #13879: [SPARK-16177] [ML] model loading backward compati...

2016-06-23 Thread hhbyyh
GitHub user hhbyyh opened a pull request: https://github.com/apache/spark/pull/13879 [SPARK-16177] [ML] model loading backward compatibility for ml.regression ## What changes were proposed in this pull request? jira: https://issues.apache.org/jira/browse/SPARK-16177 model loa

[GitHub] spark issue #13720: [SPARK-16004] [SQL] improve the display of CatalogTable ...

2016-06-23 Thread bomeng
Github user bomeng commented on the issue: https://github.com/apache/spark/pull/13720 @cloud-fan Is this one worth to be fixed? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark pull request #13815: [SPARK-15443][SQL]Fix 'explain' for streaming Dat...

2016-06-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13815 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #13815: [SPARK-15443][SQL]Fix 'explain' for streaming Dataset

2016-06-23 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/13815 LGTM. Merging to master and branch 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request #13876: [SPARK-16174][SQL] Add RemoveLiteralRepetitionFro...

2016-06-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13876#discussion_r68329169 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -793,6 +794,20 @@ object ConstantFolding extends

[GitHub] spark issue #13878: [SPARK-16175] [PYSPARK] handle None for UDT

2016-06-23 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/13878 cc @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #13794: [SPARK-15574][WIP][ML][PySpark] Python transformer wrapp...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13794 **[Test build #61133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61133/consoleFull)** for PR 13794 at commit [`b8ddcdb`](https://github.com/apache/spark/commit/b

[GitHub] spark pull request #13876: [SPARK-16174][SQL] Add RemoveLiteralRepetitionFro...

2016-06-23 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13876#discussion_r68328505 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -793,6 +794,20 @@ object ConstantFolding extends Rule[Logi

[GitHub] spark issue #13794: [SPARK-15574][WIP][ML][PySpark] Python transformer wrapp...

2016-06-23 Thread yinxusen
Github user yinxusen commented on the issue: https://github.com/apache/spark/pull/13794 test it please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark pull request #13876: [SPARK-16174][SQL] Add RemoveLiteralRepetitionFro...

2016-06-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13876#discussion_r68328083 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -793,6 +794,20 @@ object ConstantFolding extends

[GitHub] spark issue #13878: [SPARK-16175] [PYSPARK] handle None for UDT

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13878 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61132/ Test PASSed. ---

[GitHub] spark issue #13878: [SPARK-16175] [PYSPARK] handle None for UDT

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13878 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13878: [SPARK-16175] [PYSPARK] handle None for UDT

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13878 **[Test build #61132 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61132/consoleFull)** for PR 13878 at commit [`8d4b50b`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #13876: [SPARK-16174][SQL] Add RemoveLiteralRepetitionFro...

2016-06-23 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13876#discussion_r68327452 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -793,6 +794,20 @@ object ConstantFolding extends Rule[Logi

[GitHub] spark pull request #13876: [SPARK-16174][SQL] Add RemoveLiteralRepetitionFro...

2016-06-23 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13876#discussion_r68327432 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -793,6 +794,20 @@ object ConstantFolding extends Rule[Logi

[GitHub] spark issue #13872: [SPARK-16164][SQL] Update `CombineFilters` to try to con...

2016-06-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13872 Thank you, @mengxr , @liancheng , and @rxin . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #13872: [SPARK-16164][SQL] Update `CombineFilters` to try...

2016-06-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13872 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #13872: [SPARK-16164][SQL] Update `CombineFilters` to try to con...

2016-06-23 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13872 Merged into master and branch-2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13876: [SPARK-16174][SQL] Add RemoveLiteralRepetitionFromIn opt...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13876: [SPARK-16174][SQL] Add RemoveLiteralRepetitionFromIn opt...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13876 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61126/ Test PASSed. ---

[GitHub] spark issue #13876: [SPARK-16174][SQL] Add RemoveLiteralRepetitionFromIn opt...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13876 **[Test build #61126 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61126/consoleFull)** for PR 13876 at commit [`e33b7f4`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13872: [SPARK-16164][SQL] Update `CombineFilters` to try to con...

2016-06-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13872 Thank you, @rxin . I hope so, too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark issue #13872: [SPARK-16164][SQL] Update `CombineFilters` to try to con...

2016-06-23 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13872 I think it'd be difficult to guarantee ordering all the time, but since the change is small we might as well take this tiny step to ensure stronger compatibility. --- If your project is set up for i

[GitHub] spark issue #13872: [SPARK-16164][SQL] Update `CombineFilters` to try to con...

2016-06-23 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13872 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #13788: [SPARK-16077] [PYSPARK] catch the exception from ...

2016-06-23 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13788#discussion_r68324868 --- Diff: python/pyspark/cloudpickle.py --- @@ -169,7 +169,10 @@ def save_function(self, obj, name=None): if name is None:

[GitHub] spark issue #13788: [SPARK-16077] [PYSPARK] catch the exception from pickle....

2016-06-23 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/13788 Do you mind leaving a comment in the code referencing this issue / maybe linking to that `six` issue so that it's easier for a reader to figure out what's going on? Otherwise, this looks good to m

[GitHub] spark pull request #13788: [SPARK-16077] [PYSPARK] catch the exception from ...

2016-06-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13788#discussion_r68324611 --- Diff: python/pyspark/cloudpickle.py --- @@ -326,7 +329,10 @@ def save_global(self, obj, name=None, pack=struct.pack): modname = geta

[GitHub] spark pull request #13837: [SPARK-16126] [SQL] Better Error Message When usi...

2016-06-23 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/13837#discussion_r68324497 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetOptions.scala --- @@ -40,7 +40,7 @@ private[sql] class ParquetOptio

[GitHub] spark pull request #13788: [SPARK-16077] [PYSPARK] catch the exception from ...

2016-06-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13788#discussion_r68324369 --- Diff: python/pyspark/cloudpickle.py --- @@ -169,7 +169,10 @@ def save_function(self, obj, name=None): if name is None:

[GitHub] spark issue #13656: [SPARK-15938]Adding "support" property to MLlib Associat...

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13656 **[Test build #61131 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61131/consoleFull)** for PR 13656 at commit [`ed384c7`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13656: [SPARK-15938]Adding "support" property to MLlib Associat...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13656 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61131/ Test FAILed. ---

[GitHub] spark pull request #13877: [SPARK-16142] [R] group naiveBayes method docs in...

2016-06-23 Thread junyangq
Github user junyangq commented on a diff in the pull request: https://github.com/apache/spark/pull/13877#discussion_r68324191 --- Diff: R/pkg/R/mllib.R --- @@ -390,23 +376,41 @@ setMethod("predict", signature(object = "KMeansModel"), return(dataFrame(callJMethod(ob

[GitHub] spark issue #13878: [SPARK-16175] [PYSPARK] handle None for UDT

2016-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13878 **[Test build #61132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61132/consoleFull)** for PR 13878 at commit [`8d4b50b`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #13656: [SPARK-15938]Adding "support" property to MLlib Associat...

2016-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13656 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13788: [SPARK-16077] [PYSPARK] catch the exception from pickle....

2016-06-23 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/13788 cc @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #13878: [SPARK-16175] [PYSPARK] handle None for UDT

2016-06-23 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/13878 [SPARK-16175] [PYSPARK] handle None for UDT ## What changes were proposed in this pull request? Scala UDT will bypass all the null and will not pass them into serialize() and deserialize()

[GitHub] spark pull request #13866: [SPARK-16160] [STREAMING] Clear last remembered m...

2016-06-23 Thread ahmed-mahran
Github user ahmed-mahran commented on a diff in the pull request: https://github.com/apache/spark/pull/13866#discussion_r68323315 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala --- @@ -327,6 +329,50 @@ class StreamingContextSuite extends

[GitHub] spark pull request #13866: [SPARK-16160] [STREAMING] Clear last remembered m...

2016-06-23 Thread ahmed-mahran
Github user ahmed-mahran commented on a diff in the pull request: https://github.com/apache/spark/pull/13866#discussion_r68323120 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala --- @@ -327,6 +329,50 @@ class StreamingContextSuite extends

<    1   2   3   4   5   >