[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15044 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68065/ Test PASSed. ---

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15044 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15044 **[Test build #68065 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68065/consoleFull)** for PR 15044 at commit

[GitHub] spark issue #15750: [SPARK-18244][SQL] Rename partitionProviderIsHive -> tra...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15750 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15750: [SPARK-18244][SQL] Rename partitionProviderIsHive -> tra...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15750 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68061/ Test PASSed. ---

[GitHub] spark issue #15750: [SPARK-18244][SQL] Rename partitionProviderIsHive -> tra...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15750 **[Test build #68061 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68061/consoleFull)** for PR 15750 at commit

[GitHub] spark pull request #15652: [SPARK-16987] [None] Add spark-default.conf prope...

2016-11-03 Thread hayashidac
Github user hayashidac commented on a diff in the pull request: https://github.com/apache/spark/pull/15652#discussion_r86325009 --- Diff: docs/configuration.md --- @@ -1664,6 +1664,14 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark issue #14847: [SPARK-17254][SQL] Add StopAfter physical plan for the f...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14847 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14847: [SPARK-17254][SQL] Add StopAfter physical plan for the f...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68059/ Test PASSed. ---

[GitHub] spark issue #15658: [SPARK-13770] [Documentation][ML] Document the ML featur...

2016-11-03 Thread hayashidac
Github user hayashidac commented on the issue: https://github.com/apache/spark/pull/15658 I changed input column format. Please check again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14847: [SPARK-17254][SQL] Add StopAfter physical plan for the f...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14847 **[Test build #68059 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68059/consoleFull)** for PR 14847 at commit

[GitHub] spark pull request #15718: [SPARK-16839][SQL] Simplify Struct creation code ...

2016-11-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15718#discussion_r86322684 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -172,110 +174,80 @@ case class

[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13911 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13911 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68066/ Test FAILed. ---

[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13911 **[Test build #68066 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68066/consoleFull)** for PR 13911 at commit

[GitHub] spark issue #15744: [SPARK-18237][HIVE] hive.exec.stagingdir have no effect

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15744 **[Test build #3405 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3405/consoleFull)** for PR 15744 at commit

[GitHub] spark issue #15713: [SPARK-18196] [CORE] Optimise CompactBuffer implementati...

2016-11-03 Thread a-roberts
Github user a-roberts commented on the issue: https://github.com/apache/spark/pull/15713 The benchmark used is HiBench 5 which works with Spark 1.6, I expect there will be HiBench 6 officially available soon with [Spark 2 support](https://github.com/intel-hadoop/HiBench/pull/337)

[GitHub] spark pull request #15752: [SPARK-18250] [SQL] Minor fixes to UTF8String

2016-11-03 Thread mridulm
Github user mridulm closed the pull request at: https://github.com/apache/spark/pull/15752 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15752: [SPARK-18250] [SQL] Minor fixes to UTF8String

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15752 **[Test build #68068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68068/consoleFull)** for PR 15752 at commit

[GitHub] spark pull request #15752: [SPARK-18250] [SQL] Minor fixes to UTF8String

2016-11-03 Thread mridulm
GitHub user mridulm opened a pull request: https://github.com/apache/spark/pull/15752 [SPARK-18250] [SQL] Minor fixes to UTF8String ## What changes were proposed in this pull request? A few minor changes to utf8 string which were observed in the code. ## How was

[GitHub] spark issue #15751: [SPARK-18246][SQL] Throws an exception before execution ...

2016-11-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15751 cc @marmbrus, could you please take a look? This is the one schema verification we talked when adding `to_json` . --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #15751: [SPARK-18246][SQL] Throws an exception before exe...

2016-11-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15751#discussion_r86317692 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -143,6 +143,15 @@ class CSVFileFormat

[GitHub] spark issue #15751: [SPARK-18246][SQL] Throws an exception before execution ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15751 **[Test build #68067 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68067/consoleFull)** for PR 15751 at commit

[GitHub] spark pull request #15751: [SPARK-18246][SQL] Throws an exception before exe...

2016-11-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15751#discussion_r86317509 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -487,6 +487,21 @@ case class

[GitHub] spark pull request #15751: [SPARK-18246][SQL] Throws an exception before exe...

2016-11-03 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/15751 [SPARK-18246][SQL] Throws an exception before execution for unsupported types in Json, CSV and text functionailities ## What changes were proposed in this pull request? This PR

[GitHub] spark issue #15749: [SQL] minor - internal doc improvement for InsertIntoTab...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15749 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68057/ Test PASSed. ---

[GitHub] spark issue #15749: [SQL] minor - internal doc improvement for InsertIntoTab...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15749 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15749: [SQL] minor - internal doc improvement for InsertIntoTab...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15749 **[Test build #68057 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68057/consoleFull)** for PR 15749 at commit

[GitHub] spark issue #15743: [SPARK-18236] Reduce duplicate objects in Spark UI and H...

2016-11-03 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15743 Looks good to me, some minor queries and comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15743: [SPARK-18236] Reduce duplicate objects in Spark U...

2016-11-03 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15743#discussion_r86312868 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala --- @@ -197,8 +196,32 @@ private[spark] object UIData { shuffleWriteMetrics:

[GitHub] spark pull request #15743: [SPARK-18236] Reduce duplicate objects in Spark U...

2016-11-03 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15743#discussion_r86312686 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala --- @@ -197,8 +196,32 @@ private[spark] object UIData { shuffleWriteMetrics:

[GitHub] spark pull request #15743: [SPARK-18236] Reduce duplicate objects in Spark U...

2016-11-03 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15743#discussion_r86312418 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1089,7 +1089,7 @@ class DAGScheduler( // To avoid UI cruft,

[GitHub] spark issue #15662: [SPARK-18141][SQL] Fix to quote column names in the pred...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15662 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15662: [SPARK-18141][SQL] Fix to quote column names in the pred...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15662 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68056/ Test PASSed. ---

[GitHub] spark issue #15662: [SPARK-18141][SQL] Fix to quote column names in the pred...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15662 **[Test build #68056 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68056/consoleFull)** for PR 15662 at commit

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/15748 @yanboliang @jkbradley Could you please make a review? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15748 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15748 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68064/ Test PASSed. ---

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15748 **[Test build #68064 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68064/consoleFull)** for PR 15748 at commit

[GitHub] spark pull request #15743: [SPARK-18236] Reduce duplicate objects in Spark U...

2016-11-03 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15743#discussion_r86311221 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -885,8 +885,8 @@ private[spark] object JsonProtocol { if (json ==

[GitHub] spark issue #15742: [SPARK-16808][Core] History Server main page does not ho...

2016-11-03 Thread vijoshi
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/15742 @ajbozarth @srowen - for your review if possible. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15743: [SPARK-18236] Reduce duplicate objects in Spark U...

2016-11-03 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15743#discussion_r86310825 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -885,8 +885,8 @@ private[spark] object JsonProtocol { if (json ==

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15704 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68055/ Test PASSed. ---

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15704 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #10881: [SPARK-12967][Netty] Avoid NettyRpc error message during...

2016-11-03 Thread 1236897
Github user 1236897 commented on the issue: https://github.com/apache/spark/pull/10881 actually, I face the issue that "RpcEnv already stopped" when i call sparkContxt.stop in the end of the program. I add the spark1.6 through maven pom. could i know how to fix this issue ? ---

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15704 **[Test build #68055 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68055/consoleFull)** for PR 15704 at commit

[GitHub] spark issue #15743: [SPARK-18236] Reduce duplicate objects in Spark UI and H...

2016-11-03 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15743 +1 on UseStringDeduplication ! Though jdk8 specific --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #15749: [SQL] minor - internal doc improvement for Insert...

2016-11-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15749 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15742: [SPARK-16808][Core] History Server main page does not ho...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15742 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68052/ Test PASSed. ---

[GitHub] spark issue #15742: [SPARK-16808][Core] History Server main page does not ho...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15742 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15742: [SPARK-16808][Core] History Server main page does not ho...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15742 **[Test build #68052 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68052/consoleFull)** for PR 15742 at commit

[GitHub] spark issue #15731: [SPARK-18219] Move commit protocol API (internal) from s...

2016-11-03 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/15731 I plan to push my work tonight. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15731: [SPARK-18219] Move commit protocol API (internal)...

2016-11-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15731 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15749: [SQL] minor - internal doc improvement for InsertIntoTab...

2016-11-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15749 Merging in master/branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13911: [SPARK-16215][SQL] Reduce runtime overhead of a program ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13911 **[Test build #68066 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68066/consoleFull)** for PR 13911 at commit

[GitHub] spark issue #15731: [SPARK-18219] Move commit protocol API (internal) from s...

2016-11-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15731 Merging in master/branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15731: [SPARK-18219] Move commit protocol API (internal) from s...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15731 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68053/ Test PASSed. ---

[GitHub] spark issue #15731: [SPARK-18219] Move commit protocol API (internal) from s...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15731 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15731: [SPARK-18219] Move commit protocol API (internal) from s...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15731 **[Test build #68053 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68053/consoleFull)** for PR 15731 at commit

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15748 **[Test build #68064 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68064/consoleFull)** for PR 15748 at commit

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15044 **[Test build #68065 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68065/consoleFull)** for PR 15044 at commit

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-11-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/15044 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15748 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15748 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68063/ Test FAILed. ---

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15748 **[Test build #68063 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68063/consoleFull)** for PR 15748 at commit

[GitHub] spark issue #15744: [SPARK-18237][HIVE] hive.exec.stagingdir have no effect

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15744 **[Test build #3405 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3405/consoleFull)** for PR 15744 at commit

[GitHub] spark issue #15744: [SPARK-18237][HIVE] hive.exec.stagingdir have no effect

2016-11-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15744 LGTM pending Jenkins. But note that we might remove the idea of staging directory for writes completely in the future, when we do https://issues.apache.org/jira/browse/SPARK-15691 ---

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15748 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68062/ Test FAILed. ---

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15748 **[Test build #68062 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68062/consoleFull)** for PR 15748 at commit

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15748 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15628: [SPARK-17471][ML] Add compressed method to ML mat...

2016-11-03 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/15628#discussion_r86305786 --- Diff: mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala --- @@ -1076,4 +1240,15 @@ object Matrices {

[GitHub] spark issue #15743: [SPARK-18236] Reduce duplicate objects in Spark UI and H...

2016-11-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15743 Aside: if this and other processes see a lot of memory used by duplicate strings then maybe we should recommend trying -XX+UseStringDeduplication for a related and automatic win. --- If your

[GitHub] spark issue #14498: [SPARK-16904] [SQL] Removal of Hive Built-in Hash Functi...

2016-11-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14498 @gatorsmile can we take this to the finish line? This actually does not impact hash at all since it's only here for testing. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15748 **[Test build #68063 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68063/consoleFull)** for PR 15748 at commit

[GitHub] spark issue #15748: [SPARK-18240][ML][PySpark] Add Summary of BiKMeans and G...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15748 **[Test build #68062 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68062/consoleFull)** for PR 15748 at commit

[GitHub] spark issue #15750: [SPARK-18244][SQL] Rename partitionProviderIsHive -> tra...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15750 **[Test build #68061 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68061/consoleFull)** for PR 15750 at commit

[GitHub] spark issue #15633: [SPARK-18087] [SQL] Optimize insert to not require REPAI...

2016-11-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15633 I took a look at this again tonight, in the context of consolidating this with Hive. I think doing it through callback is actually not ideal, as callbacks are harder to trace. In 2.2 we should make

[GitHub] spark pull request #15750: [SPARK-18244][SQL] Rename partitionProviderIsHive...

2016-11-03 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15750#discussion_r86302413 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -30,11 +30,7 @@ import

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15044 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15044 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68054/ Test FAILed. ---

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15044 **[Test build #68054 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68054/consoleFull)** for PR 15044 at commit

[GitHub] spark issue #15750: [SPARK-18244][SQL] Rename partitionProviderIsHive -> tra...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15750 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15750: [SPARK-18244][SQL] Rename partitionProviderIsHive -> tra...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15750 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68060/ Test FAILed. ---

[GitHub] spark issue #15750: [SPARK-18244][SQL] Rename partitionProviderIsHive -> tra...

2016-11-03 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15750 cc @cloud-fan, @ericl, and @mallman --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15750: [SPARK-18244][SQL] Rename partitionProviderIsHive...

2016-11-03 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/15750 [SPARK-18244][SQL] Rename partitionProviderIsHive -> tracksPartitionsInCatalog ## What changes were proposed in this pull request? This patch renames partitionProviderIsHive to

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-11-03 Thread cjjnjust
Github user cjjnjust commented on the issue: https://github.com/apache/spark/pull/15172 The configuration part is now ready. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15718: [SPARK-16839][SQL] Simplify Struct creation code ...

2016-11-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15718#discussion_r86301490 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -172,110 +174,80 @@ case class

[GitHub] spark issue #14847: [SPARK-17254][SQL] Add StopAfter physical plan for the f...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14847 **[Test build #68058 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68058/consoleFull)** for PR 14847 at commit

[GitHub] spark issue #14847: [SPARK-17254][SQL] Add StopAfter physical plan for the f...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14847 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68058/ Test FAILed. ---

[GitHub] spark issue #14847: [SPARK-17254][SQL] Add StopAfter physical plan for the f...

2016-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14847 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14847: [SPARK-17254][SQL] Add StopAfter physical plan for the f...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14847 **[Test build #68059 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68059/consoleFull)** for PR 14847 at commit

[GitHub] spark issue #15027: [SPARK-17475] [STREAMING] Delete CRC files if the filesy...

2016-11-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15027 @frreiss Thanks for explanation! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14847: [SPARK-17254][SQL] Add StopAfter physical plan for the f...

2016-11-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14847 **[Test build #68058 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68058/consoleFull)** for PR 14847 at commit

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15704 Making sense. I think we can collect all matched partitions for all given specs. And do if exists check during the collecting. And then drop the distinct partitions. --- If your project is set up

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15704 When the partition specs have `equal to` only, our implementation first checks all before starting the actual drop. See the

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15704 No. The first spec will drop the partition and the second spec will be failed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15704 Let us think this more carefully. In the following case, in which the partition specs have the overlapping range ```Scala sql("ALTER TABLE sales DROP PARTITION (quarter <= 3), PARTITION

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86296370 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -204,6 +207,40 @@ class AstBuilder extends

[GitHub] spark pull request #15704: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2016-11-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15704#discussion_r86296115 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -418,27 +419,56 @@ case class AlterTableRenamePartitionCommand(

[GitHub] spark pull request #15741: [SPARK-18200][GRAPHX] Support zero as an initial ...

2016-11-03 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15741#discussion_r86296072 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -271,8 +271,12 @@ class OpenHashSet[@specialized(Long, Int) T:

<    1   2   3   4   5   6   >