[GitHub] spark issue #15394: [SPARK-17748][ML] One pass solver for Weighted Least Squ...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15394 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15394: [SPARK-17748][ML] One pass solver for Weighted Least Squ...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67374/ Test PASSed. ---

[GitHub] spark issue #15394: [SPARK-17748][ML] One pass solver for Weighted Least Squ...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15394 **[Test build #67374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67374/consoleFull)** for PR 15394 at commit

[GitHub] spark issue #12913: [SPARK-928][CORE] Add support for Unsafe-based serialize...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12913 **[Test build #67377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67377/consoleFull)** for PR 12913 at commit

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r84570784 --- Diff: core/src/test/scala/org/apache/spark/serializer/KryoBenchmark.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r84570771 --- Diff: core/src/test/scala/org/apache/spark/serializer/KryoBenchmark.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r84570758 --- Diff: core/src/test/scala/org/apache/spark/serializer/KryoBenchmark.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #12913: [SPARK-928][CORE] Add support for Unsafe-based serialize...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12913 **[Test build #67376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67376/consoleFull)** for PR 12913 at commit

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r84570741 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -248,8 +266,9 @@ class KryoDeserializationStream( } }

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r84570738 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -219,9 +232,14 @@ class KryoSerializationStream( private[spark]

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r84570736 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -186,9 +193,15 @@ class KryoSerializer(conf: SparkConf)

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r84570727 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -78,8 +79,14 @@ class KryoSerializer(conf: SparkConf)

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r84570704 --- Diff: core/src/test/scala/org/apache/spark/serializer/UnsafeKryoSerializerSuite.scala --- @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-21 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r84570678 --- Diff: core/src/test/scala/org/apache/spark/serializer/UnsafeKryoSerializerSuite.scala --- @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #15178: [SPARK-17556][SQL] Executor side broadcast for broadcast...

2016-10-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15178 @rxin ok. no problem. Please let me know if anything I need to explain more in details. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15587: [SPARK-18051][Spark Core] fix bug of custom PartitionCoa...

2016-10-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15587 Is this an API breaking change? How about we just add a require check in CoalesceRDD to verify the passed in PartitionCoalescer is serializable? --- If your project is set up for it, you

[GitHub] spark issue #15178: [SPARK-17556][SQL] Executor side broadcast for broadcast...

2016-10-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15178 hm sorry for the delay. This is pretty complicated and we need to compare the two implementations. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15586: [MINOR] Tiny follow-up to SPARK-16606, to correct...

2016-10-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15586 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15590: [SPARK-17949][SQL] A JVM object based aggregate operator

2016-10-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15590 BTW overall this patch is really well written! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15586: [MINOR] Tiny follow-up to SPARK-16606, to correct more i...

2016-10-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15586 Merging in master / branch-2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15573: [SPARK-18035] [SQL] Unwrapping java maps in HiveInspecto...

2016-10-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15573 High level change LGTM. @gatorsmile can you do the final review & merge? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15587: [SPARK-18051][Spark Core] fix bug of custom PartitionCoa...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15587 **[Test build #67375 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67375/consoleFull)** for PR 15587 at commit

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15377 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67368/ Test PASSed. ---

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #67368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67368/consoleFull)** for PR 15377 at commit

[GitHub] spark issue #15394: [SPARK-17748][ML] One pass solver for Weighted Least Squ...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15394 **[Test build #67374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67374/consoleFull)** for PR 15394 at commit

[GitHub] spark issue #15394: [SPARK-17748][ML] One pass solver for Weighted Least Squ...

2016-10-21 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15394 jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15596 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67371/ Test FAILed. ---

[GitHub] spark issue #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15596 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15596 **[Test build #67371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67371/consoleFull)** for PR 15596 at commit

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-21 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r84570152 --- Diff: core/src/test/scala/org/apache/spark/serializer/UnsafeKryoSerializerSuite.scala --- @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #15573: [SPARK-18035] [SQL] Unwrapping java maps in HiveInspecto...

2016-10-21 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/15573 >> Could you do a quick check for all the callers who are using I have updated all places except three : - JacksonParser : can be abstracted out but the API will be too specific to

[GitHub] spark issue #15573: [SPARK-18035] [SQL] Unwrapping java maps in HiveInspecto...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15573 **[Test build #67373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67373/consoleFull)** for PR 15573 at commit

[GitHub] spark pull request #15573: [SPARK-18035] [SQL] Unwrapping java maps in HiveI...

2016-10-21 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/15573#discussion_r84569857 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayBasedMapData.scala --- @@ -35,6 +39,23 @@ object ArrayBasedMapData {

[GitHub] spark issue #15573: [SPARK-18035] [SQL] Unwrapping java maps in HiveInspecto...

2016-10-21 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/15573 @gatorsmile > Could you explain the pattern? For example, How many do such columns exist in your query? How large is your table? Are your query complex? Or just a simple SELECT query?

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15595 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67367/ Test PASSed. ---

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15595 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15595 **[Test build #67367 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67367/consoleFull)** for PR 15595 at commit

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15398 @jodersky I think the current approach is fine if we have documented it clear. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15585: [SPARK-18049][MLLIB][TEST] Add missing tests for truePos...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15585 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15585: [SPARK-18049][MLLIB][TEST] Add missing tests for truePos...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15585 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67369/ Test PASSed. ---

[GitHub] spark issue #15585: [SPARK-18049][MLLIB][TEST] Add missing tests for truePos...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15585 **[Test build #67369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67369/consoleFull)** for PR 15585 at commit

[GitHub] spark pull request #14553: [SPARK-16963] [STREAMING] [SQL] Changes to Source...

2016-10-21 Thread frreiss
Github user frreiss commented on a diff in the pull request: https://github.com/apache/spark/pull/14553#discussion_r84569335 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -337,17 +343,27 @@ class StreamExecution(

[GitHub] spark issue #15585: [SPARK-18049][MLLIB][TEST] Add missing tests for truePos...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15585 **[Test build #67372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67372/consoleFull)** for PR 15585 at commit

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15398 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15398 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67366/ Test PASSed. ---

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15398 **[Test build #67366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67366/consoleFull)** for PR 15398 at commit

[GitHub] spark issue #15516: [SPARK-17961][SparkR][SQL] Add storageLevel to DataFrame...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15516 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15516: [SPARK-17961][SparkR][SQL] Add storageLevel to DataFrame...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15516 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67370/ Test PASSed. ---

[GitHub] spark issue #15516: [SPARK-17961][SparkR][SQL] Add storageLevel to DataFrame...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15516 **[Test build #67370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67370/consoleFull)** for PR 15516 at commit

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67365/ Test PASSed. ---

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67365 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67365/consoleFull)** for PR 15515 at commit

[GitHub] spark issue #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15596 BTW, we can see if there is an exchange added for `CollectLimit` from physical plan, e.g., CollectLimit 1 +- Exchange SinglePartition +- *LocalLimit 1 +-

[GitHub] spark pull request #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-21 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15596#discussion_r84569198 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -83,9 +83,8 @@ trait BaseLimitExec extends UnaryExecNode with

[GitHub] spark pull request #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-21 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15596#discussion_r84569163 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/BufferedRowIterator.java --- @@ -39,7 +39,7 @@ protected int partitionIndex = -1;

[GitHub] spark issue #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15596 **[Test build #67371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67371/consoleFull)** for PR 15596 at commit

[GitHub] spark issue #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15596 cc @rxin @yhuai May you take a quick look to see if this direction is ok for you? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-21 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/15596 [SQL] Remove shuffle codes in CollectLimitExec ## What changes were proposed in this pull request? `CollectLimitExec` now shuffles its child's data in `doExecute`. As we have

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15595 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67364/ Test PASSed. ---

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15595 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15595 **[Test build #67364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67364/consoleFull)** for PR 15595 at commit

[GitHub] spark issue #15516: [SPARK-17961][SparkR][SQL] Add storageLevel to DataFrame...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15516 **[Test build #67370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67370/consoleFull)** for PR 15516 at commit

[GitHub] spark issue #15516: [SPARK-17961][SparkR][SQL] Add storageLevel to DataFrame...

2016-10-21 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15516 @felixcheung Remove the unrelated jar file. and about the `String` look up table, here seems there are not the mapping relationship between these String constant, so that the code I

[GitHub] spark issue #15587: [SPARK-18051][Spark Core] fix bug of custom PartitionCoa...

2016-10-21 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15587 @rxin How about this way: modify the parameter as `Option[PartitionerCoalescer with Serializable]`, so that if user forget to mark the implementation class as serializable, compiling time will

[GitHub] spark issue #15575: [SPARK-18038] [SQL] Move output partitioning definition ...

2016-10-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15575 But `ExpandExec`'s `GroupExpressions` could generate data with arbitrary partitioning, so even it doesn't move rows across input partitions, doesn't it still can change child' output partition?

[GitHub] spark issue #15585: [SPARK-18049][MLLIB][TEST] Add missing tests for truePos...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15585 **[Test build #67369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67369/consoleFull)** for PR 15585 at commit

[GitHub] spark issue #15575: [SPARK-18038] [SQL] Move output partitioning definition ...

2016-10-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15575 `CoalesceExec` can also have `SinglePartition` as `outputPartitioning` too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15377: [SPARK-17802] Improved caller context logging.

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15377 **[Test build #67368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67368/consoleFull)** for PR 15377 at commit

[GitHub] spark issue #15539: [SPARK-17994] [SQL] Add back a file status cache for cat...

2016-10-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15539 I had some network problems, I'll ask @yhuai to merge it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #15585: [SPARK-18049][MLLIB][TEST] Add missing tests for ...

2016-10-21 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/15585#discussion_r84568230 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/RankingMetricsSuite.scala --- @@ -65,5 +65,4 @@ class RankingMetricsSuite extends

[GitHub] spark issue #15539: [SPARK-17994] [SQL] Add back a file status cache for cat...

2016-10-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15539 LGTM, next maybe we can refactor the `PartitionAwareFileCatalog` and make it use the new global cache better. I'm going to merge it to unblock other works. thanks! --- If your project is set up

[GitHub] spark issue #15394: [SPARK-17748][ML] One pass solver for Weighted Least Squ...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67363/ Test FAILed. ---

[GitHub] spark issue #15394: [SPARK-17748][ML] One pass solver for Weighted Least Squ...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15394 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15394: [SPARK-17748][ML] One pass solver for Weighted Least Squ...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15394 **[Test build #67363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67363/consoleFull)** for PR 15394 at commit

[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2016-10-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15539#discussion_r84568025 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala --- @@ -38,14 +38,16 @@ class ListingFileCatalog(

[GitHub] spark issue #15592: [SPARK-17624][SQL][STREAMING][TEST] Fixed flaky StateSto...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15592 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15592: [SPARK-17624][SQL][STREAMING][TEST] Fixed flaky StateSto...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15592 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67361/ Test PASSed. ---

[GitHub] spark issue #15592: [SPARK-17624][SQL][STREAMING][TEST] Fixed flaky StateSto...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15592 **[Test build #67361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67361/consoleFull)** for PR 15592 at commit

[GitHub] spark issue #15581: [SPARK-18044][STREAMING] FileStreamSource should not inf...

2016-10-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15581 yea of course, I'll do it soon --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15595 **[Test build #67367 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67367/consoleFull)** for PR 15595 at commit

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-21 Thread jodersky
Github user jodersky commented on the issue: https://github.com/apache/spark/pull/15398 I went ahead and changed the parsing function to throw an AnalysisException in case the input escaper sequence is invalid. Feel free to revert (or just tell me and I'll do it) if we should stick

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15398 **[Test build #67366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67366/consoleFull)** for PR 15398 at commit

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r84566821 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r84565977 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/BitSampling.scala --- @@ -0,0 +1,163 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r84566599 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -245,50 +246,50 @@ private[ml] abstract class LSHModel[T <: LSHModel[T]]

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r84566581 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r84566224 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/BitSamplingSuite.scala --- @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r84566504 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15595 **[Test build #67364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67364/consoleFull)** for PR 15595 at commit

[GitHub] spark issue #15594: [SPARK-18061][SQL][Security] Spark Thriftserver needs to...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15594 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67365/consoleFull)** for PR 15515 at commit

[GitHub] spark pull request #15595: [SPARK-18058][SQL] Comparing column types ignorin...

2016-10-21 Thread CodingCat
GitHub user CodingCat opened a pull request: https://github.com/apache/spark/pull/15595 [SPARK-18058][SQL] Comparing column types ignoring Nullability in BinaryOperator ## What changes were proposed in this pull request? The PR tries to fix

[GitHub] spark pull request #15594: [SPARK-18061][SQL][Security] Spark Thriftserver n...

2016-10-21 Thread cmirash
GitHub user cmirash opened a pull request: https://github.com/apache/spark/pull/15594 [SPARK-18061][SQL][Security] Spark Thriftserver needs to create SPNego principal ## What changes were proposed in this pull request? Spark Thriftserver when running in HTTP mode with Kerberos

[GitHub] spark issue #15463: [SPARK-17894] [CORE] Ensure uniqueness of TaskSetManager...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15463 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67349/ Test FAILed. ---

[GitHub] spark issue #15463: [SPARK-17894] [CORE] Ensure uniqueness of TaskSetManager...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15463 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15463: [SPARK-17894] [CORE] Ensure uniqueness of TaskSetManager...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15463 **[Test build #67349 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67349/consoleFull)** for PR 15463 at commit

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67359/ Test FAILed. ---

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67359 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67359/consoleFull)** for PR 15515 at commit

  1   2   3   4   5   >