[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14036 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61695/ Test PASSed. ---

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14036 **[Test build #61695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61695/consoleFull)** for PR 14036 at commit

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14004 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14004 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61694/ Test PASSed. ---

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14004 **[Test build #61694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61694/consoleFull)** for PR 14004 at commit

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14033 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61693/ Test PASSed. ---

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14033 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14033 **[Test build #61693 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61693/consoleFull)** for PR 14033 at commit

[GitHub] spark pull request #13971: [SPARK-16289][SQL] Implement posexplode table gen...

2016-07-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13971#discussion_r69395876 --- Diff: python/pyspark/sql/functions.py --- @@ -1637,6 +1637,27 @@ def explode(col): return Column(jc) +@since(2.1) +def

[GitHub] spark issue #13976: [SPARK-16288][SQL] Implement inline table generating fun...

2016-07-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13976 Thank you, @cloud-fan and @rxin ! :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13680 Sorry. This is my misunderstanding. I was confused between ```UnsafeRow``` and ```UnsafeArrayData```. ```UnsafeArrayData``` keeps only one type in an instance. ```[integer] [offset] [float]

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14036 **[Test build #61695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61695/consoleFull)** for PR 14036 at commit

[GitHub] spark pull request #13971: [SPARK-16289][SQL] Implement posexplode table gen...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13971#discussion_r69395230 --- Diff: python/pyspark/sql/functions.py --- @@ -1637,6 +1637,27 @@ def explode(col): return Column(jc) +@since(2.1) +def

[GitHub] spark pull request #13976: [SPARK-16288][SQL] Implement inline table generat...

2016-07-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13976 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #13976: [SPARK-16288][SQL] Implement inline table generating fun...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13976 merging to master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #13976: [SPARK-16288][SQL] Implement inline table generat...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13976#discussion_r69395123 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -195,3 +195,38 @@ case class Explode(child:

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13494 LGTM exception some naming/testing comments. This is not a small patch and definitely need more reviewers. Let's also add more comments inside the new rule to make it easier to review.

[GitHub] spark issue #14008: [SPARK-16281][SQL] Implement parse_url SQL function

2016-07-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14008 So far, there were some different opinions on `new URL` error handling and `Literal Pattern` handling. It's a frequent pattern of comments. :) I agree @cloud-fan 's opinions also. If I

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] support optimization for meta...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r69395021 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2865,4 +2865,15 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] support optimization for meta...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r69395014 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/MetadataOnlyOptimizerSuite.scala --- @@ -0,0 +1,87 @@ +/* + * Licensed to the

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] support optimization for meta...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r69394970 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/MetadataOnlyOptimizerSuite.scala --- @@ -0,0 +1,87 @@ +/* + * Licensed to the

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] support optimization for meta...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r69394921 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala --- @@ -30,6 +30,7 @@ class SparkOptimizer( extends

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] support optimization for meta...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r69394890 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/MetadataOnlyOptimizer.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #13976: [SPARK-16288][SQL] Implement inline table generat...

2016-07-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/13976#discussion_r69394891 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -195,3 +195,38 @@ case class Explode(child:

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] support optimization for meta...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r69394848 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2865,4 +2865,15 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14033 **[Test build #61693 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61693/consoleFull)** for PR 14033 at commit

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14004 **[Test build #61694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61694/consoleFull)** for PR 14004 at commit

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14033 cc @rxin and @cloud-fan . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14033 Rebased to resolve conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14004: [SPARK-16285][SQL] Implement sentences SQL functions

2016-07-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14004 Rebased to resolve conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13494: [SPARK-15752] [SQL] support optimization for meta...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13494#discussion_r69394772 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/MetadataOnlyOptimizer.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14036 **[Test build #61692 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61692/consoleFull)** for PR 14036 at commit

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14036 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61692/ Test FAILed. ---

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13680 Option 1 can work for this array: ```UnsafeDataArray: ...[integer] [offset] [offset] [float]```. This is because 2 offsets are adjacent. Can option 1 work for this ```UnsafeDataArray:

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14036 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13680 hmmm, looks like we are not in the same page... How could 2 offsets not adjacent? We only keep offsets in the `value or offset` region, and put them one by one. --- If your project is set up

[GitHub] spark issue #13967: [SPARK-16278][SPARK-16279][SQL] Implement map_keys/map_v...

2016-07-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13967 Thank you, @cloud-fan and @rxin ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69394475 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/UnsafeArrayDataBenchmark.scala --- @@ -0,0 +1,256 @@ +/* + * Licensed to

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14012 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61691/ Test PASSed. ---

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14012 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14012 **[Test build #61691 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61691/consoleFull)** for PR 14012 at commit

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13990 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61690/ Test PASSed. ---

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13990 **[Test build #61690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61690/consoleFull)** for PR 13990 at commit

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14036 **[Test build #61692 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61692/consoleFull)** for PR 14036 at commit

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14036 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61689/ Test FAILed. ---

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14036 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14036 **[Test build #61689 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61689/consoleFull)** for PR 14036 at commit

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14012 **[Test build #61691 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61691/consoleFull)** for PR 14012 at commit

[GitHub] spark issue #14012: [SPARK-16343][SQL] Improve the PushDownPredicate rule to...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14012 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` optimiz...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13765 LGTM, cc @yhuai to take another look --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14007: [SPARK-16329] [SQL] Star Expansion over Table Containing...

2016-07-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14007 Sure, will do it soon. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #13990: [SPARK-16287][SQL] Implement str_to_map SQL funct...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13990#discussion_r69393130 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -393,3 +393,73 @@ case class

[GitHub] spark pull request #13990: [SPARK-16287][SQL] Implement str_to_map SQL funct...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13990#discussion_r69393113 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -393,3 +393,73 @@ case class

[GitHub] spark issue #13680: [SPARK-15962][SQL] Introduce implementation with a dense...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13680 One more thought about the format: `UnsafeRow` use 8 bytes to store offset and length for variable-length type, this is because `UnsafeRow` is word-aligned, so we can't calculate the element size

[GitHub] spark issue #13409: [SPARK-15667][SQL]Throw exception if columns number of o...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13409 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13409: [SPARK-15667][SQL]Throw exception if columns number of o...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13409 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61688/ Test FAILed. ---

[GitHub] spark issue #13409: [SPARK-15667][SQL]Throw exception if columns number of o...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13409 **[Test build #61688 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61688/consoleFull)** for PR 13409 at commit

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392928 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/UnsafeArrayDataBenchmark.scala --- @@ -0,0 +1,256 @@ +/* + * Licensed

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392901 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/UnsafeArrayDataBenchmark.scala --- @@ -0,0 +1,256 @@ +/* + * Licensed

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392883 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/UnsafeArrayDataBenchmark.scala --- @@ -0,0 +1,256 @@ +/* + * Licensed

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392853 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/UnsafeArrayDataBenchmark.scala --- @@ -0,0 +1,256 @@ +/* + * Licensed

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392823 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/UnsafeArrayDataBenchmark.scala --- @@ -0,0 +1,298 @@ +/* + * Licensed

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392720 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/UnsafeArrayDataBenchmark.scala --- @@ -0,0 +1,256 @@ +/* + * Licensed

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392701 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/UnsafeArrayDataBenchmark.scala --- @@ -0,0 +1,256 @@ +/* + * Licensed

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392691 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala --- @@ -18,27 +18,110 @@ package

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392605 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala --- @@ -222,16 +226,17 @@ object

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392598 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala --- @@ -189,29 +189,33 @@ object

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392581 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java --- @@ -33,91 +38,144 @@ // The

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392548 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java --- @@ -33,91 +38,144 @@ // The

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392530 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -341,63 +325,115 @@ public UnsafeArrayData

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392506 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -341,63 +325,115 @@ public UnsafeArrayData

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392481 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -341,63 +325,115 @@ public UnsafeArrayData

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13990 **[Test build #61690 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61690/consoleFull)** for PR 13990 at commit

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392452 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -341,63 +325,115 @@ public UnsafeArrayData

[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-03 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r69392406 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -285,6 +284,75 @@ case class Divide(left:

[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-03 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r69392402 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -234,6 +234,7 @@ object FunctionRegistry {

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392355 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -341,63 +325,115 @@ public UnsafeArrayData

[GitHub] spark issue #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid unnecessa...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14036 **[Test build #61689 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61689/consoleFull)** for PR 14036 at commit

[GitHub] spark pull request #13990: [SPARK-16287][SQL] Implement str_to_map SQL funct...

2016-07-03 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/13990#discussion_r69392299 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -393,3 +393,73 @@ case class

[GitHub] spark issue #13409: [SPARK-15667][SQL]Throw exception if columns number of o...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13409 **[Test build #61688 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61688/consoleFull)** for PR 13409 at commit

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r69392218 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -341,63 +324,113 @@ public UnsafeArrayData

[GitHub] spark pull request #13990: [SPARK-16287][SQL] Implement str_to_map SQL funct...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13990#discussion_r69392198 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -393,3 +393,73 @@ case class

[GitHub] spark issue #13517: [SPARK-14839][SQL] Support for other types for `tablePro...

2016-07-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13517 Looks pretty good. Left one test related comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #13517: [SPARK-14839][SQL] Support for other types for `t...

2016-07-03 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13517#discussion_r69392182 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala --- @@ -1117,4 +1117,26 @@ class MetastoreDataSourcesSuite

[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r69392173 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -285,6 +284,75 @@ case class Divide(left:

[GitHub] spark pull request #13990: [SPARK-16287][SQL] Implement str_to_map SQL funct...

2016-07-03 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/13990#discussion_r69392113 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -393,3 +393,73 @@ case class

[GitHub] spark pull request #13990: [SPARK-16287][SQL] Implement str_to_map SQL funct...

2016-07-03 Thread techaddict
Github user techaddict commented on a diff in the pull request: https://github.com/apache/spark/pull/13990#discussion_r69392080 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -393,3 +393,73 @@ case class

[GitHub] spark pull request #14036: [SPARK-16323] [SQL] Add IntegerDivide to avoid un...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14036#discussion_r69392061 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -234,6 +234,7 @@ object FunctionRegistry {

[GitHub] spark pull request #13990: [SPARK-16287][SQL] Implement str_to_map SQL funct...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13990#discussion_r69392045 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2471,6 +2471,24 @@ object functions { def second(e: Column): Column =

[GitHub] spark pull request #13990: [SPARK-16287][SQL] Implement str_to_map SQL funct...

2016-07-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13990#discussion_r69392032 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -393,3 +393,73 @@ case class

[GitHub] spark issue #14031: [SPARK-16353][BUILD][DOC] Missing javadoc options for ja...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14031 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61687/ Test PASSed. ---

[GitHub] spark issue #14031: [SPARK-16353][BUILD][DOC] Missing javadoc options for ja...

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14031 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13704: [SPARK-15985][SQL] Reduce runtime overhead of a p...

2016-07-03 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/13704#discussion_r69391625 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -837,8 +837,36 @@ case class Cast(child: Expression,

[GitHub] spark issue #14031: [SPARK-16353][BUILD][DOC] Missing javadoc options for ja...

2016-07-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14031 **[Test build #61687 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61687/consoleFull)** for PR 14031 at commit

[GitHub] spark pull request #14035: [SPARK-16356][ML] Add testImplicits for ML unit t...

2016-07-03 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14035#discussion_r69391176 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/util/MLUtilsSuite.scala --- @@ -282,9 +281,7 @@ class MLUtilsSuite extends SparkFunSuite with

[GitHub] spark pull request #14035: [SPARK-16356][ML] Add testImplicits for ML unit t...

2016-07-03 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14035#discussion_r69391151 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -52,23 +53,20 @@ class

[GitHub] spark pull request #14035: [SPARK-16356][ML] Add testImplicits for ML unit t...

2016-07-03 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14035#discussion_r69391139 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/VectorIndexerSuite.scala --- @@ -102,7 +103,7 @@ class VectorIndexerSuite extends

[GitHub] spark pull request #14035: [SPARK-16356][ML] Add testImplicits for ML unit t...

2016-07-03 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/14035#discussion_r69391120 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/StringIndexerSuite.scala --- @@ -39,7 +40,7 @@ class StringIndexerSuite

[GitHub] spark issue #13990: [SPARK-16287][SQL] Implement str_to_map SQL function

2016-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61685/ Test PASSed. ---

<    1   2   3   4   >