[GitHub] spark pull request #14033: [SPARK-16286][SQL] Implement stack table generati...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14033#discussion_r69510671 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -94,6 +94,63 @@ case class UserDefinedGenerator(

[GitHub] spark pull request #14033: [SPARK-16286][SQL] Implement stack table generati...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14033#discussion_r69510162 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -94,6 +94,63 @@ case class UserDefinedGenerator(

[GitHub] spark issue #14044: [SPARK-16360][SQL] Speed up SQL query performance by rem...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14044 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #14008: [SPARK-16281][SQL] Implement parse_url SQL functi...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14008#discussion_r69509768 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala --- @@ -725,4 +725,51 @@ class

[GitHub] spark pull request #14008: [SPARK-16281][SQL] Implement parse_url SQL functi...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14008#discussion_r69509714 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala --- @@ -725,4 +725,51 @@ class

[GitHub] spark pull request #14008: [SPARK-16281][SQL] Implement parse_url SQL functi...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14008#discussion_r69509699 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala --- @@ -725,4 +725,51 @@ class

[GitHub] spark issue #14044: [SPARK-16360][SQL] Speed up SQL query performance by rem...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14044 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14044: [SPARK-16360][SQL] Speed up SQL query performance by rem...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14044 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61744/ Test PASSed. ---

[GitHub] spark issue #14044: [SPARK-16360][SQL] Speed up SQL query performance by rem...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14044 **[Test build #61744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61744/consoleFull)** for PR 14044 at commit

[GitHub] spark pull request #13218: [SPARK-15440] [Core] [Deploy] Add CSRF Filter for...

2016-07-04 Thread yanboliang
Github user yanboliang closed the pull request at: https://github.com/apache/spark/pull/13218 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #13218: [SPARK-15440] [Core] [Deploy] Add CSRF Filter for REST A...

2016-07-04 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/13218 Updated PR at #14052 , close this one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14052: [SPARK-15440] [Core] [Deploy] Add CSRF Filter for REST A...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14052 **[Test build #61745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61745/consoleFull)** for PR 14052 at commit

[GitHub] spark pull request #14052: [SPARK-15440] [Core] [Deploy] Add CSRF Filter for...

2016-07-04 Thread yanboliang
GitHub user yanboliang opened a pull request: https://github.com/apache/spark/pull/14052 [SPARK-15440] [Core] [Deploy] Add CSRF Filter for REST APIs to Spark ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ##

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14033 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61741/ Test PASSed. ---

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14033 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61742/ Test PASSed. ---

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14033 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14033 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14033 **[Test build #61742 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61742/consoleFull)** for PR 14033 at commit

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14033 **[Test build #61741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61741/consoleFull)** for PR 14033 at commit

[GitHub] spark issue #12972: [SPARK-15198][SQL] Support for pushing down filters for ...

2016-07-04 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/12972 No problem! thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #14024: [SPARK-15923][YARN] Spark Application rest api re...

2016-07-04 Thread Sherry302
Github user Sherry302 closed the pull request at: https://github.com/apache/spark/pull/14024 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #12972: [SPARK-15198][SQL] Support for pushing down filters for ...

2016-07-04 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/12972 LGTM, merging to master. Sorry for leaving this PR for so long... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14044: [SPARK-16360][SQL] Speed up SQL query performance by rem...

2016-07-04 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/14044 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14045: [SPARK-16362][SQL][WIP] Support ArrayType and StructType...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14045 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14045: [SPARK-16362][SQL][WIP] Support ArrayType and StructType...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14045 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61743/ Test FAILed. ---

[GitHub] spark issue #14045: [SPARK-16362][SQL][WIP] Support ArrayType and StructType...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14045 **[Test build #61743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61743/consoleFull)** for PR 14045 at commit

[GitHub] spark issue #14041: [SPARK-16359][STREAMING][KAFKA] unidoc skip kafka 0.10

2016-07-04 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14041 I just noticed that our nightly docs build has been failing with an error related to kafka (Example [1]). Will this PR fix this or should we open a new JIRA for this ? [1]

[GitHub] spark issue #14044: [SPARK-16360][SQL] Speed up SQL query performance by rem...

2016-07-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14044 Now, I update the title and description of PR/JIRA. The only patch in this PR is the following one word change. ``` -new Dataset[Row](sparkSession, logicalPlan,

[GitHub] spark issue #14044: [SPARK-16360][SQL] Speed up SQL query performance by rem...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14044 **[Test build #61744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61744/consoleFull)** for PR 14044 at commit

[GitHub] spark issue #14045: [SPARK-16362][SQL][WIP] Support ArrayType and StructType...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14045 **[Test build #61743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61743/consoleFull)** for PR 14045 at commit

[GitHub] spark issue #14044: [SPARK-16360][SQL] Speed up SQL query performance by rem...

2016-07-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14044 Hi, @cloud-fan , @hvanhovell , @liancheng . According to @cloud-fan 's advice, after changing the following, it turns out that the difference is not noticeable. ``` -new

[GitHub] spark issue #14051: [SPARK-16372][MLlib] RowMatrix constructor should use re...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14051 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14051: [SPARK-16372][MLlib] RowMatrix constructor should use re...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14051 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61739/ Test PASSed. ---

[GitHub] spark issue #14051: [SPARK-16372][MLlib] RowMatrix constructor should use re...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14051 **[Test build #61739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61739/consoleFull)** for PR 14051 at commit

[GitHub] spark issue #14045: [SPARK-16362][SQL][WIP] Support ArrayType and StructType...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14045 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14045: [SPARK-16362][SQL][WIP] Support ArrayType and StructType...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14045 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61740/ Test FAILed. ---

[GitHub] spark issue #14045: [SPARK-16362][SQL][WIP] Support ArrayType and StructType...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14045 **[Test build #61740 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61740/consoleFull)** for PR 14045 at commit

[GitHub] spark pull request #14044: [SPARK-16360][SQL] Speed up SQL query performance...

2016-07-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14044#discussion_r69503621 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -62,7 +62,7 @@ private[sql] object Dataset { def

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14033 **[Test build #61742 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61742/consoleFull)** for PR 14033 at commit

[GitHub] spark issue #14039: [SPARK-15896][SQL] Clean up shuffle files just after job...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14039 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14039: [SPARK-15896][SQL] Clean up shuffle files just after job...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14039 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61738/ Test PASSed. ---

[GitHub] spark issue #14039: [SPARK-15896][SQL] Clean up shuffle files just after job...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14039 **[Test build #61738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61738/consoleFull)** for PR 14039 at commit

[GitHub] spark pull request #14033: [SPARK-16286][SQL] Implement stack table generati...

2016-07-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14033#discussion_r69503280 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -94,6 +94,70 @@ case class

[GitHub] spark issue #14033: [SPARK-16286][SQL] Implement stack table generating func...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14033 **[Test build #61741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61741/consoleFull)** for PR 14033 at commit

[GitHub] spark pull request #14033: [SPARK-16286][SQL] Implement stack table generati...

2016-07-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14033#discussion_r69503069 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -94,6 +94,70 @@ case class

[GitHub] spark issue #12691: [Spark-14761][SQL][WIP] Reject invalid join methods when...

2016-07-04 Thread bkpathak
Github user bkpathak commented on the issue: https://github.com/apache/spark/pull/12691 Hi @JoshRosen, could you please look at the pull request? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14044: [SPARK-16360][SQL] Speed up SQL query performance by rem...

2016-07-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14044 Thank you for review, @liancheng . I'm sure that the performance of Analyzer need to be improved. But, in any cases, the cost of analyzer cannot be zero. We should skip the redundant

[GitHub] spark pull request #14044: [SPARK-16360][SQL] Speed up SQL query performance...

2016-07-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14044#discussion_r69502213 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -62,7 +62,7 @@ private[sql] object Dataset { def

[GitHub] spark issue #14051: [SPARK-16372][MLlib] RowMatrix constructor should use re...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14051 **[Test build #61739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61739/consoleFull)** for PR 14051 at commit

[GitHub] spark issue #14045: [SPARK-16362][SQL][WIP] Support ArrayType and StructType...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14045 **[Test build #61740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61740/consoleFull)** for PR 14045 at commit

[GitHub] spark issue #14044: [SPARK-16360][SQL] Speed up SQL query performance by rem...

2016-07-04 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/14044 Agree with @hvanhovell. Analysis should never take so long a time for such a simple query. We should avoid duplicated analysis work, but fixing performance issue(s) within the analyzer seems to

[GitHub] spark pull request #14046: [SPARK-16366][SPARKR] Fix time comparison failure...

2016-07-04 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/14046#discussion_r69501135 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -1258,10 +1258,12 @@ test_that("date functions on a DataFrame", { df2 <-

[GitHub] spark issue #13818: [SPARK-15968][SQL] Nonempty partitioned metastore tables...

2016-07-04 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13818 Shall we also have this in branch-2.0? This seems to be a pretty serious bug. cc @rxin. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14039: [SPARK-15896][SQL] Clean up shuffle files just after job...

2016-07-04 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/14039 @markhamstra Thanks for the comment. I think the reuse of fragments highly depends on user's queries, catalyst optimizer, cluster resources... Reusing `ShuffledRowRDD` shuffle data in a single job

[GitHub] spark pull request #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When L...

2016-07-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14034#discussion_r69499913 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -46,6 +46,15 @@ trait CheckAnalysis extends

[GitHub] spark pull request #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When L...

2016-07-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14034#discussion_r69499892 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -46,6 +46,15 @@ trait CheckAnalysis extends

[GitHub] spark pull request #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When L...

2016-07-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14034#discussion_r69499666 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -46,6 +46,15 @@ trait CheckAnalysis extends

[GitHub] spark pull request #13818: [SPARK-15968][SQL] Nonempty partitioned metastore...

2016-07-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13818 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #13818: [SPARK-15968][SQL] Nonempty partitioned metastore tables...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13818 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14039: [SPARK-15896][SQL] Clean up shuffle files just after job...

2016-07-04 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/14039 @srowen My understanding is that shuffle data in stages are possibly shared in a job. However, once the job is finished, the current implementation cannot reuse the shuffle data anymore. So, we can

[GitHub] spark issue #14039: [SPARK-15896][SQL] Clean up shuffle files just after job...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14039 **[Test build #61738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61738/consoleFull)** for PR 14039 at commit

[GitHub] spark pull request #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When L...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14034#discussion_r69498807 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -46,6 +46,15 @@ trait CheckAnalysis extends

[GitHub] spark pull request #14044: [SPARK-16360][SQL] Speed up SQL query performance...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14044#discussion_r69498744 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -62,7 +62,7 @@ private[sql] object Dataset { def ofRows(sparkSession:

[GitHub] spark pull request #14044: [SPARK-16360][SQL] Speed up SQL query performance...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14044#discussion_r69498260 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -62,7 +62,7 @@ private[sql] object Dataset { def ofRows(sparkSession:

[GitHub] spark pull request #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When L...

2016-07-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14034#discussion_r69498258 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -46,6 +46,15 @@ trait CheckAnalysis extends

[GitHub] spark pull request #14033: [SPARK-16286][SQL] Implement stack table generati...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14033#discussion_r69498156 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -94,6 +94,70 @@ case class UserDefinedGenerator(

[GitHub] spark pull request #14033: [SPARK-16286][SQL] Implement stack table generati...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14033#discussion_r69498130 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -94,6 +94,70 @@ case class UserDefinedGenerator(

[GitHub] spark pull request #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When L...

2016-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14034#discussion_r69498054 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -46,6 +46,15 @@ trait CheckAnalysis extends

[GitHub] spark issue #14051: [SPARK-16372][MLlib] RowMatrix constructor should use re...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14051 **[Test build #61737 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61737/consoleFull)** for PR 14051 at commit

[GitHub] spark issue #14051: [SPARK-16372][MLlib] RowMatrix constructor should use re...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14051 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61737/ Test FAILed. ---

[GitHub] spark issue #14051: [SPARK-16372][MLlib] RowMatrix constructor should use re...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14051 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When LIMIT/TA...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14034 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When LIMIT/TA...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14034 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61736/ Test PASSed. ---

[GitHub] spark issue #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When LIMIT/TA...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14034 **[Test build #61736 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61736/consoleFull)** for PR 14034 at commit

[GitHub] spark issue #14051: [SPARK-16372][MLlib] RowMatrix constructor should use re...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14051 **[Test build #61737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61737/consoleFull)** for PR 14051 at commit

[GitHub] spark pull request #14051: [SPARK-16372][MLlib] RowMatrix constructor should...

2016-07-04 Thread yinxusen
GitHub user yinxusen opened a pull request: https://github.com/apache/spark/pull/14051 [SPARK-16372][MLlib] RowMatrix constructor should use retag for Java compatibility ## What changes were proposed in this pull request? The following Java code because of type erasing:

[GitHub] spark issue #14048: [SPARK-16370][SQL] Union queries with side effects shoul...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14048 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61734/ Test PASSed. ---

[GitHub] spark issue #14048: [SPARK-16370][SQL] Union queries with side effects shoul...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14048 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14048: [SPARK-16370][SQL] Union queries with side effects shoul...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14048 **[Test build #61734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61734/consoleFull)** for PR 14048 at commit

[GitHub] spark issue #14049: [SPARK-16369][MLlib] tallSkinnyQR of RowMatrix should aw...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14049 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14049: [SPARK-16369][MLlib] tallSkinnyQR of RowMatrix should aw...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14049 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61735/ Test PASSed. ---

[GitHub] spark issue #14049: [SPARK-16369][MLlib] tallSkinnyQR of RowMatrix should aw...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14049 **[Test build #61735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61735/consoleFull)** for PR 14049 at commit

[GitHub] spark issue #8013: [SPARK-3181][MLLIB]: Add Robust Regression Algorithm with...

2016-07-04 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/8013 @rxin @mengxr I'm back to US from a leave. Going to revisit PRs under me. I had worked with @MechCoder to implement Huber estimator in python scikit

[GitHub] spark issue #13729: [SPARK-16008][ML] Remove unnecessary serialization in lo...

2016-07-04 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/13729 @sethah Late comment. Great improvement for high dimensional problems. I didn't test it out myself, and I wonder whether `@transient` annotation works in the constructor of `LogisticAggregator`.

[GitHub] spark issue #14050: [MINOR][EXAMPLES] Window function examples

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14050 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14050: [MINOR][EXAMPLES] Window function examples

2016-07-04 Thread aokolnychyi
GitHub user aokolnychyi opened a pull request: https://github.com/apache/spark/pull/14050 [MINOR][EXAMPLES] Window function examples ## What changes were proposed in this pull request? An example that explains the usage of window functions. It shows the difference

[GitHub] spark issue #14049: [SPARK-16369][MLlib] tallSkinnyQR of RowMatrix should aw...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14049 **[Test build #61735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61735/consoleFull)** for PR 14049 at commit

[GitHub] spark issue #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When LIMIT/TA...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14034 **[Test build #61736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61736/consoleFull)** for PR 14034 at commit

[GitHub] spark pull request #14049: [SPARK-16369][MLlib] tallSkinnyQR of RowMatrix sh...

2016-07-04 Thread yinxusen
GitHub user yinxusen opened a pull request: https://github.com/apache/spark/pull/14049 [SPARK-16369][MLlib] tallSkinnyQR of RowMatrix should aware of empty partition ## What changes were proposed in this pull request? tallSkinnyQR of RowMatrix should aware of empty

[GitHub] spark pull request #14034: [SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When L...

2016-07-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14034#discussion_r69494036 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -251,6 +251,22 @@ trait CheckAnalysis extends

[GitHub] spark pull request #14047: [SPARK-16368] [SQL] Fix Strange Errors When Creat...

2016-07-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14047#discussion_r69493923 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -391,6 +391,29 @@ class HiveDDLSuite } }

[GitHub] spark pull request #14047: [SPARK-16368] [SQL] Fix Strange Errors When Creat...

2016-07-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14047#discussion_r69493896 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -391,6 +391,29 @@ class HiveDDLSuite } }

[GitHub] spark pull request #14033: [SPARK-16286][SQL] Implement stack table generati...

2016-07-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14033#discussion_r69493271 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -94,6 +94,70 @@ case class

[GitHub] spark issue #14048: [SPARK-16370][SQL] Union queries with side effects shoul...

2016-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14048 **[Test build #61734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61734/consoleFull)** for PR 14048 at commit

[GitHub] spark pull request #14048: [SPARK-16370][SQL] Union queries with side effect...

2016-07-04 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/14048 [SPARK-16370][SQL] Union queries with side effects should be executed eagerly ## What changes were proposed in this pull request? Currently, some queries having side effects like

[GitHub] spark issue #14031: [SPARK-16353][BUILD][DOC] Missing javadoc options for ja...

2016-07-04 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/14031 Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #13796: [SPARK-7159][ML] Add multiclass logistic regression to S...

2016-07-04 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/13796 @sethah I apologize for the delay. I just came back to US. Gonna make the first pass. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #13984: [SPARK-16310][SPARKR] R na.string-like default fo...

2016-07-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/13984#discussion_r69492220 --- Diff: R/pkg/R/SQLContext.R --- @@ -744,6 +747,9 @@ read.df.default <- function(path = NULL, source = NULL, schema = NULL, ...) { if

[GitHub] spark issue #14047: [SPARK-16368] [SQL] Fix Strange Errors When Creating Vie...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14047 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61733/ Test PASSed. ---

[GitHub] spark issue #14047: [SPARK-16368] [SQL] Fix Strange Errors When Creating Vie...

2016-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14047 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

  1   2   3   4   >