[jira] [Resolved] (SPARK-17098) "SELECT COUNT(NULL) OVER ()" throws UnsupportedOperationException during analysis

2016-08-21 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17098. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.1.0

[jira] [Comment Edited] (SPARK-17164) Query with colon in the table name fails to parse in 2.0

2016-08-20 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429326#comment-15429326 ] Herman van Hovell edited comment on SPARK-17164 at 8/20/16 11:13 AM: -

[jira] [Commented] (SPARK-17164) Query with colon in the table name fails to parse in 2.0

2016-08-20 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15429326#comment-15429326 ] Herman van Hovell commented on SPARK-17164: --- I tried this in Hive enabled Spark 1.6: {noformat}

[jira] [Resolved] (SPARK-17117) 'SELECT 1 / NULL` throws AnalysisException, while 'SELECT 1 * NULL` works

2016-08-18 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17117. --- Resolution: Fixed Assignee: Peter Lee Fix Version/s: 2.1.0

[jira] [Resolved] (SPARK-17069) Expose spark.range() as table-valued function in SQL

2016-08-18 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17069. --- Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.1.0 > Expose

[jira] [Resolved] (SPARK-17107) Remove redundant pushdown rule for Union

2016-08-18 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17107. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.1.0 >

[jira] [Comment Edited] (SPARK-10747) add support for window specification to include how NULLS are ordered

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425521#comment-15425521 ] Herman van Hovell edited comment on SPARK-10747 at 8/17/16 10:57 PM: -

[jira] [Commented] (SPARK-10747) add support for window specification to include how NULLS are ordered

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425521#comment-15425521 ] Herman van Hovell commented on SPARK-10747: --- [~xwu0226] Would be interested to open a PR for

[jira] [Commented] (SPARK-17108) BIGINT and INT comparison failure in spark sql

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424831#comment-15424831 ] Herman van Hovell commented on SPARK-17108: --- As a work around you could add a cast to bigint or

[jira] [Updated] (SPARK-17108) BIGINT and INT comparison failure in spark sql

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17108: -- Description: I have a Hive table with the following definition: {noformat} create

[jira] [Resolved] (SPARK-17032) Add test cases for methods in ParserUtils

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17032. --- Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.1.0 > Add

[jira] [Comment Edited] (SPARK-17099) Incorrect result when HAVING clause is added to group by query

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424356#comment-15424356 ] Herman van Hovell edited comment on SPARK-17099 at 8/17/16 12:12 PM: -

[jira] [Comment Edited] (SPARK-17099) Incorrect result when HAVING clause is added to group by query

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424356#comment-15424356 ] Herman van Hovell edited comment on SPARK-17099 at 8/17/16 12:02 PM: -

[jira] [Commented] (SPARK-17099) Incorrect result when HAVING clause is added to group by query

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424356#comment-15424356 ] Herman van Hovell commented on SPARK-17099: --- TL;DR: This is caused by a bug in the optimizer's

[jira] [Updated] (SPARK-17106) Simplify subquery interface

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17106: -- Description: The current subquery expression interface contains a little bit of

[jira] [Created] (SPARK-17106) Simplify subquery interface

2016-08-17 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-17106: - Summary: Simplify subquery interface Key: SPARK-17106 URL: https://issues.apache.org/jira/browse/SPARK-17106 Project: Spark Issue Type:

[jira] [Updated] (SPARK-16947) Support type coercion and foldable expression for inline tables

2016-08-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-16947: -- Assignee: Peter Lee (was: Herman van Hovell) > Support type coercion and foldable

[jira] [Created] (SPARK-17084) Rename ParserUtils.assert to require

2016-08-16 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-17084: - Summary: Rename ParserUtils.assert to require Key: SPARK-17084 URL: https://issues.apache.org/jira/browse/SPARK-17084 Project: Spark Issue Type:

[jira] [Created] (SPARK-17083) Make the use of the default database more explicit

2016-08-16 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-17083: - Summary: Make the use of the default database more explicit Key: SPARK-17083 URL: https://issues.apache.org/jira/browse/SPARK-17083 Project: Spark

[jira] [Created] (SPARK-17068) Retain view visibility information through out Analysis

2016-08-15 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-17068: - Summary: Retain view visibility information through out Analysis Key: SPARK-17068 URL: https://issues.apache.org/jira/browse/SPARK-17068 Project: Spark

[jira] [Commented] (SPARK-6235) Address various 2G limits

2016-08-12 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419151#comment-15419151 ] Herman van Hovell commented on SPARK-6235: -- [~gq] it might be a good idea to share some design

[jira] [Resolved] (SPARK-16771) Infinite recursion loop in org.apache.spark.sql.catalyst.trees.TreeNode when table name collides.

2016-08-12 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16771. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.1.0 >

[jira] [Resolved] (SPARK-10601) Spark SQL - Support for MINUS

2016-08-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-10601. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.1.0 >

[jira] [Commented] (SPARK-10601) Spark SQL - Support for MINUS

2016-08-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414074#comment-15414074 ] Herman van Hovell commented on SPARK-10601: --- Go for it. > Spark SQL - Support for MINUS >

[jira] [Commented] (SPARK-10601) Spark SQL - Support for MINUS

2016-08-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413824#comment-15413824 ] Herman van Hovell commented on SPARK-10601: --- I am impartial to this one. It doesn't seem like

[jira] [Commented] (SPARK-16951) Alternative implementation of NOT IN to Anti-join

2016-08-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413817#comment-15413817 ] Herman van Hovell commented on SPARK-16951: --- If you read the paper a bit further along (second

[jira] [Commented] (SPARK-16951) Alternative implementation of NOT IN to Anti-join

2016-08-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413153#comment-15413153 ] Herman van Hovell commented on SPARK-16951: --- Case 1 is a result of an optimizer bug. Apparently

[jira] [Resolved] (SPARK-16804) Correlated subqueries containing non-deterministic operators return incorrect results

2016-08-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16804. --- Resolution: Fixed Fix Version/s: 2.1.0 > Correlated subqueries containing

[jira] [Created] (SPARK-16947) Improve type coercion of inline tables

2016-08-08 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-16947: - Summary: Improve type coercion of inline tables Key: SPARK-16947 URL: https://issues.apache.org/jira/browse/SPARK-16947 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-16879) unify logical plans for CREATE TABLE and CTAS

2016-08-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16879. --- Resolution: Fixed Fix Version/s: 2.1.0 > unify logical plans for CREATE TABLE

[jira] [Commented] (SPARK-16836) Hive date/time function error

2016-08-01 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15402545#comment-15402545 ] Herman van Hovell commented on SPARK-16836: --- Add braces as a work around: {{select

[jira] [Resolved] (SPARK-16726) Improve `Union/Intersect/Except` error messages on incompatible types

2016-08-01 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16726. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s:

[jira] [Comment Edited] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395218#comment-15395218 ] Herman van Hovell edited comment on SPARK-15232 at 7/27/16 8:24 AM:

[jira] [Commented] (SPARK-15232) Add subquery SQL building tests to LogicalPlanToSQLSuite

2016-07-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15395218#comment-15395218 ] Herman van Hovell commented on SPARK-15232: --- [~dongjoon] off course you can! > Add subquery

[jira] [Created] (SPARK-16749) Clean-up OffsetWindowFrame

2016-07-27 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-16749: - Summary: Clean-up OffsetWindowFrame Key: SPARK-16749 URL: https://issues.apache.org/jira/browse/SPARK-16749 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-16543) Rename the columns of `SHOW PARTITION/COLUMNS` commands

2016-07-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16543. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.1.0 >

[jira] [Resolved] (SPARK-16530) Wrong Keyword in ALTER TABLE CHANGE COLUMN

2016-07-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16530. --- Resolution: Fixed Assignee: Xiao Li Fix Version/s: 2.1.0 > Wrong

[jira] [Resolved] (SPARK-15752) Optimize metadata only query that has an aggregate whose children are deterministic project or filter operators

2016-07-12 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15752. --- Resolution: Fixed Assignee: Lianhui Wang > Optimize metadata only query that

[jira] [Commented] (SPARK-16334) [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

2016-07-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371633#comment-15371633 ] Herman van Hovell commented on SPARK-16334: --- It would also be great if we can reproduce this.

[jira] [Commented] (SPARK-16334) [SQL] SQL query on parquet table java.lang.ArrayIndexOutOfBoundsException

2016-07-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371606#comment-15371606 ] Herman van Hovell commented on SPARK-16334: --- Could you try to disable the vectorized parquet

[jira] [Resolved] (SPARK-16458) SessionCatalog should support `listColumns` for temporary tables

2016-07-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16458. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.0.0 >

[jira] [Commented] (SPARK-16484) Incremental Cardinality estimation operations with Hyperloglog

2016-07-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371571#comment-15371571 ] Herman van Hovell commented on SPARK-16484: --- I do think much of the machinery is already in

[jira] [Updated] (SPARK-16459) Prevent dropping current database

2016-07-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-16459: -- Fix Version/s: (was: 2.1.0) 2.0.0 > Prevent dropping current

[jira] [Resolved] (SPARK-16459) Prevent dropping current database

2016-07-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16459. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.1.0 >

[jira] [Resolved] (SPARK-14839) Support for other types as option in OPTIONS clause

2016-07-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-14839. --- Resolution: Resolved Assignee: Hyukjin Kwon Fix Version/s: 2.1.0 >

[jira] [Created] (SPARK-16406) Reference resolution for large number of columns should be faster

2016-07-06 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-16406: - Summary: Reference resolution for large number of columns should be faster Key: SPARK-16406 URL: https://issues.apache.org/jira/browse/SPARK-16406 Project:

[jira] [Commented] (SPARK-16211) DataFrame filter is buggy when used with "and"

2016-06-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350010#comment-15350010 ] Herman van Hovell commented on SPARK-16211: --- It would also be great if we could reproduce this.

[jira] [Commented] (SPARK-16217) Support SELECT INTO statement

2016-06-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350008#comment-15350008 ] Herman van Hovell commented on SPARK-16217: --- This seems like a sensible thing to do. Please go

[jira] [Updated] (SPARK-16217) Support SELECT INTO statement

2016-06-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-16217: -- Target Version/s: 2.1.0 > Support SELECT INTO statement >

[jira] [Updated] (SPARK-16217) Support SELECT INTO statement

2016-06-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-16217: -- Affects Version/s: (was: 2.0.0) 2.1.0 > Support SELECT INTO

[jira] [Commented] (SPARK-16183) Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql

2016-06-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15349746#comment-15349746 ] Herman van Hovell commented on SPARK-16183: --- [~dongjoon] We don't have a parser suite. The

[jira] [Commented] (SPARK-16203) regexp_extract to return an ArrayType(StringType())

2016-06-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15349737#comment-15349737 ] Herman van Hovell commented on SPARK-16203: --- I do agree that this is not efficient, but we

[jira] [Commented] (SPARK-16183) Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql

2016-06-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15349069#comment-15349069 ] Herman van Hovell commented on SPARK-16183: --- In Spark 1.6 the HiveContext uses a different

[jira] [Resolved] (SPARK-16192) Improve the type check of CollectSet in CheckAnalysis

2016-06-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16192. --- Resolution: Resolved Assignee: Takeshi Yamamuro Fix

[jira] [Updated] (SPARK-16192) Improve the type check of CollectSet in CheckAnalysis

2016-06-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-16192: -- Affects Version/s: (was: 1.6.1) 2.0.0 > Improve the type

[jira] [Created] (SPARK-16201) Expose information schema

2016-06-24 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-16201: - Summary: Expose information schema Key: SPARK-16201 URL: https://issues.apache.org/jira/browse/SPARK-16201 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-16183) Large Spark SQL commands cause StackOverflowError in parser when using sqlContext.sql

2016-06-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348986#comment-15348986 ] Herman van Hovell commented on SPARK-16183: --- We have overhauled the parser in Spark 2.0. Could

[jira] [Resolved] (SPARK-16195) Allow users to specify empty over clause in window expressions through dataset API

2016-06-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16195. --- Resolution: Resolved Assignee: Dilip Biswal > Allow users to specify empty

[jira] [Resolved] (SPARK-16123) Avoid NegativeArraySizeException while reserving additional capacity in VectorizedColumnReader

2016-06-23 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16123. --- Resolution: Resolved Assignee: Sameer Agarwal > Avoid

[jira] [Resolved] (SPARK-16078) from_utc_timestamp/to_utc_timestamp may give different result in different timezone

2016-06-22 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16078. --- Resolution: Resolved > from_utc_timestamp/to_utc_timestamp may give different result

[jira] [Resolved] (SPARK-15956) When unwrapping ORC avoid pattern matching at runtime

2016-06-22 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15956. --- Resolution: Resolved > When unwrapping ORC avoid pattern matching at runtime >

[jira] [Closed] (SPARK-15326) Doing multiple unions on a Dataframe will result in a very inefficient query plan

2016-06-21 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-15326. - Resolution: Not A Problem > Doing multiple unions on a Dataframe will result in a very

[jira] [Commented] (SPARK-15326) Doing multiple unions on a Dataframe will result in a very inefficient query plan

2016-06-21 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15343175#comment-15343175 ] Herman van Hovell commented on SPARK-15326: --- So we have code that will flatten nested Unions.

[jira] [Updated] (SPARK-15989) PySpark SQL python-only UDTs don't support nested types

2016-06-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-15989: -- Priority: Major (was: Blocker) > PySpark SQL python-only UDTs don't support nested

[jira] [Resolved] (SPARK-15977) TRUNCATE TABLE does not work with Datasource tables outside of Hive

2016-06-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15977. --- Resolution: Resolved > TRUNCATE TABLE does not work with Datasource tables outside

[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-06-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334541#comment-15334541 ] Herman van Hovell commented on SPARK-15467: --- [~kiszk] Shouldn't this be opened against the new

[jira] [Commented] (SPARK-10962) DataFrame "except" method...

2016-06-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334209#comment-15334209 ] Herman van Hovell commented on SPARK-10962: --- Except now uses {{LEFT ANTI}} joins in Spark SQL

[jira] [Created] (SPARK-15977) TRUNCATE TABLE does not work with Datasource tables outside of Hive

2016-06-15 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-15977: - Summary: TRUNCATE TABLE does not work with Datasource tables outside of Hive Key: SPARK-15977 URL: https://issues.apache.org/jira/browse/SPARK-15977

[jira] [Closed] (SPARK-15961) Audit new SQL confs

2016-06-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-15961. - Resolution: Duplicate > Audit new SQL confs > > >

[jira] [Created] (SPARK-15961) Audit new SQL confs

2016-06-14 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-15961: - Summary: Audit new SQL confs Key: SPARK-15961 URL: https://issues.apache.org/jira/browse/SPARK-15961 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-15960) Audit new SQL confs

2016-06-14 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-15960: - Summary: Audit new SQL confs Key: SPARK-15960 URL: https://issues.apache.org/jira/browse/SPARK-15960 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

2016-06-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-15730: -- Description: {noformat} /usr/lib/spark/bin/spark-sql -v --driver-memory 4g

[jira] [Updated] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

2016-06-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-15730: -- Description: {no format} /usr/lib/spark/bin/spark-sql -v --driver-memory 4g

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328200#comment-15328200 ] Herman van Hovell commented on SPARK-15822: --- [~robbinspg] Could you try this without caching?

[jira] [Commented] (SPARK-15666) Join on two tables generated from a same table throwing query analyzer issue

2016-06-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327919#comment-15327919 ] Herman van Hovell commented on SPARK-15666: --- Looking at the exception {{... in operator

[jira] [Commented] (SPARK-15666) Join on two tables generated from a same table throwing query analyzer issue

2016-06-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327818#comment-15327818 ] Herman van Hovell commented on SPARK-15666: --- [~mkbond777] Is this also a problem on 2.0? Any

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-06-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327803#comment-15327803 ] Herman van Hovell commented on SPARK-15345: --- [~m1lan] Just to be sure, is this the actual code

[jira] [Resolved] (SPARK-15814) Aggregator can return null result

2016-06-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15814. --- Resolution: Resolved > Aggregator can return null result >

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327703#comment-15327703 ] Herman van Hovell commented on SPARK-15822: --- [~robbinspg] You can dump the plan to the console

[jira] [Commented] (SPARK-15118) spark couldn't get hive properyties in hive-site.xml

2016-06-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327674#comment-15327674 ] Herman van Hovell commented on SPARK-15118: --- [~eksmile] any update on this? > spark couldn't

[jira] [Commented] (SPARK-15370) Some correlated subqueries return incorrect answers

2016-06-12 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326643#comment-15326643 ] Herman van Hovell commented on SPARK-15370: --- [~freiss] For some reason I cannot assign the

[jira] [Resolved] (SPARK-15832) Embedded IN/EXISTS predicate subquery throws TreeNodeException

2016-06-12 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15832. --- Resolution: Resolved Assignee: Ioana Delaney Fix Version/s: 2.0.0 >

[jira] [Resolved] (SPARK-15370) Some correlated subqueries return incorrect answers

2016-06-12 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15370. --- Resolution: Resolved > Some correlated subqueries return incorrect answers >

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325567#comment-15325567 ] Herman van Hovell commented on SPARK-15822: --- [~robbinspg] I have tried to reproduce the problem

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325100#comment-15325100 ] Herman van Hovell commented on SPARK-15822: --- Using spark session:

[jira] [Comment Edited] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324455#comment-15324455 ] Herman van Hovell edited comment on SPARK-15822 at 6/10/16 1:30 PM:

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324455#comment-15324455 ] Herman van Hovell commented on SPARK-15822: --- [~aroberts][~robbinspg] Can you share a

[jira] [Commented] (SPARK-15867) TABLESAMPLE BUCKET semantics don't match Hive's

2016-06-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1532#comment-1532 ] Herman van Hovell commented on SPARK-15867: --- This would change the behavior in comparison with

[jira] [Resolved] (SPARK-15850) Remove function grouping in SparkSession

2016-06-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15850. --- Resolution: Resolved > Remove function grouping in SparkSession >

[jira] [Updated] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-15822: -- Priority: Blocker (was: Critical) > segmentation violation in

[jira] [Updated] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-15822: -- Description: Executors fail with segmentation violation while running application with

[jira] [Resolved] (SPARK-15735) Allow specifying min time to run in microbenchmarks

2016-06-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15735. --- Resolution: Resolved Assignee: Eric Liang Target Version/s:

[jira] [Closed] (SPARK-15755) java.lang.NullPointerException when run spark 2.0 setting spark.serializer=org.apache.spark.serializer.KryoSerializer

2016-06-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-15755. - Resolution: Duplicate > java.lang.NullPointerException when run spark 2.0 setting >

[jira] [Resolved] (SPARK-15674) Deprecates "CREATE TEMPORARY TABLE USING...", use "CREATE TEMPORARY VIEW USING..." instead.

2016-06-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15674. --- Resolution: Resolved Assignee: Sean Zhong > Deprecates "CREATE TEMPORARY TABLE

[jira] [Created] (SPARK-15789) Allow reserved keywords in most places

2016-06-06 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-15789: - Summary: Allow reserved keywords in most places Key: SPARK-15789 URL: https://issues.apache.org/jira/browse/SPARK-15789 Project: Spark Issue Type:

[jira] [Closed] (SPARK-15758) Reenable test in hive StatisticsSuite

2016-06-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-15758. - Resolution: Duplicate > Reenable test in hive StatisticsSuite >

[jira] [Created] (SPARK-15758) Reenable test in hive StatisticsSuite

2016-06-03 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-15758: - Summary: Reenable test in hive StatisticsSuite Key: SPARK-15758 URL: https://issues.apache.org/jira/browse/SPARK-15758 Project: Spark Issue Type:

[jira] [Commented] (SPARK-15407) Floor division

2016-05-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301957#comment-15301957 ] Herman van Hovell commented on SPARK-15407: --- You can use {{div}} (this will return a long

[jira] [Resolved] (SPARK-15436) Remove DescribeFunction and ShowFunctions

2016-05-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15436. --- Resolution: Fixed Assignee: Reynold Xin > Remove DescribeFunction and

[jira] [Commented] (SPARK-15525) Clean sbt build fails to resolve sbt-antlr4 plugin

2016-05-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15300073#comment-15300073 ] Herman van Hovell commented on SPARK-15525: --- So I was kinda affraid of this:

<    5   6   7   8   9   10   11   12   13   >