[jira] [Resolved] (SPARK-18874) First phase: Deferring the correlated predicate pull up to Optimizer phase

2017-03-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18874. --- Resolution: Fixed Assignee: Dilip Biswal Fix Version/s: 2.2.0 >

[jira] [Commented] (SPARK-19933) TPCDS Q70 went wrong while explaining

2017-03-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907562#comment-15907562 ] Herman van Hovell commented on SPARK-19933: --- Here is a reproducible example: {noformat}

[jira] [Commented] (SPARK-19933) TPCDS Q70 went wrong while explaining

2017-03-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907524#comment-15907524 ] Herman van Hovell commented on SPARK-19933: --- Yeah, this is on me. I added a rule that removes

[jira] [Commented] (SPARK-19928) Incorrect error message when grouping function used with wrong types

2017-03-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907085#comment-15907085 ] Herman van Hovell commented on SPARK-19928: --- It does 'work' with one column (it gives a

[jira] [Commented] (SPARK-19928) Incorrect error message when grouping function used with wrong types

2017-03-12 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15906758#comment-15906758 ] Herman van Hovell commented on SPARK-19928: --- [~jlaskowski] It is not wrong. Grouping only takes

[jira] [Resolved] (SPARK-19786) Facilitate loop optimizations in a JIT compiler regarding range()

2017-03-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19786. --- Resolution: Fixed Assignee: Kazuaki Ishizaki Fix Version/s: 2.2.0 >

[jira] [Updated] (SPARK-19889) Make TaskContext callbacks synchronized

2017-03-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19889: -- Description: In some cases you want to fork of some part of a task to a different

[jira] [Updated] (SPARK-19889) Make TaskContext callbacks synchronized

2017-03-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19889: -- Summary: Make TaskContext callbacks synchronized (was: Make TaskContext synchronized)

[jira] [Created] (SPARK-19889) Make TaskContext synchronized

2017-03-09 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-19889: - Summary: Make TaskContext synchronized Key: SPARK-19889 URL: https://issues.apache.org/jira/browse/SPARK-19889 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19884) Add the ability to get all registered functions from a SparkSession

2017-03-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903257#comment-15903257 ] Herman van Hovell commented on SPARK-19884: --- You can use the catalog for that by calling

[jira] [Resolved] (SPARK-19865) remove the view identifier in SubqueryAlias

2017-03-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19865. --- Resolution: Fixed Fix Version/s: 2.2.0 > remove the view identifier in

[jira] [Resolved] (SPARK-17080) join reorder

2017-03-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17080. --- Resolution: Fixed Assignee: Zhenhua Wang Fix Version/s: 2.2.0 > join

[jira] [Updated] (SPARK-19850) Support aliased expressions in function parameters

2017-03-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19850: -- Issue Type: Improvement (was: Bug) > Support aliased expressions in function

[jira] [Created] (SPARK-19850) Support aliased expressions in function parameters

2017-03-07 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-19850: - Summary: Support aliased expressions in function parameters Key: SPARK-19850 URL: https://issues.apache.org/jira/browse/SPARK-19850 Project: Spark

[jira] [Commented] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897240#comment-15897240 ] Herman van Hovell commented on SPARK-19835: --- You are doing a cross join between B and C; that

[jira] [Closed] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-19835. - Resolution: Not A Problem > Runing CTAS sql on spark2.0.1 hung >

[jira] [Updated] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19835: -- Description: when I run CAST sql such as: {noformat} create table A as select a.id,

[jira] [Commented] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897182#comment-15897182 ] Herman van Hovell commented on SPARK-19835: --- >From the looks of it your job is still running;

[jira] [Commented] (SPARK-19503) Execution Plan Optimizer: avoid sort or shuffle when it does not change end result such as df.sort(...).count()

2017-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896913#comment-15896913 ] Herman van Hovell commented on SPARK-19503: --- I suppose it is fine as long as we do not change

[jira] [Resolved] (SPARK-19254) Support Seq, Map, and Struct in functions.lit

2017-03-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19254. --- Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-19805) Log the row type when query result dose not match

2017-03-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19805. --- Resolution: Fixed Assignee: Genmao Yu Fix Version/s: 2.2.0 > Log the

[jira] [Resolved] (SPARK-19710) Test Failures in SQLQueryTests on big endian platforms

2017-03-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19710. --- Resolution: Fixed Assignee: Pete Robbins Fix Version/s: 2.2.0 > Test

[jira] [Resolved] (SPARK-19758) Casting string to timestamp in inline table definition fails with AnalysisException

2017-03-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19758. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.2.0 >

[jira] [Commented] (SPARK-19503) Execution Plan Optimizer: avoid sort or shuffle when it does not change end result such as df.sort(...).count()

2017-03-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894496#comment-15894496 ] Herman van Hovell commented on SPARK-19503: --- We do not prune local sorts yet; however a user

[jira] [Commented] (SPARK-19730) Predicate Subqueries do not push results of subqueries to data source

2017-02-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884770#comment-15884770 ] Herman van Hovell commented on SPARK-19730: --- On a high level I agree that this is useful.

[jira] [Commented] (SPARK-19731) IN Operator should support arrays

2017-02-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884763#comment-15884763 ] Herman van Hovell commented on SPARK-19731: --- Why is array_contains not sufficient? {{IN}} by

[jira] [Updated] (SPARK-19691) Calculating percentile of decimal column fails with ClassCastException

2017-02-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19691: -- Fix Version/s: 2.1.1 > Calculating percentile of decimal column fails with

[jira] [Resolved] (SPARK-19691) Calculating percentile of decimal column fails with ClassCastException

2017-02-23 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19691. --- Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.2.0 >

[jira] [Commented] (SPARK-19655) select count(*) , requests 1 for each row

2017-02-18 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15873130#comment-15873130 ] Herman van Hovell commented on SPARK-19655: --- We current only push filters and columns down into

[jira] [Assigned] (SPARK-19650) Metastore-only operations shouldn't trigger a spark job

2017-02-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-19650: - Assignee: Sameer Agarwal > Metastore-only operations shouldn't trigger a spark

[jira] [Resolved] (SPARK-18873) New test cases for scalar subquery

2017-02-15 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18873. --- Resolution: Fixed Fix Version/s: 2.2.0 > New test cases for scalar subquery >

[jira] [Commented] (SPARK-19518) IGNORE NULLS in first_value / last_value should be supported in SQL statements

2017-02-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866523#comment-15866523 ] Herman van Hovell commented on SPARK-19518: --- Go for it. > IGNORE NULLS in first_value /

[jira] [Comment Edited] (SPARK-19518) IGNORE NULLS in first_value / last_value should be supported in SQL statements

2017-02-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866523#comment-15866523 ] Herman van Hovell edited comment on SPARK-19518 at 2/14/17 8:00 PM:

[jira] [Resolved] (SPARK-19589) Removal of SQLGEN files

2017-02-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19589. --- Resolution: Fixed Fix Version/s: 2.2.0 > Removal of SQLGEN files >

[jira] [Resolved] (SPARK-19544) Improve error message when some column types are compatible and others are not in set/union operations

2017-02-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19544. --- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.2.0 >

[jira] [Updated] (SPARK-19532) [Core]`DataStreamer for file` threads of DFSOutputStream leak if set `spark.speculation` to true

2017-02-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19532: -- Priority: Critical (was: Blocker) > [Core]`DataStreamer for file` threads of

[jira] [Commented] (SPARK-19532) [Core]`DataStreamer for file` threads of DFSOutputStream leak if set `spark.speculation` to true

2017-02-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863340#comment-15863340 ] Herman van Hovell commented on SPARK-19532: --- I understand that this is a blocker for you;

[jira] [Resolved] (SPARK-19496) to_date with format has weird behavior

2017-02-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19496. --- Resolution: Fixed Assignee: Song Jun Fix Version/s: 2.2.0 > to_date

[jira] [Resolved] (SPARK-19543) from_json fails when the input row is empty

2017-02-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19543. --- Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 2.2.0

[jira] [Created] (SPARK-19548) Hive UDF should support List and Map types

2017-02-10 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-19548: - Summary: Hive UDF should support List and Map types Key: SPARK-19548 URL: https://issues.apache.org/jira/browse/SPARK-19548 Project: Spark Issue

[jira] [Resolved] (SPARK-19509) GROUPING SETS throws NullPointerException when use an empty column

2017-02-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19509. --- Resolution: Fixed Assignee: StanZhai Fix Version/s: 2.1.1

[jira] [Resolved] (SPARK-19025) Remove SQL builder for operators

2017-02-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19025. --- Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-19512) codegen for compare structs fails

2017-02-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19512. --- Resolution: Fixed Assignee: Bogdan Raducanu Fix Version/s: 2.2.0 >

[jira] [Updated] (SPARK-19509) [SQL]GROUPING SETS throws NullPointerException when use an empty column

2017-02-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19509: -- Priority: Major (was: Critical) > [SQL]GROUPING SETS throws NullPointerException when

[jira] [Updated] (SPARK-19509) GROUPING SETS throws NullPointerException when use an empty column

2017-02-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19509: -- Summary: GROUPING SETS throws NullPointerException when use an empty column (was:

[jira] [Commented] (SPARK-19518) IGNORE NULLS in first_value / last_value should be supported in SQL statements

2017-02-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858163#comment-15858163 ] Herman van Hovell commented on SPARK-19518: --- Spark SQL has its own parser as of Spark 2.0. We

[jira] [Commented] (SPARK-19509) [SQL]GROUPING SETS throws NullPointerException when use a CASE WHEN statement

2017-02-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857933#comment-15857933 ] Herman van Hovell commented on SPARK-19509: --- It is in 2.1 I cannot reproduce this issue on the

[jira] [Commented] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2017-02-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857808#comment-15857808 ] Herman van Hovell commented on SPARK-18609: --- I am not fixing this for 2.0. The PR is to

[jira] [Commented] (SPARK-19510) org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 44, Column 42: A method named "evaluate" is not declared in any enclosing class nor any sup

2017-02-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857758#comment-15857758 ] Herman van Hovell commented on SPARK-19510: --- This is causes by the

[jira] [Closed] (SPARK-19510) org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 44, Column 42: A method named "evaluate" is not declared in any enclosing class nor any supert

2017-02-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-19510. - Resolution: Not A Problem > org.codehaus.commons.compiler.CompileException: File

[jira] [Closed] (SPARK-19509) [SQL]GROUPING SETS throws NullPointerException when use a CASE WHEN statement

2017-02-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-19509. - Resolution: Duplicate > [SQL]GROUPING SETS throws NullPointerException when use a CASE

[jira] [Commented] (SPARK-19509) [SQL]GROUPING SETS throws NullPointerException when use a CASE WHEN statement

2017-02-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857753#comment-15857753 ] Herman van Hovell commented on SPARK-19509: --- Hi I have tried this on master and 2.1, and this

[jira] [Updated] (SPARK-18841) PushProjectionThroughUnion exception when there are same column

2017-02-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18841: -- Fix Version/s: 2.1.1 > PushProjectionThroughUnion exception when there are same column

[jira] [Updated] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2017-02-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18609: -- Fix Version/s: 2.1.1 > [SQL] column mixup with CROSS JOIN >

[jira] [Commented] (SPARK-19503) Execution Plan Optimizer: avoid sort or shuffle when it does not change end result such as df.sort(...).count()

2017-02-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857015#comment-15857015 ] Herman van Hovell commented on SPARK-19503: --- We could prune sort and distribute operators at

[jira] [Resolved] (SPARK-18841) PushProjectionThroughUnion exception when there are same column

2017-02-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18841. --- Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-18609) [SQL] column mixup with CROSS JOIN

2017-02-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18609. --- Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-16609) Single function for parsing timestamps/dates

2017-02-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16609. --- Resolution: Fixed Assignee: Bill Chambers Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-19118) Percentile support for frequency distribution table

2017-02-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19118. --- Resolution: Fixed Assignee: gagan taneja Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-18601) Simplify Create/Get complex expression pairs in optimizer

2017-02-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18601. --- Resolution: Fixed Fix Version/s: 2.2.0 > Simplify Create/Get complex

[jira] [Updated] (SPARK-19471) [SQL]A confusing NullPointerException when creating table

2017-02-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19471: -- Description: After upgrading our Spark from 1.6.2 to 2.1.0, I encounter a confusing

[jira] [Commented] (SPARK-19451) Long values in Window function

2017-02-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15853963#comment-15853963 ] Herman van Hovell commented on SPARK-19451: --- At the end of the day I would like to support

[jira] [Commented] (SPARK-19451) Long values in Window function

2017-02-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15853800#comment-15853800 ] Herman van Hovell commented on SPARK-19451: --- Yeah, you are right about that. We should

[jira] [Commented] (SPARK-19451) Long values in Window function

2017-02-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15853782#comment-15853782 ] Herman van Hovell commented on SPARK-19451: --- [~jchamp] how may rows are in your partitions? 2

[jira] [Commented] (SPARK-19428) Ability to select first row of groupby

2017-02-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15853272#comment-15853272 ] Herman van Hovell commented on SPARK-19428: --- You could also use a window function: {noformat}

[jira] [Created] (SPARK-19459) ORC tables cannot be read when they contain char/varchar columns

2017-02-04 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-19459: - Summary: ORC tables cannot be read when they contain char/varchar columns Key: SPARK-19459 URL: https://issues.apache.org/jira/browse/SPARK-19459 Project:

[jira] [Closed] (SPARK-13678) transformExpressions should only apply on QueryPlan.expressions

2017-02-01 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-13678. - Resolution: Not A Problem > transformExpressions should only apply on

[jira] (SPARK-19383) Spark Sql Fails with Cassandra 3.6 and later PER PARTITION LIMIT option

2017-01-30 Thread Herman van Hovell (JIRA)
Title: Message Title Herman van Hovell commented on SPARK-19383

[jira] [Commented] (SPARK-19383) Spark Sql Fails with Cassandra 3.6 and later PER PARTITION LIMIT option

2017-01-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15842524#comment-15842524 ] Herman van Hovell commented on SPARK-19383: --- This is definitely not Hive/Spark syntax and

[jira] [Comment Edited] (SPARK-19383) Spark Sql Fails with Cassandra 3.6 and later PER PARTITION LIMIT option

2017-01-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15842524#comment-15842524 ] Herman van Hovell edited comment on SPARK-19383 at 1/27/17 10:39 AM: -

[jira] [Closed] (SPARK-19383) Spark Sql Fails with Cassandra 3.6 and later PER PARTITION LIMIT option

2017-01-27 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-19383. - Resolution: Not A Problem > Spark Sql Fails with Cassandra 3.6 and later PER PARTITION

[jira] [Commented] (SPARK-15505) Explode nested Array in DF Column into Multiple Columns

2017-01-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840390#comment-15840390 ] Herman van Hovell commented on SPARK-15505: --- I am closing this as a won't fix. Thanks for the

[jira] [Closed] (SPARK-15505) Explode nested Array in DF Column into Multiple Columns

2017-01-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-15505. - Resolution: Won't Fix > Explode nested Array in DF Column into Multiple Columns >

[jira] [Commented] (SPARK-15505) Explode nested Array in DF Column into Multiple Columns

2017-01-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839908#comment-15839908 ] Herman van Hovell commented on SPARK-15505: --- H This is will be quite bad in terms of

[jira] [Commented] (SPARK-19122) Unnecessary shuffle+sort added if join predicates ordering differ from bucketing and sorting order

2017-01-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839845#comment-15839845 ] Herman van Hovell commented on SPARK-19122: --- [~tejasp] We should fix this. The rules make

[jira] [Resolved] (SPARK-18936) Infrastructure for session local timezone support

2017-01-26 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18936. --- Resolution: Fixed Fix Version/s: 2.2.0 > Infrastructure for session local

[jira] [Resolved] (SPARK-18863) Output non-aggregate expressions without GROUP BY in a subquery does not yield an error

2017-01-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18863. --- Resolution: Fixed Assignee: Nattavut Sutyanyong Fix Version/s: 2.2.0

[jira] [Updated] (SPARK-19334) Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19334: -- Assignee: Kousuke Saruta > Fix the code injection vulnerability related to Generator

[jira] [Commented] (SPARK-19334) Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836770#comment-15836770 ] Herman van Hovell commented on SPARK-19334: --- I have set the target version to 2.2, since we

[jira] [Comment Edited] (SPARK-19334) Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836770#comment-15836770 ] Herman van Hovell edited comment on SPARK-19334 at 1/24/17 10:37 PM: -

[jira] [Resolved] (SPARK-19334) Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19334. --- Resolution: Fixed Fix Version/s: 2.2.0 Target Version/s: 2.2.0

[jira] [Resolved] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2017-01-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19017. --- Resolution: Fixed Assignee: Nattavut Sutyanyong Fix Version/s: 2.2.0

[jira] [Updated] (SPARK-18589) persist() resolves "java.lang.RuntimeException: Invalid PythonUDF (...), requires attributes from more than one child"

2017-01-20 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18589: -- Fix Version/s: 2.2.0 2.1.1 > persist() resolves

[jira] [Resolved] (SPARK-18589) persist() resolves "java.lang.RuntimeException: Invalid PythonUDF (...), requires attributes from more than one child"

2017-01-20 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18589. --- Resolution: Fixed > persist() resolves "java.lang.RuntimeException: Invalid

[jira] [Updated] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

2017-01-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-13721: -- Assignee: Bogdan Raducanu > Add support for LATERAL VIEW OUTER explode() >

[jira] [Resolved] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

2017-01-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-13721. --- Resolution: Fixed Fix Version/s: 2.2.0 > Add support for LATERAL VIEW OUTER

[jira] [Resolved] (SPARK-18801) Support resolve a nested view

2017-01-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18801. --- Resolution: Fixed Assignee: Jiang Xingbo > Support resolve a nested view >

[jira] [Commented] (SPARK-12076) countDistinct behaves inconsistently

2017-01-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813966#comment-15813966 ] Herman van Hovell commented on SPARK-12076: --- What is the problem? That the plans are different?

[jira] [Commented] (SPARK-18075) UDF doesn't work on non-local spark

2017-01-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15810132#comment-15810132 ] Herman van Hovell commented on SPARK-18075: --- I am happy to reopen it, if you are willing to fix

[jira] [Resolved] (SPARK-19093) Cached tables are not used in SubqueryExpression

2017-01-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19093. --- Resolution: Fixed Assignee: Dilip Biswal Fix Version/s: 2.2.0 >

[jira] [Commented] (SPARK-19086) Improper scoping of name resolution of columns in HAVING clause

2017-01-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802885#comment-15802885 ] Herman van Hovell commented on SPARK-19086: --- We alternate column resolution between the inner

[jira] [Comment Edited] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2017-01-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799374#comment-15799374 ] Herman van Hovell edited comment on SPARK-19017 at 1/4/17 9:16 PM: ---

[jira] [Commented] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2017-01-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799374#comment-15799374 ] Herman van Hovell commented on SPARK-19017: --- Thanks! > NOT IN subquery with more than one

[jira] [Commented] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2017-01-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799369#comment-15799369 ] Herman van Hovell commented on SPARK-19017: --- I agree that they are equal. It just seems weird

[jira] [Commented] (SPARK-19017) NOT IN subquery with more than one column may return incorrect results

2017-01-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15798758#comment-15798758 ] Herman van Hovell commented on SPARK-19017: --- Ok, my bad. Lets try this again. If I follow the

[jira] [Resolved] (SPARK-19060) remove the supportsPartial flag in AggregateFunction

2017-01-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19060. --- Resolution: Fixed Fix Version/s: 2.2.0 > remove the supportsPartial flag in

[jira] [Assigned] (SPARK-19070) Clean-up dataset actions

2017-01-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-19070: - Assignee: Herman van Hovell > Clean-up dataset actions >

[jira] [Created] (SPARK-19070) Clean-up dataset actions

2017-01-03 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-19070: - Summary: Clean-up dataset actions Key: SPARK-19070 URL: https://issues.apache.org/jira/browse/SPARK-19070 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-19032) Non-deterministic results using aggregation first across multiple workers

2017-01-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796650#comment-15796650 ] Herman van Hovell commented on SPARK-19032: --- [~harryw] First/Last only produce deterministic

[jira] [Resolved] (SPARK-19037) Run count(distinct x) from sub query found some errors

2016-12-30 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19037. --- Resolution: Duplicate > Run count(distinct x) from sub query found some errors >

<    1   2   3   4   5   6   7   8   9   10   >