[jira] [Commented] (SPARK-18172) AnalysisException in first/last during aggregation

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15671001#comment-15671001 ] Herman van Hovell commented on SPARK-18172: --- [~emlyn] can you still reproduce this? >

[jira] [Commented] (SPARK-18172) AnalysisException in first/last during aggregation

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670990#comment-15670990 ] Herman van Hovell commented on SPARK-18172: --- I cannot reproduce this on branch-2.0 or master. I

[jira] [Updated] (SPARK-18220) ClassCastException occurs when using select query on ORC file

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18220: -- Target Version/s: 2.1.0 Fix Version/s: (was: 2.1.0) > ClassCastException

[jira] [Updated] (SPARK-18220) ClassCastException occurs when using select query on ORC file

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18220: -- Fix Version/s: 2.1.0 > ClassCastException occurs when using select query on ORC file >

[jira] [Closed] (SPARK-18242) Help with spark 1.5.2 Error

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-18242. - Resolution: Not A Problem > Help with spark 1.5.2 Error > --- >

[jira] [Commented] (SPARK-18242) Help with spark 1.5.2 Error

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670925#comment-15670925 ] Herman van Hovell commented on SPARK-18242: --- Please ask this question on the user list or on

[jira] [Commented] (SPARK-18247) not rlike is not supported in DataFrame filter function

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670918#comment-15670918 ] Herman van Hovell commented on SPARK-18247: --- I am going to close this since this has been fixed

[jira] [Closed] (SPARK-18247) not rlike is not supported in DataFrame filter function

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-18247. - Resolution: Won't Fix > not rlike is not supported in DataFrame filter function >

[jira] [Resolved] (SPARK-18415) Weird Plan Output when CTE used in RunnableCommand

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18415. --- Resolution: Fixed Target Version/s: 2.1.0 > Weird Plan Output when CTE

[jira] [Commented] (SPARK-18251) DataSet API | RuntimeException: Null value appeared in non-nullable field when holding Option Case Class

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670877#comment-15670877 ] Herman van Hovell commented on SPARK-18251: --- I cannot reproduce this on master or branch 2.1.

[jira] [Commented] (SPARK-18249) Spark 2.0.1 - StackOverflowError when saving dataset to parquet

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670836#comment-15670836 ] Herman van Hovell commented on SPARK-18249: --- Could provide an example that we can reproduce? >

[jira] [Updated] (SPARK-18389) Disallow cyclic view reference

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18389: -- Target Version/s: 2.2.0 > Disallow cyclic view reference >

[jira] [Updated] (SPARK-18358) Multiple Aggregation Using 'countDistinct' and 'first' result in error

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18358: -- Target Version/s: 2.1.0 > Multiple Aggregation Using 'countDistinct' and 'first'

[jira] [Updated] (SPARK-18388) Running aggregation on many columns throws SOE

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18388: -- Target Version/s: 2.1.0 > Running aggregation on many columns throws SOE >

[jira] [Updated] (SPARK-18394) Executing the same query twice in a row results in CodeGenerator cache misses

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18394: -- Target Version/s: 2.1.0 > Executing the same query twice in a row results in

[jira] [Updated] (SPARK-18436) isin causing SQL syntax error with JDBC

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18436: -- Target Version/s: 2.1.0 > isin causing SQL syntax error with JDBC >

[jira] [Updated] (SPARK-18455) General support for subquery processing

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18455: -- Target Version/s: 2.2.0 Description: Subquery support has been introduced in

[jira] [Commented] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670762#comment-15670762 ] Herman van Hovell commented on SPARK-18458: --- [~kiszk] Could you keep us posted of the progress

[jira] [Updated] (SPARK-18458) core dumped running Spark SQL on large data volume (100TB)

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18458: -- Priority: Critical (was: Major) > core dumped running Spark SQL on large data volume

[jira] [Closed] (SPARK-6482) Remove synchronization of Hive Native commands

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-6482. Resolution: Not A Problem Fix Version/s: 2.0.0 > Remove synchronization of Hive

[jira] [Commented] (SPARK-6482) Remove synchronization of Hive Native commands

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670750#comment-15670750 ] Herman van Hovell commented on SPARK-6482: -- The dependency on Hive commands has been removed in

[jira] [Closed] (SPARK-8466) Bug in SQL Optimizer: Unresolved Attribute after pushing Filter below Project

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-8466. Resolution: Not A Problem > Bug in SQL Optimizer: Unresolved Attribute after pushing Filter

[jira] [Commented] (SPARK-8466) Bug in SQL Optimizer: Unresolved Attribute after pushing Filter below Project

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670742#comment-15670742 ] Herman van Hovell commented on SPARK-8466: -- I cannot reproduce this under Spark 1.6/2.0. See:

[jira] [Closed] (SPARK-10794) Spark-SQL- select query on table column with binary Data Type displays error message- java.lang.ClassCastException: java.lang.String cannot be cast to [B

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-10794. - Resolution: Unresolved Closing this due to lack of information. > Spark-SQL- select

[jira] [Commented] (SPARK-11266) Peak memory tests swallow failures

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670694#comment-15670694 ] Herman van Hovell commented on SPARK-11266: --- [~andrewor14] is this an issue? > Peak memory

[jira] [Commented] (SPARK-11785) When deployed against remote Hive metastore with lower versions, JDBC metadata calls throws exception

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670692#comment-15670692 ] Herman van Hovell commented on SPARK-11785: --- [~lian cheng] is this still an issue? > When

[jira] [Closed] (SPARK-12179) Spark SQL get different result with the same code

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-12179. - Resolution: Cannot Reproduce > Spark SQL get different result with the same code >

[jira] [Comment Edited] (SPARK-12179) Spark SQL get different result with the same code

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670679#comment-15670679 ] Herman van Hovell edited comment on SPARK-12179 at 11/16/16 3:19 PM: -

[jira] [Commented] (SPARK-12179) Spark SQL get different result with the same code

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670679#comment-15670679 ] Herman van Hovell commented on SPARK-12179: --- [~litao1990] is this still a problem? > Spark SQL

[jira] [Updated] (SPARK-12278) Move the shuffle related test case from Yarn module to Core module

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-12278: -- Priority: Major (was: Critical) > Move the shuffle related test case from Yarn module

[jira] [Closed] (SPARK-12352) Reuse the result of split in SQL

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-12352. - Resolution: Duplicate > Reuse the result of split in SQL >

[jira] [Closed] (SPARK-14067) spark-shell WARN ObjectStore: Failed to get database default,returning NoSuchObjectException

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-14067. - Resolution: Not A Problem > spark-shell WARN ObjectStore: Failed to get database

[jira] [Commented] (SPARK-14003) Multi-session can not work when one session is moving files for "INSERT ... SELECT" clause

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670649#comment-15670649 ] Herman van Hovell commented on SPARK-14003: --- This is a know issue. We use a single Hive client

[jira] [Updated] (SPARK-14003) Multi-session can not work when one session is moving files for "INSERT ... SELECT" clause

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-14003: -- Priority: Major (was: Critical) > Multi-session can not work when one session is

[jira] [Closed] (SPARK-13935) Other clients' connection hang up when someone do huge load

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-13935. - Resolution: Duplicate > Other clients' connection hang up when someone do huge load >

[jira] [Commented] (SPARK-14003) Multi-session can not work when one session is moving files for "INSERT ... SELECT" clause

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670650#comment-15670650 ] Herman van Hovell commented on SPARK-14003: --- This is a know issue. We use a single Hive client

[jira] [Commented] (SPARK-14067) spark-shell WARN ObjectStore: Failed to get database default,returning NoSuchObjectException

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670644#comment-15670644 ] Herman van Hovell commented on SPARK-14067: --- This is expected behavior. We need the default

[jira] [Closed] (SPARK-14171) UDAF aggregates argument object inspector not parsed correctly

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-14171. - Resolution: Fixed Fix Version/s: 2.1.0 Spark SQL now implements its Approx

[jira] [Commented] (SPARK-16981) For CSV files nullValue is not respected for Date/Time data type

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670614#comment-15670614 ] Herman van Hovell commented on SPARK-16981: --- Could you retest this on the current master? I

[jira] [Commented] (SPARK-17977) DataFrameReader and DataStreamReader should have an ancestor class

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670578#comment-15670578 ] Herman van Hovell commented on SPARK-17977: --- [~marmbrus] Is there any reason for

[jira] [Updated] (SPARK-18091) Deep if expressions cause Generated SpecificUnsafeProjection code to exceed JVM code size limit

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18091: -- Target Version/s: 2.1.0 > Deep if expressions cause Generated SpecificUnsafeProjection

[jira] [Updated] (SPARK-18004) DataFrame filter Predicate push-down fails for Oracle Timestamp type columns

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18004: -- Description: DataFrame filter Predicate push-down fails for Oracle Timestamp type

[jira] [Updated] (SPARK-18004) DataFrame filter Predicate push-down fails for Oracle Timestamp type columns

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18004: -- Description: DataFrame filter Predicate push-down fails for Oracle Timestamp type

[jira] [Commented] (SPARK-18004) DataFrame filter Predicate push-down fails for Oracle Timestamp type columns

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670571#comment-15670571 ] Herman van Hovell commented on SPARK-18004: --- So there seems to something going on with the date

[jira] [Updated] (SPARK-9221) Support IntervalType in Range Frame

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-9221: - Affects Version/s: (was: 1.4.0) Target Version/s: 2.2.0 > Support IntervalType

[jira] [Updated] (SPARK-9221) Support IntervalType in Range Frame

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-9221: - Issue Type: Improvement (was: Sub-task) Parent: (was: SPARK-7712) > Support

[jira] [Commented] (SPARK-8816) Improve Performance of Unbounded Following Window Frame Processing

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15670524#comment-15670524 ] Herman van Hovell commented on SPARK-8816: -- The following paper provides a nice solution if we

[jira] [Updated] (SPARK-18430) Returned Message Null when Hitting an Invocation Exception of Function Lookup.

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18430: -- Fix Version/s: 2.0.3 > Returned Message Null when Hitting an Invocation Exception of

[jira] [Updated] (SPARK-18424) Improve Date Parsing Semantics & Functionality

2016-11-16 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18424: -- Shepherd: Herman van Hovell Target Version/s: 2.2.0 > Improve Date Parsing

[jira] [Updated] (SPARK-17732) ALTER TABLE DROP PARTITION should support comparators

2016-11-15 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17732: -- Fix Version/s: (was: 2.2.0) 2.1.0 > ALTER TABLE DROP PARTITION

[jira] [Commented] (SPARK-18388) Running aggregation on many columns throws SOE

2016-11-15 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15668805#comment-15668805 ] Herman van Hovell commented on SPARK-18388: --- [~bond_] Yeah this should be fixed. I lowered the

[jira] [Resolved] (SPARK-17732) ALTER TABLE DROP PARTITION should support comparators

2016-11-15 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17732. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.2.0 >

[jira] [Resolved] (SPARK-17348) Incorrect results from subquery transformation

2016-11-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17348. --- Resolution: Fixed Assignee: Nattavut Sutyanyong Fix Version/s: 2.1.0

[jira] [Commented] (SPARK-18436) isin causing SQL syntax error with JDBC

2016-11-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15664803#comment-15664803 ] Herman van Hovell commented on SPARK-18436: --- An empty {{IN}} is currently not folded during

[jira] [Updated] (SPARK-18436) isin causing SQL syntax error with JDBC

2016-11-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18436: -- Description: When using a JDBC data source, the "isin" function generates invalid SQL

[jira] [Commented] (SPARK-18134) SQL: MapType in Group BY and Joins not working

2016-11-14 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15663503#comment-15663503 ] Herman van Hovell commented on SPARK-18134: --- This is a non-trivial change. Could you explain me

[jira] [Updated] (SPARK-18424) Improve Date Parsing Functionality

2016-11-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18424: -- Assignee: Bill Chambers > Improve Date Parsing Functionality >

[jira] [Commented] (SPARK-18300) ClassCastException during count distinct

2016-11-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15658171#comment-15658171 ] Herman van Hovell commented on SPARK-18300: --- Hmmm... The

[jira] [Updated] (SPARK-18300) ClassCastException during count distinct

2016-11-11 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18300: -- Labels: correctness (was: ) > ClassCastException during count distinct >

[jira] [Updated] (SPARK-18388) Running aggregation on many columns throws SOE

2016-11-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18388: -- Priority: Major (was: Critical) > Running aggregation on many columns throws SOE >

[jira] [Commented] (SPARK-18388) Running aggregation on many columns throws SOE

2016-11-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655568#comment-15655568 ] Herman van Hovell commented on SPARK-18388: --- This is caused by the

[jira] [Commented] (SPARK-18388) Running aggregation on many columns throws SOE

2016-11-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655379#comment-15655379 ] Herman van Hovell commented on SPARK-18388: --- Ok, thanks. I'll take a look. > Running

[jira] [Commented] (SPARK-18387) Test that expressions can be serialized

2016-11-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655224#comment-15655224 ] Herman van Hovell commented on SPARK-18387: --- [~rdblue] Are you working on this one? I'd be

[jira] [Commented] (SPARK-18367) DataFrame join spawns unreasonably high number of open files

2016-11-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655036#comment-15655036 ] Herman van Hovell commented on SPARK-18367: --- Ok, I am very curious to know what is going on. >

[jira] [Commented] (SPARK-18367) limit() makes the lame walk again

2016-11-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654873#comment-15654873 ] Herman van Hovell commented on SPARK-18367: --- I think this one is on Spark. As a user you cannot

[jira] [Commented] (SPARK-18388) Running aggregation on many columns throws SOE

2016-11-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652104#comment-15652104 ] Herman van Hovell commented on SPARK-18388: --- Could you try this on master? We added an

[jira] [Commented] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15652022#comment-15652022 ] Herman van Hovell commented on SPARK-18390: --- I don't think this is a bug. It is doing exactly

[jira] [Commented] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651947#comment-15651947 ] Herman van Hovell commented on SPARK-18390: --- [~vssrinath] Can you take a look? > Optimized

[jira] [Updated] (SPARK-18390) Optimized plan tried to use Cartesian join when it is not enabled

2016-11-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18390: -- Assignee: Srinath > Optimized plan tried to use Cartesian join when it is not enabled

[jira] [Created] (SPARK-18370) InsertIntoHadoopFsRelationCommand should keep track of its table

2016-11-08 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-18370: - Summary: InsertIntoHadoopFsRelationCommand should keep track of its table Key: SPARK-18370 URL: https://issues.apache.org/jira/browse/SPARK-18370 Project:

[jira] [Commented] (SPARK-18367) limit() makes the lame walk again

2016-11-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15649110#comment-15649110 ] Herman van Hovell commented on SPARK-18367: --- Could you try this without caching? > limit()

[jira] [Commented] (SPARK-18367) limit() makes the lame walk again

2016-11-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15649022#comment-15649022 ] Herman van Hovell commented on SPARK-18367: --- You might be trying to write a lot of partitions

[jira] [Commented] (SPARK-18137) RewriteDistinctAggregates UnresolvedException when a UDAF has a foldable TypeCheck

2016-11-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15648357#comment-15648357 ] Herman van Hovell commented on SPARK-18137: --- Thanks! > RewriteDistinctAggregates

[jira] [Commented] (SPARK-18137) RewriteDistinctAggregates UnresolvedException when a UDAF has a foldable TypeCheck

2016-11-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15648291#comment-15648291 ] Herman van Hovell commented on SPARK-18137: --- [~srowen] How did you set the Assignee? I spend a

[jira] [Commented] (SPARK-18350) Support session local timezone

2016-11-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15647753#comment-15647753 ] Herman van Hovell commented on SPARK-18350: --- I am totally onboard with doing this. I think we

[jira] [Resolved] (SPARK-17868) Do not use bitmasks during parsing and analysis of CUBE/ROLLUP/GROUPING SETS

2016-11-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17868. --- Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.2.0 > Do

[jira] [Resolved] (SPARK-18137) RewriteDistinctAggregates UnresolvedException when a UDAF has a foldable TypeCheck

2016-11-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18137. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.3 >

[jira] [Resolved] (SPARK-18207) class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection" grows beyond 64 KB

2016-11-08 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18207. --- Resolution: Fixed Assignee: Kazuaki Ishizaki Fix Version/s: 2.1.0 >

[jira] [Resolved] (SPARK-17490) Optimize SerializeFromObject for primitive array

2016-11-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17490. --- Resolution: Fixed Assignee: Kazuaki Ishizaki Fix Version/s: 2.1.0 >

[jira] [Resolved] (SPARK-17108) BIGINT and INT comparison failure in spark sql

2016-11-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17108. --- Resolution: Fixed Assignee: Weiqing Yang Fix Version/s: 2.1.0 >

[jira] [Updated] (SPARK-18125) Spark generated code causes CompileException when groupByKey, reduceGroups and map(_._2) are used

2016-11-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-18125: -- Fix Version/s: 2.0.3 > Spark generated code causes CompileException when groupByKey,

[jira] [Resolved] (SPARK-18125) Spark generated code causes CompileException when groupByKey, reduceGroups and map(_._2) are used

2016-11-07 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18125. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.1.0 >

[jira] [Resolved] (SPARK-17849) grouping set throws NPE

2016-11-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17849. --- Resolution: Fixed Assignee: Yang Wang Fix Version/s: 2.1.0

[jira] [Updated] (SPARK-17693) Fixed Insert Failure To Data Source Tables when the Schema has the Comment Field

2016-11-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17693: -- Fix Version/s: 2.0.3 > Fixed Insert Failure To Data Source Tables when the Schema has

[jira] [Updated] (SPARK-17981) Incorrectly Set Nullability to False in FilterExec

2016-11-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17981: -- Fix Version/s: (was: 2.0.2) 2.0.3 > Incorrectly Set Nullability

[jira] [Updated] (SPARK-17981) Incorrectly Set Nullability to False in FilterExec

2016-11-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17981: -- Fix Version/s: 2.0.2 > Incorrectly Set Nullability to False in FilterExec >

[jira] [Resolved] (SPARK-18287) Move hash expressions from misc.scala into hash.scala

2016-11-05 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-18287. --- Resolution: Fixed Fix Version/s: 2.1.0 > Move hash expressions from

[jira] [Assigned] (SPARK-17337) Incomplete algorithm for name resolution in Catalyst paser may lead to incorrect result

2016-11-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-17337: - Assignee: Herman van Hovell > Incomplete algorithm for name resolution in

[jira] [Resolved] (SPARK-17337) Incomplete algorithm for name resolution in Catalyst paser may lead to incorrect result

2016-11-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17337. --- Resolution: Fixed Fix Version/s: 2.1.0 > Incomplete algorithm for name

[jira] [Commented] (SPARK-17337) Incomplete algorithm for name resolution in Catalyst paser may lead to incorrect result

2016-11-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15637167#comment-15637167 ] Herman van Hovell commented on SPARK-17337: --- [~nsyca] You are right to say that this is part of

[jira] [Updated] (SPARK-17337) Incomplete algorithm for name resolution in Catalyst paser may lead to incorrect result

2016-11-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17337: -- Labels: correctness (was: ) > Incomplete algorithm for name resolution in Catalyst

[jira] [Commented] (SPARK-17348) Incorrect results from subquery transformation

2016-11-04 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15637099#comment-15637099 ] Herman van Hovell commented on SPARK-17348: --- Yeah, it would be nice if you can consolidate the

[jira] [Commented] (SPARK-18254) UDFs don't see aliased column names

2016-11-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15634432#comment-15634432 ] Herman van Hovell commented on SPARK-18254: --- We 'accidentally' fixed this yesterday with

[jira] [Created] (SPARK-18259) QueryExecution should not catch Throwable

2016-11-03 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-18259: - Summary: QueryExecution should not catch Throwable Key: SPARK-18259 URL: https://issues.apache.org/jira/browse/SPARK-18259 Project: Spark Issue

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-11-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633509#comment-15633509 ] Herman van Hovell commented on SPARK-15822: --- Could you share your plan? Are you using caching?

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-11-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633480#comment-15633480 ] Herman van Hovell commented on SPARK-15822: --- What are you seeing? The InMemory cache issue or

[jira] [Commented] (SPARK-18254) UDFs don't see aliased column names

2016-11-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633436#comment-15633436 ] Herman van Hovell commented on SPARK-18254: --- Oh, BatchEvalPython is pushed below the

[jira] [Commented] (SPARK-18254) UDFs don't see aliased column names

2016-11-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633429#comment-15633429 ] Herman van Hovell commented on SPARK-18254: --- So FIRST and LAST work? > UDFs don't see aliased

[jira] [Comment Edited] (SPARK-18254) UDFs don't see aliased column names

2016-11-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633429#comment-15633429 ] Herman van Hovell edited comment on SPARK-18254 at 11/3/16 4:50 PM:

[jira] [Commented] (SPARK-18254) UDFs don't see aliased column names

2016-11-03 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633379#comment-15633379 ] Herman van Hovell commented on SPARK-18254: --- How can you take the length of a row? Do you

<    2   3   4   5   6   7   8   9   10   11   >