[jira] [Created] (SPARK-13557) Migrate gather-to-driver actions

2016-02-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13557: -- Summary: Migrate gather-to-driver actions Key: SPARK-13557 URL: https://issues.apache.org/jira/browse/SPARK-13557 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-13556) Migrate untyped joins

2016-02-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13556: -- Summary: Migrate untyped joins Key: SPARK-13556 URL: https://issues.apache.org/jira/browse/SPARK-13556 Project: Spark Issue Type: Sub-task Components:

[jira] [Created] (SPARK-13555) Migrate untyped relational operations

2016-02-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13555: -- Summary: Migrate untyped relational operations Key: SPARK-13555 URL: https://issues.apache.org/jira/browse/SPARK-13555 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-13554) Migrate typed relational operations

2016-02-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13554: -- Summary: Migrate typed relational operations Key: SPARK-13554 URL: https://issues.apache.org/jira/browse/SPARK-13554 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-13553) Migrate basic inspection operations

2016-02-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13553: -- Summary: Migrate basic inspection operations Key: SPARK-13553 URL: https://issues.apache.org/jira/browse/SPARK-13553 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-13540) Nested classes within Scala objects can't be used as Dataset element type

2016-02-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13540: --- Summary: Nested classes within Scala objects can't be used as Dataset element type (was: Scala

[jira] [Updated] (SPARK-13540) Scala object can't be added to OuterScopes

2016-02-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13540: --- Description: Add the following test case to {{DatasetSuite}} to reproduce this issue: {code} object

[jira] [Updated] (SPARK-13540) Scala object can't be added to OuterScopes

2016-02-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13540: --- Assignee: Cheng Lian Affects Version/s: 2.0.0 Target Version/s: 2.0.0

[jira] [Created] (SPARK-13540) Scala object can't be added to OuterScopes

2016-02-28 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13540: -- Summary: Scala object can't be added to OuterScopes Key: SPARK-13540 URL: https://issues.apache.org/jira/browse/SPARK-13540 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-13457) Remove DataFrame RDD operations

2016-02-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13457. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11388

[jira] [Resolved] (SPARK-13486) Move SQLConf into an internal package

2016-02-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13486. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11363

[jira] [Updated] (SPARK-13473) Predicate can't be pushed through project with nondeterministic field

2016-02-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13473: --- Assignee: Cheng Lian Affects Version/s: 2.0.0 1.5.2

[jira] [Created] (SPARK-13473) Predicate can't be pushed through project with nondeterministic field

2016-02-24 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13473: -- Summary: Predicate can't be pushed through project with nondeterministic field Key: SPARK-13473 URL: https://issues.apache.org/jira/browse/SPARK-13473 Project: Spark

[jira] [Created] (SPARK-13457) Remove DataFrame RDD operations

2016-02-23 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13457: -- Summary: Remove DataFrame RDD operations Key: SPARK-13457 URL: https://issues.apache.org/jira/browse/SPARK-13457 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-13456) Cannot create encoders for case classes defined in Spark shell after upgrading to Scala 2.11

2016-02-23 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13456: -- Summary: Cannot create encoders for case classes defined in Spark shell after upgrading to Scala 2.11 Key: SPARK-13456 URL: https://issues.apache.org/jira/browse/SPARK-13456

[jira] [Commented] (SPARK-13454) Cannot drop table whose name starts with underscore

2016-02-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158930#comment-15158930 ] Cheng Lian commented on SPARK-13454: cc [~yhuai] > Cannot drop table whose name starts with

[jira] [Created] (SPARK-13454) Cannot drop table whose name starts with underscore

2016-02-23 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13454: -- Summary: Cannot drop table whose name starts with underscore Key: SPARK-13454 URL: https://issues.apache.org/jira/browse/SPARK-13454 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-13263) SQL generation support for tablesample

2016-02-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13263. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11148

[jira] [Updated] (SPARK-13263) SQL generation support for tablesample

2016-02-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13263: --- Assignee: Xiao Li > SQL generation support for tablesample > --

[jira] [Resolved] (SPARK-13236) SQL generation support for Set Operations

2016-02-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13236. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11195

[jira] [Updated] (SPARK-13236) SQL generation support for Set Operations

2016-02-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13236: --- Assignee: Xiao Li > SQL generation support for Set Operations >

[jira] [Commented] (SPARK-13418) SQL generation for uncorrelated scalar subqueries

2016-02-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156534#comment-15156534 ] Cheng Lian commented on SPARK-13418: Summary of offline discussion with [~rxin]: Moving {{.sql}} to

[jira] [Resolved] (SPARK-12799) Simplify various string output for expressions

2016-02-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12799. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10757

[jira] [Created] (SPARK-13370) Lexer not handling whitespaces properly

2016-02-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13370: -- Summary: Lexer not handling whitespaces properly Key: SPARK-13370 URL: https://issues.apache.org/jira/browse/SPARK-13370 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-13314) Malformed WholeStageCodegen tree string

2016-02-14 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13314. Resolution: Not A Problem I didn't realize that the extra vertical lines are used to indicate

[jira] [Updated] (SPARK-12720) SQL generation support for cube, rollup, and grouping set

2016-02-14 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12720: --- Assignee: Xiao Li > SQL generation support for cube, rollup, and grouping set >

[jira] [Created] (SPARK-13314) Malformed WholeStageCodegen tree string

2016-02-14 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13314: -- Summary: Malformed WholeStageCodegen tree string Key: SPARK-13314 URL: https://issues.apache.org/jira/browse/SPARK-13314 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-13314) Malformed WholeStageCodegen tree string

2016-02-14 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-13314: -- Assignee: Cheng Lian > Malformed WholeStageCodegen tree string >

[jira] [Resolved] (SPARK-12725) SQL generation suffers from name conficts introduced by some analysis rules

2016-02-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12725. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11050

[jira] [Updated] (SPARK-13205) SQL generation support for self join

2016-02-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13205: --- Assignee: Xiao Li > SQL generation support for self join > > >

[jira] [Resolved] (SPARK-13205) SQL generation support for self join

2016-02-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13205. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11084

[jira] [Created] (SPARK-13244) Unify DataFrame and Dataset API

2016-02-09 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13244: -- Summary: Unify DataFrame and Dataset API Key: SPARK-13244 URL: https://issues.apache.org/jira/browse/SPARK-13244 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-13157) ADD JAR command cannot handle path with @ character

2016-02-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13157: --- Assignee: Herman van Hovell > ADD JAR command cannot handle path with @ character >

[jira] [Commented] (SPARK-13157) ADD JAR command cannot handle path with @ character

2016-02-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130754#comment-15130754 ] Cheng Lian commented on SPARK-13157: Thanks for the explanation. > ADD JAR command cannot handle

[jira] [Commented] (SPARK-13150) Flaky test: org.apache.spark.sql.hive.thriftserver.SingleSessionSuite.test single session

2016-02-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130760#comment-15130760 ] Cheng Lian commented on SPARK-13150: Both flaky tests seem to be caused by SPARK-13157. Ideally,

[jira] [Commented] (SPARK-13157) ADD JAR command cannot handle path with @ character

2016-02-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130738#comment-15130738 ] Cheng Lian commented on SPARK-13157: Thanks [~hvanhovell]! Just curious, is this a bug of the

[jira] [Commented] (SPARK-13150) Flaky test: org.apache.spark.sql.hive.thriftserver.SingleSessionSuite.test single session

2016-02-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15129556#comment-15129556 ] Cheng Lian commented on SPARK-13150: Seems that ADD JAR command in both flaky tests may fail silently

[jira] [Commented] (SPARK-13150) Flaky test: org.apache.spark.sql.hive.thriftserver.SingleSessionSuite.test single session

2016-02-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15129857#comment-15129857 ] Cheng Lian commented on SPARK-13150: Please refer to [this PR

[jira] [Created] (SPARK-13157) ADD JAR command cannot handle path with @ character

2016-02-02 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13157: -- Summary: ADD JAR command cannot handle path with @ character Key: SPARK-13157 URL: https://issues.apache.org/jira/browse/SPARK-13157 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-12725) SQL generation suffers from name conficts introduced by some analysis rules

2016-02-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-12725: -- Assignee: Xiao Li > SQL generation suffers from name conficts introduced by some analysis

[jira] [Commented] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127021#comment-15127021 ] Cheng Lian commented on SPARK-13101: [~deenar] Just had an offline discussion with [~marmbrus] and

[jira] [Commented] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126890#comment-15126890 ] Cheng Lian commented on SPARK-13101: I think this is expected behavior. The problem in snippet

[jira] [Comment Edited] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127021#comment-15127021 ] Cheng Lian edited comment on SPARK-13101 at 2/1/16 9:03 PM: [~deenar] Just

[jira] [Comment Edited] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127021#comment-15127021 ] Cheng Lian edited comment on SPARK-13101 at 2/1/16 9:02 PM: [~deenar] Just

[jira] [Commented] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126894#comment-15126894 ] Cheng Lian commented on SPARK-13101: [~deenar] I also tried the last two snippets and they both work.

[jira] [Commented] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126932#comment-15126932 ] Cheng Lian commented on SPARK-13101: The reason why 1.6.0 allows this illegal situation is that we

[jira] [Comment Edited] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126894#comment-15126894 ] Cheng Lian edited comment on SPARK-13101 at 2/1/16 7:45 PM: [~deenar] I also

[jira] [Comment Edited] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126890#comment-15126890 ] Cheng Lian edited comment on SPARK-13101 at 2/1/16 7:55 PM: I think this is

[jira] [Assigned] (SPARK-12718) SQL generation support for window functions

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-12718: -- Assignee: Xiao Li > SQL generation support for window functions >

[jira] [Commented] (PARQUET-401) Deprecate Log and move to SLF4J Logger

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127668#comment-15127668 ] Cheng Lian commented on PARQUET-401: Fix of this issue is nice to have but probably shouldn't block

[jira] [Commented] (SPARK-6319) Should throw analysis exception when using binary type in groupby/join

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126668#comment-15126668 ] Cheng Lian commented on SPARK-6319: --- One possible but not necessarily the best workaround is to have a

[jira] [Resolved] (PARQUET-495) Fix mismatches in Types class comments

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved PARQUET-495. Resolution: Fixed Issue resolved by pull request 317

[jira] [Updated] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13101: --- Description: There seems to be a regression between 1.6.0 and 1.6.1 (snapshot build). By default a

[jira] [Commented] (SPARK-12725) SQL generation suffers from name conficts introduced by some analysis rules

2016-01-31 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125587#comment-15125587 ] Cheng Lian commented on SPARK-12725: There are other analysis rules that may use generated attributes

[jira] [Updated] (SPARK-12725) SQL generation suffers from name conficts introduced by some analysis rules

2016-01-31 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12725: --- Description: Some analysis rules generate auxiliary attribute references with the same name but

[jira] [Commented] (SPARK-12624) When schema is specified, we should give better error message if actual row length doesn't match

2016-01-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15124556#comment-15124556 ] Cheng Lian commented on SPARK-12624: Yes, it should. > When schema is specified, we should give

[jira] [Resolved] (PARQUET-432) Complete a todo for method ColumnDescriptor.compareTo()

2016-01-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved PARQUET-432. Resolution: Fixed Issue resolved by pull request 314

[jira] [Resolved] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13050. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10954

[jira] [Commented] (SPARK-12725) SQL generation suffers from name conficts introduced by some analysis rules

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122013#comment-15122013 ] Cheng Lian commented on SPARK-12725: One possible solution I was thinking about is that we can add a

[jira] [Commented] (SPARK-12723) Comprehensive SQL generation support for expressions

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122102#comment-15122102 ] Cheng Lian commented on SPARK-12723: Most of this issue is probably fixed in SPARK-12799 and PR

[jira] [Commented] (SPARK-12727) SQL generation support for distinct aggregation patterns that fit DistinctAggregationRewriter analysis rule

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122124#comment-15122124 ] Cheng Lian commented on SPARK-12727: It would be nice if we can recover the original distinct

[jira] [Updated] (SPARK-11012) Canonicalize view definitions

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11012: --- Description: In SPARK-10337, we added the first step of supporting view natively, which is

[jira] [Updated] (SPARK-12719) SQL generation support for generators (including UDTF)

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12719: --- Description: {{HiveCompatibilitySuite}} can be useful for bootstrapping test coverage. Please refer

[jira] [Updated] (SPARK-12721) SQL generation support for script transformation

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12721: --- Description: {{HiveCompatibilitySuite}} can be useful for bootstrapping test coverage. Please refer

[jira] [Updated] (SPARK-12720) SQL generation support for cube, rollup, and grouping set

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12720: --- Description: {{HiveCompatibilitySuite}} can be useful for bootstrapping test coverage. Please refer

[jira] [Updated] (SPARK-12718) SQL generation support for window functions

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12718: --- Description: {{HiveWindowFunctionQuerySuite}} and {{HiveWindowFunctionQueryFileSuite}} can be useful

[jira] [Resolved] (SPARK-12401) Add support for enums in postgres

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12401. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10596

[jira] [Updated] (SPARK-12401) Add support for enums in postgres

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12401: --- Assignee: Takeshi Yamamuro > Add support for enums in postgres > - >

[jira] [Updated] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13050: --- Description: Builds fail at the new sketch module when a scalatest tag is used. Found when using

[jira] [Updated] (SPARK-11955) Mark one side fields in merging schema for safely pushdowning filters in parquet

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11955: --- Assignee: Liang-Chi Hsieh > Mark one side fields in merging schema for safely pushdowning filters in

[jira] [Resolved] (SPARK-11955) Mark one side fields in merging schema for safely pushdowning filters in parquet

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11955. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 9940

[jira] [Commented] (SPARK-12725) SQL generation suffers from name conficts introduced by some analysis rules

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122043#comment-15122043 ] Cheng Lian commented on SPARK-12725: Thanks, this also sounds good to me. Will try this approach

[jira] [Updated] (SPARK-11012) Canonicalize view definitions

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11012: --- Description: In SPARK-10337, we added the first step of supporting view natively, which is

[jira] [Updated] (SPARK-13050) Scalatest tags fail builds with the addition of the sketch module

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13050: --- Assignee: Alex Bozarth > Scalatest tags fail builds with the addition of the sketch module >

[jira] [Created] (SPARK-13070) Points which physical file is the trouble maker when Parquet schema merging fails

2016-01-28 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-13070: -- Summary: Points which physical file is the trouble maker when Parquet schema merging fails Key: SPARK-13070 URL: https://issues.apache.org/jira/browse/SPARK-13070

[jira] [Updated] (SPARK-13070) Points out which physical file is the trouble maker when Parquet schema merging fails

2016-01-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13070: --- Summary: Points out which physical file is the trouble maker when Parquet schema merging fails

[jira] [Updated] (SPARK-12624) When schema is specified, we should give better error message if actual row length doesn't match

2016-01-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12624: --- Summary: When schema is specified, we should give better error message if actual row length doesn't

[jira] [Commented] (SPARK-12624) When schema is specified, we should give better error message if actual row length doesn't match

2016-01-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15114123#comment-15114123 ] Cheng Lian commented on SPARK-12624: Quoted Davies' offline comment {quote} We always raise a

[jira] [Updated] (SPARK-12624) When schema is specified, we should give better error message if actual row length doesn't match

2016-01-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12624: --- Description: The following code snippet reproduces this issue: {code} from pyspark.sql.types import

[jira] [Issue Comment Deleted] (SPARK-12818) Implement Bloom filter and count-min sketch in DataFrames

2016-01-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12818: --- Comment: was deleted (was: User 'liancheng' has created a pull request for this issue:

[jira] [Updated] (SPARK-12938) Bloom filter DataFrame API integration

2016-01-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12938: --- Assignee: Wenchen Fan (was: Cheng Lian) > Bloom filter DataFrame API integration >

[jira] [Updated] (SPARK-12936) Initial bloom filter implementation

2016-01-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12936: --- Assignee: Wenchen Fan (was: Cheng Lian) > Initial bloom filter implementation >

[jira] [Created] (SPARK-12935) Count-min sketch DataFrame API integration

2016-01-20 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-12935: -- Summary: Count-min sketch DataFrame API integration Key: SPARK-12935 URL: https://issues.apache.org/jira/browse/SPARK-12935 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-12933) Initial count-min sketch implementation

2016-01-20 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-12933: -- Summary: Initial count-min sketch implementation Key: SPARK-12933 URL: https://issues.apache.org/jira/browse/SPARK-12933 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-12934) Count-min sketch serialization

2016-01-20 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-12934: -- Summary: Count-min sketch serialization Key: SPARK-12934 URL: https://issues.apache.org/jira/browse/SPARK-12934 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-12938) Bloom filter DataFrame API integration

2016-01-20 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-12938: -- Summary: Bloom filter DataFrame API integration Key: SPARK-12938 URL: https://issues.apache.org/jira/browse/SPARK-12938 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-12937) Bloom filter serialization

2016-01-20 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-12937: -- Summary: Bloom filter serialization Key: SPARK-12937 URL: https://issues.apache.org/jira/browse/SPARK-12937 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-12936) Initial bloom filter implementation

2016-01-20 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-12936: -- Summary: Initial bloom filter implementation Key: SPARK-12936 URL: https://issues.apache.org/jira/browse/SPARK-12936 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-12937) Bloom filter serialization

2016-01-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12937: --- Assignee: Wenchen Fan (was: Cheng Lian) > Bloom filter serialization > -- >

[jira] [Resolved] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2016-01-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12560. Resolution: Fixed Assignee: Imran Rashid Resolved by

[jira] [Updated] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2016-01-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12560: --- Fix Version/s: 2.0.0 > SqlTestUtils.stripSparkFilter needs to copy utf8strings >

[jira] [Resolved] (SPARK-12867) Nullability of Intersect can be stricter

2016-01-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12867. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10812

[jira] [Commented] (SPARK-12867) Nullability of Intersect can be stricter

2016-01-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105616#comment-15105616 ] Cheng Lian commented on SPARK-12867: Thanks for helping! Go ahead, please. I'm assigning this to you.

[jira] [Updated] (SPARK-12867) Nullability of Intersect can be stricter

2016-01-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12867: --- Assignee: Xiao Li > Nullability of Intersect can be stricter >

[jira] [Created] (SPARK-12867) Nullability of Intersect can be stricter

2016-01-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-12867: -- Summary: Nullability of Intersect can be stricter Key: SPARK-12867 URL: https://issues.apache.org/jira/browse/SPARK-12867 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore

2016-01-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094471#comment-15094471 ] Cheng Lian commented on SPARK-12403: Also, could you please provide the exact version number of the

[jira] [Assigned] (SPARK-12724) SQL generation support for persisted data source relations

2016-01-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-12724: -- Assignee: Cheng Lian > SQL generation support for persisted data source relations >

[jira] [Resolved] (SPARK-12724) SQL generation support for persisted data source relations

2016-01-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-12724. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10712

[jira] [Commented] (SPARK-12403) "Simba Spark ODBC Driver 1.0" not working with 1.5.2 anymore

2016-01-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094466#comment-15094466 ] Cheng Lian commented on SPARK-12403: Hi [~lunendl], I wonder what kind of Hive metastore were you

[jira] [Updated] (SPARK-12742) org.apache.spark.sql.hive.LogicalPlanToSQLSuite failure due to Table already exists

2016-01-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-12742: --- Assignee: Fei Wang > org.apache.spark.sql.hive.LogicalPlanToSQLSuite failure due to Table already >

<    2   3   4   5   6   7   8   9   10   11   >