[jira] [Resolved] (SPARK-2970) spark-sql script ends with IOException when EventLogging is enabled

2014-08-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-2970. --- Resolution: Fixed spark-sql script ends with IOException when EventLogging is enabled

[jira] [Created] (SPARK-3294) Avoid boxing/unboxing when handling in-memory columnar storage

2014-08-28 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3294: - Summary: Avoid boxing/unboxing when handling in-memory columnar storage Key: SPARK-3294 URL: https://issues.apache.org/jira/browse/SPARK-3294 Project: Spark

[jira] [Commented] (SPARK-3294) Avoid boxing/unboxing when handling in-memory columnar storage

2014-08-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114836#comment-14114836 ] Cheng Lian commented on SPARK-3294: --- [~marmbrus], would you mind to assign this issue to

[jira] [Updated] (SPARK-3234) SPARK_HADOOP_VERSION and SPARK_HIVE depend on deprecated make-distribution.sh command line options

2014-08-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3234: -- Summary: SPARK_HADOOP_VERSION and SPARK_HIVE depend on deprecated make-distribution.sh command line

[jira] [Updated] (SPARK-3234) SPARK_HADOOP_VERSION and SPARK_HIVE depend on deprecated make-distribution.sh command line options

2014-08-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3234: -- Description: In {{make-distribution.sh}}, {{SPARK_HADOOP_VERSION}} is determined by {{\-\-hadoop}},

[jira] [Updated] (SPARK-3234) SPARK_HADOOP_VERSION and SPARK_HIVE depend on deprecated make-distribution.sh command line options

2014-08-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3234: -- Description: In {{make-distribution.sh}}, {{SPARK_HADOOP_VERSION}} is determined by {{\-\-hadoop}},

[jira] [Updated] (SPARK-3234) SPARK_HADOOP_VERSION and SPARK_HIVE depend on deprecated make-distribution.sh command line options

2014-08-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3234: -- Description: In {{make-distribution.sh}}, {{SPARK_HADOOP_VERSION}} is determined by {{\-\-hadoop}},

[jira] [Created] (SPARK-3320) Batched in-memory column buffer building doesn't work for SchemaRDDs with empty partitions

2014-08-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3320: - Summary: Batched in-memory column buffer building doesn't work for SchemaRDDs with empty partitions Key: SPARK-3320 URL: https://issues.apache.org/jira/browse/SPARK-3320

[jira] [Updated] (SPARK-3324) YARN module has nonstandard structure which cause compile error In IntelliJ

2014-08-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3324: -- Description: The YARN module has nonstandard path structure like: {code} ${SPARK_HOME} |--yarn

[jira] [Created] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with upper case letter in their names

2014-09-05 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3414: - Summary: Case insensitivity breaks when unresolved relation contains attributes with upper case letter in their names Key: SPARK-3414 URL:

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with upper case letter in their names

2014-09-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Description: Paste the following snippet to {{spark-shell}} (need Hive support) to reproduce this

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with upper case letter in their names

2014-09-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Description: Paste the following snippet to {{spark-shell}} (need Hive support) to reproduce this

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with uppercase letters in their names

2014-09-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Summary: Case insensitivity breaks when unresolved relation contains attributes with uppercase letters

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with upper case letter in their names

2014-09-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Description: Paste the following snippet to {{spark-shell}} (need Hive support) to reproduce this

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with uppercase letters in their names

2014-09-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Description: Paste the following snippet to {{spark-shell}} (need Hive support) to reproduce this

[jira] [Updated] (SPARK-3414) Case insensitivity breaks when unresolved relation contains attributes with uppercase letters in their names

2014-09-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3414: -- Description: Paste the following snippet to {{spark-shell}} (need Hive support) to reproduce this

[jira] [Created] (SPARK-3421) StructField.toString should quote the name field to allow arbitrary character as struct field name

2014-09-05 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3421: - Summary: StructField.toString should quote the name field to allow arbitrary character as struct field name Key: SPARK-3421 URL: https://issues.apache.org/jira/browse/SPARK-3421

[jira] [Commented] (SPARK-2537) Workaround Timezone specific Hive tests

2014-09-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123556#comment-14123556 ] Cheng Lian commented on SPARK-2537: --- PR

[jira] [Resolved] (SPARK-2537) Workaround Timezone specific Hive tests

2014-09-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-2537. --- Resolution: Fixed Fix Version/s: 1.1.0 Target Version/s: 1.1.0 Workaround Timezone

[jira] [Created] (SPARK-3448) SpecificMutableRow.update doesn't check for null

2014-09-08 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3448: - Summary: SpecificMutableRow.update doesn't check for null Key: SPARK-3448 URL: https://issues.apache.org/jira/browse/SPARK-3448 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3515) ParquetMetastoreSuite fails when executed together with other suites under Maven

2014-09-12 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3515: - Summary: ParquetMetastoreSuite fails when executed together with other suites under Maven Key: SPARK-3515 URL: https://issues.apache.org/jira/browse/SPARK-3515 Project:

[jira] [Commented] (SPARK-3515) ParquetMetastoreSuite fails when executed together with other suites under Maven

2014-09-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14132406#comment-14132406 ] Cheng Lian commented on SPARK-3515: --- The bug SPARK-3481 fixed actually covered up the

[jira] [Created] (SPARK-3552) Thrift server doesn't reset current database for each connection

2014-09-16 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3552: - Summary: Thrift server doesn't reset current database for each connection Key: SPARK-3552 URL: https://issues.apache.org/jira/browse/SPARK-3552 Project: Spark

[jira] [Created] (SPARK-3609) Add sizeInBytes statistics to Limit operator

2014-09-19 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3609: - Summary: Add sizeInBytes statistics to Limit operator Key: SPARK-3609 URL: https://issues.apache.org/jira/browse/SPARK-3609 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3654) Implement all extended HiveQL statements/commands with a separate parser combinator

2014-09-22 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3654: - Summary: Implement all extended HiveQL statements/commands with a separate parser combinator Key: SPARK-3654 URL: https://issues.apache.org/jira/browse/SPARK-3654 Project:

[jira] [Created] (SPARK-3713) Use JSON to serialize DataType

2014-09-28 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3713: - Summary: Use JSON to serialize DataType Key: SPARK-3713 URL: https://issues.apache.org/jira/browse/SPARK-3713 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-3713) Use JSON to serialize DataType

2014-09-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3713: -- Issue Type: Improvement (was: Bug) Use JSON to serialize DataType --

[jira] [Created] (SPARK-3738) InsertIntoHiveTable can't handle strings with \n

2014-09-29 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3738: - Summary: InsertIntoHiveTable can't handle strings with \n Key: SPARK-3738 URL: https://issues.apache.org/jira/browse/SPARK-3738 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3738) InsertIntoHiveTable can't handle strings with \n

2014-09-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152765#comment-14152765 ] Cheng Lian commented on SPARK-3738: --- False alarm... it's because of Hive's default SerDe

[jira] [Closed] (SPARK-3738) InsertIntoHiveTable can't handle strings with \n

2014-09-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian closed SPARK-3738. - Resolution: Invalid False alarm, it's because of Hive's default SerDe, which uses '\n' as record

[jira] [Created] (SPARK-3791) HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call

2014-10-04 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3791: - Summary: HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call Key: SPARK-3791 URL: https://issues.apache.org/jira/browse/SPARK-3791 Project: Spark Issue Type:

[jira] [Updated] (SPARK-3791) HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call

2014-10-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3791: -- Target Version/s: 1.1.1 (was: 1.2.0) HiveThriftServer2 returns 0.12.0 to ODBC SQLGetInfo call

[jira] [Created] (SPARK-3810) Rule PreInsertionCasts doesn't handle partitioned table properly

2014-10-06 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3810: - Summary: Rule PreInsertionCasts doesn't handle partitioned table properly Key: SPARK-3810 URL: https://issues.apache.org/jira/browse/SPARK-3810 Project: Spark

[jira] [Commented] (SPARK-3810) Rule PreInsertionCasts doesn't handle partitioned table properly

2014-10-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160151#comment-14160151 ] Cheng Lian commented on SPARK-3810: --- This issue is marked as MINOR because it doesn't

[jira] [Comment Edited] (SPARK-3810) Rule PreInsertionCasts doesn't handle partitioned table properly

2014-10-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160151#comment-14160151 ] Cheng Lian edited comment on SPARK-3810 at 10/6/14 10:26 AM: -

[jira] [Resolved] (SPARK-3421) StructField.toString should quote the name field to allow arbitrary character as struct field name

2014-10-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-3421. --- Resolution: Fixed StructField.toString should quote the name field to allow arbitrary character as

[jira] [Commented] (SPARK-3892) Map type should have typeName

2014-10-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166812#comment-14166812 ] Cheng Lian commented on SPARK-3892: --- Actually {{MapType.simpleName}} can be simply

[jira] [Commented] (SPARK-3892) Map type should have typeName

2014-10-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166822#comment-14166822 ] Cheng Lian commented on SPARK-3892: --- [~adrian-wang] You're right, it's a typo. So would

[jira] [Commented] (SPARK-3892) Map type do not need simpleName

2014-10-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166848#comment-14166848 ] Cheng Lian commented on SPARK-3892: --- Ah, while working on the {{DataType}} JSON ser/de

[jira] [Commented] (SPARK-3892) Map type do not need simpleName

2014-10-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166886#comment-14166886 ] Cheng Lian commented on SPARK-3892: --- Please see my comments in the PR :) Map type do

[jira] [Created] (SPARK-3919) HiveThriftServer2 fails to start because of Hive 0.12 metastore schema verification failure

2014-10-12 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-3919: - Summary: HiveThriftServer2 fails to start because of Hive 0.12 metastore schema verification failure Key: SPARK-3919 URL: https://issues.apache.org/jira/browse/SPARK-3919

[jira] [Updated] (SPARK-3919) HiveThriftServer2 fails to start because of Hive 0.12 metastore schema verification failure

2014-10-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-3919: -- Description: When using MySQL backed Metastore with {{hive.metastore.schema.verification}} set to

[jira] [Commented] (SPARK-3919) HiveThriftServer2 fails to start because of Hive 0.12 metastore schema verification failure

2014-10-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14168664#comment-14168664 ] Cheng Lian commented on SPARK-3919: --- [~pwendell] Hive Metastore schema verification

[jira] [Commented] (SPARK-3938) Set RDD name to table name during cache operations

2014-10-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170454#comment-14170454 ] Cheng Lian commented on SPARK-3938: --- A problem here is that after PR

[jira] [Created] (SPARK-4000) Gathers unit tests logs to Jenkins master at the end of a Jenkins build

2014-10-19 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4000: - Summary: Gathers unit tests logs to Jenkins master at the end of a Jenkins build Key: SPARK-4000 URL: https://issues.apache.org/jira/browse/SPARK-4000 Project: Spark

[jira] [Updated] (SPARK-4000) Gathers unit tests logs to Jenkins master at the end of a Jenkins build

2014-10-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4000: -- Description: Unit tests logs can be useful for debugging Jenkins failures. Currently these logs are

[jira] [Commented] (SPARK-4037) NPE in JDBC server when calling SET

2014-10-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179305#comment-14179305 ] Cheng Lian commented on SPARK-4037: --- This is a regression of SPARK-2814, added in

[jira] [Commented] (SPARK-4037) NPE in JDBC server when calling SET

2014-10-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179598#comment-14179598 ] Cheng Lian commented on SPARK-4037: --- I think we can safely remove the global singleton

[jira] [Closed] (SPARK-3939) NPE caused by SessionState.out not set in thriftserver2

2014-10-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian closed SPARK-3939. - Resolution: Duplicate NPE caused by SessionState.out not set in thriftserver2

[jira] [Commented] (SPARK-3939) NPE caused by SessionState.out not set in thriftserver2

2014-10-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179749#comment-14179749 ] Cheng Lian commented on SPARK-3939: --- Ah, actually it's SPARK-4037 who duplicates this

[jira] [Created] (SPARK-4091) Occasionally spark.local.dir can be deleted twice and causes test failure

2014-10-26 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4091: - Summary: Occasionally spark.local.dir can be deleted twice and causes test failure Key: SPARK-4091 URL: https://issues.apache.org/jira/browse/SPARK-4091 Project: Spark

[jira] [Updated] (SPARK-4091) Occasionally spark.local.dir can be deleted twice and causes test failure

2014-10-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4091: -- Description: By persisting an arbitrary RDD with storage level {{MEMORY_AND_DISK}}, Spark may

[jira] [Commented] (SPARK-4091) Occasionally spark.local.dir can be deleted twice and causes test failure

2014-10-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14184741#comment-14184741 ] Cheng Lian commented on SPARK-4091: --- Yes, thanks [~joshrosen], closing this.

[jira] [Closed] (SPARK-4091) Occasionally spark.local.dir can be deleted twice and causes test failure

2014-10-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian closed SPARK-4091. - Resolution: Duplicate Occasionally spark.local.dir can be deleted twice and causes test failure

[jira] [Created] (SPARK-4119) Don't rely on HIVE_DEV_HOME to find .q files

2014-10-28 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4119: - Summary: Don't rely on HIVE_DEV_HOME to find .q files Key: SPARK-4119 URL: https://issues.apache.org/jira/browse/SPARK-4119 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-3683) PySpark Hive query generates NULL instead of None

2014-10-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14187757#comment-14187757 ] Cheng Lian commented on SPARK-3683: --- [~davies] I removed this special case for NULL

[jira] [Commented] (SPARK-3683) PySpark Hive query generates NULL instead of None

2014-10-29 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188194#comment-14188194 ] Cheng Lian commented on SPARK-3683: --- [~jamborta] Your concern is legitimate. However,

[jira] [Commented] (SPARK-2220) Fix remaining Hive Commands

2014-10-31 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191841#comment-14191841 ] Cheng Lian commented on SPARK-2220: --- It turned out that {{ShellCommand}} and

[jira] [Created] (SPARK-4182) Caching tables containing boolean, binary, array, struct and/or map columns doesn't work

2014-11-01 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4182: - Summary: Caching tables containing boolean, binary, array, struct and/or map columns doesn't work Key: SPARK-4182 URL: https://issues.apache.org/jira/browse/SPARK-4182

[jira] [Updated] (SPARK-4182) Caching tables containing boolean, binary, array, struct and/or map columns doesn't work

2014-11-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4182: -- Description: If a table contains a column whose type is binary, array, struct, map, and for some

[jira] [Updated] (SPARK-4182) Caching tables containing boolean, binary, array, struct and/or map columns doesn't work

2014-11-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4182: -- Description: If a table contains a column whose type is binary, array, struct, map, and for some

[jira] [Updated] (SPARK-4182) Caching tables containing boolean, binary, array, struct and/or map columns doesn't work

2014-11-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4182: -- Description: If a table contains a column whose type is binary, array, struct, map, and for some

[jira] [Updated] (SPARK-4182) Caching tables containing boolean, binary, array, struct and/or map columns doesn't work

2014-11-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4182: -- Description: If a table contains a column whose type is binary, array, struct, map, and for some

[jira] [Resolved] (SPARK-3914) InMemoryRelation should inherit statistics of its child to enable broadcast join

2014-11-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-3914. --- Resolution: Fixed InMemoryRelation should inherit statistics of its child to enable broadcast join

[jira] [Created] (SPARK-4202) DSL support for Scala UDF

2014-11-02 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4202: - Summary: DSL support for Scala UDF Key: SPARK-4202 URL: https://issues.apache.org/jira/browse/SPARK-4202 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-4199) Drop table if exists raises table not found exception in HiveContext

2014-11-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14194376#comment-14194376 ] Cheng Lian commented on SPARK-4199: --- Hi [~huangjs], which version/commit are you using?

[jira] [Closed] (SPARK-4217) Result of SparkSQL is incorrect after a table join and group by operation

2014-11-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian closed SPARK-4217. - Resolution: Invalid Result of SparkSQL is incorrect after a table join and group by operation

[jira] [Closed] (SPARK-4252) SparkSQL behaves differently from Hive when encountering illegal record

2014-11-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian closed SPARK-4252. - Resolution: Duplicate Had an offline discussion with [~patrickliu], the Hive version he was using is

[jira] [Commented] (SPARK-4252) SparkSQL behaves differently from Hive when encountering illegal record

2014-11-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14198568#comment-14198568 ] Cheng Lian commented on SPARK-4252: --- [~patrickliu] Hadn't asked for more details before

[jira] [Created] (SPARK-4308) SQL operation state is not properly set when exception is thrown

2014-11-09 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4308: - Summary: SQL operation state is not properly set when exception is thrown Key: SPARK-4308 URL: https://issues.apache.org/jira/browse/SPARK-4308 Project: Spark

[jira] [Created] (SPARK-4309) Date type support missing in HiveThriftServer2

2014-11-09 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4309: - Summary: Date type support missing in HiveThriftServer2 Key: SPARK-4309 URL: https://issues.apache.org/jira/browse/SPARK-4309 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4322) Struct fields can't be used as sub-expression of grouping fields

2014-11-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4322: -- Summary: Struct fields can't be used as sub-expression of grouping fields (was: Analysis incorrectly

[jira] [Updated] (SPARK-4322) Struct fields can't be used as sub-expression of grouping fields

2014-11-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4322: -- Description: Some examples: {code} sqlContext.jsonRDD(sc.parallelize({a: {b: [{c: 1}]}} ::

[jira] [Created] (SPARK-4407) Thrift server for 0.13.1 doesn't deserialize complex types properly

2014-11-14 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4407: - Summary: Thrift server for 0.13.1 doesn't deserialize complex types properly Key: SPARK-4407 URL: https://issues.apache.org/jira/browse/SPARK-4407 Project: Spark

[jira] [Created] (SPARK-4453) Simplify Parquet record filter generation

2014-11-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4453: - Summary: Simplify Parquet record filter generation Key: SPARK-4453 URL: https://issues.apache.org/jira/browse/SPARK-4453 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-4468) Wrong Parquet filters are created for all inequality predicates with literals on the left hand side

2014-11-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4468: - Summary: Wrong Parquet filters are created for all inequality predicates with literals on the left hand side Key: SPARK-4468 URL: https://issues.apache.org/jira/browse/SPARK-4468

[jira] [Commented] (SPARK-4258) NPE with new Parquet Filters

2014-11-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14216329#comment-14216329 ] Cheng Lian commented on SPARK-4258: --- Reproduced this issue with the following test case

[jira] [Comment Edited] (SPARK-4258) NPE with new Parquet Filters

2014-11-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14216329#comment-14216329 ] Cheng Lian edited comment on SPARK-4258 at 11/18/14 3:51 PM: -

[jira] [Created] (SPARK-4493) Don't pushdown Eq, NotEq, Lt, LtEq, Gt and GtEq predicates with nulls for Parquet

2014-11-19 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4493: - Summary: Don't pushdown Eq, NotEq, Lt, LtEq, Gt and GtEq predicates with nulls for Parquet Key: SPARK-4493 URL: https://issues.apache.org/jira/browse/SPARK-4493 Project:

[jira] [Updated] (SPARK-4493) Don't pushdown Eq, NotEq, Lt, LtEq, Gt and GtEq predicates with nulls for Parquet

2014-11-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4493: -- Description: Predicates like {{a = NULL}} and {{a NULL}} can't be pushed down since Parquet {{Lt}},

[jira] [Updated] (SPARK-4493) Don't pushdown Eq, NotEq, Lt, LtEq, Gt and GtEq predicates with nulls for Parquet

2014-11-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4493: -- Description: Predicates like {{a = NULL}} and {{a NULL}} can't be pushed down since Parquet {{Lt}},

[jira] [Commented] (SPARK-4499) Problems building and running 1.2 with Hive support

2014-11-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220513#comment-14220513 ] Cheng Lian commented on SPARK-4499: --- Hey [~GKyriacou], thanks for reporting, but both

[jira] [Comment Edited] (SPARK-4499) Problems building and running 1.2 with Hive support

2014-11-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220513#comment-14220513 ] Cheng Lian edited comment on SPARK-4499 at 11/21/14 4:40 AM: -

[jira] [Commented] (SPARK-4114) Use stable Hive API (if one exists) for communication with Metastore

2014-11-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221795#comment-14221795 ] Cheng Lian commented on SPARK-4114: --- Did some research into this and came to the

[jira] [Commented] (SPARK-4395) Running a Spark SQL SELECT command from PySpark causes a hang for ~ 1 hour

2014-11-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223992#comment-14223992 ] Cheng Lian commented on SPARK-4395: --- [~davies] Sure, I'll take a look. Running a Spark

[jira] [Commented] (SPARK-4395) Running a Spark SQL SELECT command from PySpark causes a hang for ~ 1 hour

2014-11-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224183#comment-14224183 ] Cheng Lian commented on SPARK-4395: --- Couldn't reproduce the long pause locally, and

[jira] [Created] (SPARK-4645) Asynchronous execution in HiveThriftServer2 with Hive 0.13.1 doesn't play well with Simba ODBC driver

2014-11-27 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4645: - Summary: Asynchronous execution in HiveThriftServer2 with Hive 0.13.1 doesn't play well with Simba ODBC driver Key: SPARK-4645 URL: https://issues.apache.org/jira/browse/SPARK-4645

[jira] [Updated] (SPARK-4645) Asynchronous execution in HiveThriftServer2 with Hive 0.13.1 doesn't play well with Simba ODBC driver

2014-11-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4645: -- Description: Hive 0.13.1 enables asynchronous execution for {{SQLOperation}} by default. So does Spark

[jira] [Updated] (SPARK-4645) Asynchronous execution in HiveThriftServer2 with Hive 0.13.1 doesn't play well with Simba ODBC driver

2014-11-28 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4645: -- Description: Hive 0.13.1 enables asynchronous execution for {{SQLOperation}} by default. So does Spark

[jira] [Commented] (SPARK-4761) With JDBC server, set Kryo as default serializer and disable reference tracking

2014-12-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14235604#comment-14235604 ] Cheng Lian commented on SPARK-4761: --- The JDBC Thrift server is started by

[jira] [Created] (SPARK-4785) When called with arguments referring column fields, PMOD throws NPE

2014-12-07 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4785: - Summary: When called with arguments referring column fields, PMOD throws NPE Key: SPARK-4785 URL: https://issues.apache.org/jira/browse/SPARK-4785 Project: Spark

[jira] [Updated] (SPARK-4785) When called with arguments referring column fields, PMOD throws NPE

2014-12-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4785: -- Description: Reproducible when compiled with {{-Phive-0.13.1}}, haven't tested {{-Phive0.12.0}} yet.

[jira] [Updated] (SPARK-4785) When called with arguments referring column fields, PMOD throws NPE

2014-12-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4785: -- Description: Reproducible when compiled with {{-Phive-0.13.1}}, {{-Phive0.12.0}} is OK. Reproduction

[jira] [Commented] (SPARK-4785) When called with arguments referring column fields, PMOD throws NPE

2014-12-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237649#comment-14237649 ] Cheng Lian commented on SPARK-4785: --- Looked into this a bit. Seems that this issue is

[jira] [Created] (SPARK-4786) Parquet filter pushdown for BYTE and SHORT types

2014-12-08 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4786: - Summary: Parquet filter pushdown for BYTE and SHORT types Key: SPARK-4786 URL: https://issues.apache.org/jira/browse/SPARK-4786 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-4798) Refactor Parquet test suites

2014-12-09 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-4798: - Summary: Refactor Parquet test suites Key: SPARK-4798 URL: https://issues.apache.org/jira/browse/SPARK-4798 Project: Spark Issue Type: Test Components:

[jira] [Commented] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

2014-12-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14241983#comment-14241983 ] Cheng Lian commented on SPARK-4814: --- Thanks [~srowen]! I'll take a look. Enable

[jira] [Commented] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

2014-12-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242611#comment-14242611 ] Cheng Lian commented on SPARK-4814: --- This assertion failure seems to be related to

[jira] [Updated] (SPARK-4852) Hive query plan deserialization failure caused by shaded hive-exec jar file when generating golden answers

2014-12-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4852: -- Priority: Minor (was: Major) Hive query plan deserialization failure caused by shaded hive-exec jar

[jira] [Commented] (SPARK-4852) Hive query plan deserialization failure caused by shaded hive-exec jar file when generating golden answers

2014-12-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246799#comment-14246799 ] Cheng Lian commented on SPARK-4852: --- Lowered priority to Minor since this issue only

<    1   2   3   4   5   6   7   8   9   10   >