[jira] [Resolved] (SPARK-15431) Support LIST FILE(s)|JAR(s) command natively

2016-05-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15431. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13212

[jira] [Commented] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

2016-05-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297408#comment-15297408 ] Cheng Lian commented on SPARK-15269: Two facts make this issue pretty hard to be fixed cleanly: #

[jira] [Commented] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

2016-05-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297282#comment-15297282 ] Cheng Lian commented on SPARK-14343: Seems that we were reading from the wrong column, and happened

[jira] [Resolved] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14031. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13229

[jira] [Updated] (SPARK-14543) SQL/Hive insertInto has unexpected results

2016-05-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14543: --- Assignee: Ryan Blue > SQL/Hive insertInto has unexpected results >

[jira] [Resolved] (SPARK-15307) Super slow to load a partitioned table from local disks

2016-05-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15307. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13094

[jira] [Updated] (SPARK-15307) Super slow to load a partitioned table from local disks

2016-05-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15307: --- Assignee: Davies Liu > Super slow to load a partitioned table from local disks >

[jira] [Resolved] (SPARK-15334) HiveClient facade not compatible with Hive 0.12

2016-05-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15334. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13127

[jira] [Updated] (SPARK-15334) HiveClient facade not compatible with Hive 0.12

2016-05-18 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15334: --- Assignee: Sean Zhong > HiveClient facade not compatible with Hive 0.12 >

[jira] [Updated] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

2016-05-16 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15269: --- Assignee: Xin Wu > Creating external table leaves empty directory under warehouse directory >

[jira] [Updated] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

2016-05-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15269: --- Description: Adding the following test case in {{HiveDDLSuite}} may reproduce this issue: {code}

[jira] [Commented] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

2016-05-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281657#comment-15281657 ] Cheng Lian commented on SPARK-15269: [~xwu0226] Thanks a lot for the detailed investigation! Would

[jira] [Updated] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

2016-05-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15269: --- Summary: Creating external table leaves empty directory under warehouse directory (was: Creating

[jira] [Resolved] (SPARK-15171) Deprecate registerTempTable and add dataset.createTempView

2016-05-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15171. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12945

[jira] [Updated] (SPARK-15171) Deprecate registerTempTable and add dataset.createTempView

2016-05-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15171: --- Assignee: Sean Zhong > Deprecate registerTempTable and add dataset.createTempView >

[jira] [Resolved] (SPARK-14933) Failed to create view out of a parquet or orc table

2016-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14933. Issue resolved by pull request 12716 [https://github.com/apache/spark/pull/12716] > Failed to create

[jira] [Comment Edited] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

2016-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279992#comment-15279992 ] Cheng Lian edited comment on SPARK-15269 at 5/11/16 11:44 AM: -- Investigated

[jira] [Commented] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

2016-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279992#comment-15279992 ] Cheng Lian commented on SPARK-15269: Investigated this issue for a while, and observed the following

[jira] [Created] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

2016-05-11 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15269: -- Summary: Creating external table in test code leaves empty directory under warehouse directory Key: SPARK-15269 URL: https://issues.apache.org/jira/browse/SPARK-15269

[jira] [Updated] (SPARK-15253) For a data source table, Describe table needs to handle spark.sql.sources.schema

2016-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15253: --- Assignee: Sean Zhong > For a data source table, Describe table needs to handle >

[jira] [Updated] (SPARK-15192) RowEncoder needs to verify nullability in a more explicit way

2016-05-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15192: --- Description: When we create a Dataset from an RDD of rows with a specific schema, if the

[jira] [Updated] (SPARK-14459) SQL partitioning must match existing tables, but is not checked.

2016-05-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14459: --- Assignee: Ryan Blue > SQL partitioning must match existing tables, but is not checked. >

[jira] [Resolved] (SPARK-14459) SQL partitioning must match existing tables, but is not checked.

2016-05-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14459. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12239

[jira] [Updated] (SPARK-14459) SQL partitioning must match existing tables, but is not checked.

2016-05-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14459: --- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 > SQL partitioning must match existing tables,

[jira] [Updated] (SPARK-15211) Select features column from LibSVMRelation causes failure

2016-05-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15211: --- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 Description: It will cause failure

[jira] [Resolved] (SPARK-15211) Select features column from LibSVMRelation causes failure

2016-05-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15211. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12986

[jira] [Updated] (SPARK-15211) Select features column from LibSVMRelation causes failure

2016-05-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15211: --- Assignee: Liang-Chi Hsieh > Select features column from LibSVMRelation causes failure >

[jira] [Resolved] (SPARK-14962) spark.sql.orc.filterPushdown=true breaks DataFrame where functionality

2016-05-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14962. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12777

[jira] [Updated] (SPARK-14962) spark.sql.orc.filterPushdown=true breaks DataFrame where functionality

2016-05-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14962: --- Assignee: Hyukjin Kwon > spark.sql.orc.filterPushdown=true breaks DataFrame where functionality >

[jira] [Comment Edited] (SPARK-15112) Dataset filter returns garbage

2016-05-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273837#comment-15273837 ] Cheng Lian edited comment on SPARK-15112 at 5/6/16 10:22 AM: - Actually

[jira] [Commented] (SPARK-15112) Dataset filter returns garbage

2016-05-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273837#comment-15273837 ] Cheng Lian commented on SPARK-15112: Actually there's another issue that contributes to this bug. The

[jira] [Updated] (SPARK-14803) A bug in EliminateSerialization rule in Catalyst Optimizer

2016-05-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14803: --- Description: When I rebased my PR https://github.com/apache/spark/pull/12493 to master, I found a

[jira] [Assigned] (SPARK-15112) Dataset filter returns garbage

2016-05-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-15112: -- Assignee: Cheng Lian > Dataset filter returns garbage > -- > >

[jira] [Updated] (SPARK-14139) Dataset loses nullability in operations with RowEncoder

2016-05-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14139: --- Assignee: Wenchen Fan > Dataset loses nullability in operations with RowEncoder >

[jira] [Resolved] (SPARK-14139) Dataset loses nullability in operations with RowEncoder

2016-05-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14139. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12364

[jira] [Updated] (SPARK-14933) Failed to create view out of a parquet or orc table

2016-05-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14933: --- Assignee: Xin Wu > Failed to create view out of a parquet or orc table >

[jira] [Commented] (SPARK-14139) Dataset loses nullability in operations with RowEncoder

2016-05-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272074#comment-15272074 ] Cheng Lian commented on SPARK-14139: As discussed in SPARK-15112. I think we should revert to use

[jira] [Created] (SPARK-15147) Catalog should have a property to indicate case-sensitivity

2016-05-05 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15147: -- Summary: Catalog should have a property to indicate case-sensitivity Key: SPARK-15147 URL: https://issues.apache.org/jira/browse/SPARK-15147 Project: Spark

[jira] [Comment Edited] (SPARK-15112) Dataset filter returns garbage

2016-05-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271025#comment-15271025 ] Cheng Lian edited comment on SPARK-15112 at 5/5/16 12:28 AM: - The following

[jira] [Commented] (SPARK-15112) Dataset filter returns garbage

2016-05-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271025#comment-15271025 ] Cheng Lian commented on SPARK-15112: The following Spark shell session illustrates this issue:

[jira] [Resolved] (SPARK-14127) [Table related commands] Describe table

2016-05-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14127. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12844

[jira] [Resolved] (SPARK-14237) De-duplicate partition value appending logic in various buildReader() implementations

2016-05-04 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14237. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12866

[jira] [Created] (SPARK-14981) CatalogTable should contain sorting directions of sorting columns

2016-04-28 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-14981: -- Summary: CatalogTable should contain sorting directions of sorting columns Key: SPARK-14981 URL: https://issues.apache.org/jira/browse/SPARK-14981 Project: Spark

[jira] [Updated] (SPARK-14954) Add PARTITIONED BY and CLUSTERED BY clause for data source CTAS syntax

2016-04-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14954: --- Summary: Add PARTITIONED BY and CLUSTERED BY clause for data source CTAS syntax (was: Add PARTITION

[jira] [Updated] (SPARK-14427) Support persisting partitioned data source relations in Hive compatible format

2016-04-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14427: --- Assignee: Liang-Chi Hsieh > Support persisting partitioned data source relations in Hive compatible

[jira] [Updated] (SPARK-14954) Add PARTITION BY and BUCKET BY clause for data source CTAS syntax

2016-04-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14954: --- Summary: Add PARTITION BY and BUCKET BY clause for data source CTAS syntax (was: Add PARTITION BY

[jira] [Assigned] (SPARK-14954) Add PARTITION BY and BUCKET BY clause for "CREATE TABLE ... USING ..." syntax

2016-04-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-14954: -- Assignee: Cheng Lian > Add PARTITION BY and BUCKET BY clause for "CREATE TABLE ... USING ..."

[jira] [Updated] (SPARK-14954) Add PARTITION BY and BUCKET BY clause for "CREATE TABLE ... USING ..." syntax

2016-04-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14954: --- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 > Add PARTITION BY and BUCKET BY clause for

[jira] [Created] (SPARK-14954) Add PARTITION BY and BUCKET BY clause for "CREATE TABLE ... USING ..." syntax

2016-04-27 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-14954: -- Summary: Add PARTITION BY and BUCKET BY clause for "CREATE TABLE ... USING ..." syntax Key: SPARK-14954 URL: https://issues.apache.org/jira/browse/SPARK-14954 Project:

[jira] [Created] (SPARK-14953) LocalBackend should revive offers periodically

2016-04-27 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-14953: -- Summary: LocalBackend should revive offers periodically Key: SPARK-14953 URL: https://issues.apache.org/jira/browse/SPARK-14953 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-14918) ExternalCatalog.TablePartitionSpec doesn't preserve partition column order

2016-04-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14918. Resolution: Not A Problem > ExternalCatalog.TablePartitionSpec doesn't preserve partition column

[jira] [Commented] (SPARK-14918) ExternalCatalog.TablePartitionSpec doesn't preserve partition column order

2016-04-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259906#comment-15259906 ] Cheng Lian commented on SPARK-14918: Decided to leave this one as "not an issue" since the problem I

[jira] [Resolved] (SPARK-14445) Show columns/partitions

2016-04-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14445. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 1

[jira] [Comment Edited] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

2016-04-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258139#comment-15258139 ] Cheng Lian edited comment on SPARK-13983 at 4/26/16 4:44 PM: - Here's my

[jira] [Commented] (SPARK-14918) ExternalCatalog.TablePartitionSpec doesn't preserve partition column order

2016-04-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258191#comment-15258191 ] Cheng Lian commented on SPARK-14918: Hit this issue while reviewing

[jira] [Created] (SPARK-14918) ExternalCatalog.TablePartitionSpec doesn't preserve partition column order

2016-04-26 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-14918: -- Summary: ExternalCatalog.TablePartitionSpec doesn't preserve partition column order Key: SPARK-14918 URL: https://issues.apache.org/jira/browse/SPARK-14918 Project:

[jira] [Commented] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

2016-04-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258139#comment-15258139 ] Cheng Lian commented on SPARK-13983: Here's my (incomplete) finding: Configurations set using

[jira] [Resolved] (SPARK-14875) OutputWriterFactory.newInstance shouldn't be private[sql]

2016-04-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14875. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12652

[jira] [Commented] (SPARK-14875) OutputWriterFactory.newInstance shouldn't be private[sql]

2016-04-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255249#comment-15255249 ] Cheng Lian commented on SPARK-14875: Checked with [~cloud_fan], it was accidentally made private

[jira] [Commented] (SPARK-14875) OutputWriterFactory.newInstance shouldn't be private[sql]

2016-04-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15255248#comment-15255248 ] Cheng Lian commented on SPARK-14875: [~marmbrus] Is there any reason why we made it private in Spark

[jira] [Created] (SPARK-14875) OutputWriterFactory.newInstance shouldn't be private[sql]

2016-04-23 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-14875: -- Summary: OutputWriterFactory.newInstance shouldn't be private[sql] Key: SPARK-14875 URL: https://issues.apache.org/jira/browse/SPARK-14875 Project: Spark Issue

[jira] [Resolved] (SPARK-14843) Error while encoding: java.lang.ClassCastException with LibSVMRelation

2016-04-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14843. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12611

[jira] [Updated] (SPARK-13928) Move org.apache.spark.Logging into org.apache.spark.internal.Logging

2016-04-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13928: --- Target Version/s: 2.0.0 > Move org.apache.spark.Logging into org.apache.spark.internal.Logging >

[jira] [Updated] (SPARK-13928) Move org.apache.spark.Logging into org.apache.spark.internal.Logging

2016-04-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13928: --- Assignee: Wenchen Fan > Move org.apache.spark.Logging into org.apache.spark.internal.Logging >

[jira] [Updated] (SPARK-13928) Move org.apache.spark.Logging into org.apache.spark.internal.Logging

2016-04-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13928: --- Fix Version/s: 2.0.0 > Move org.apache.spark.Logging into org.apache.spark.internal.Logging >

[jira] [Commented] (SPARK-14463) read.text broken for partitioned tables

2016-04-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240640#comment-15240640 ] Cheng Lian commented on SPARK-14463: Should we simply throw an exception when text data source is

[jira] [Commented] (SPARK-14389) OOM during BroadcastNestedLoopJoin

2016-04-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15239687#comment-15239687 ] Cheng Lian commented on SPARK-14389: Exception thrown by UnsafeRow.copy() inside BNL join doesn't

[jira] [Commented] (SPARK-14495) Distinct aggregation cannot be used in the having clause

2016-04-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15239674#comment-15239674 ] Cheng Lian commented on SPARK-14495: This ticket is for branch-1.6. > Distinct aggregation cannot be

[jira] [Commented] (SPARK-13753) Column nullable is derived incorrectly

2016-04-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15239637#comment-15239637 ] Cheng Lian commented on SPARK-13753: [~jingweilu] Could you please provide the schema of tables

[jira] [Commented] (SPARK-14463) read.text broken for partitioned tables

2016-04-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15239617#comment-15239617 ] Cheng Lian commented on SPARK-14463: Seems that this is because {{buildReader()}} doesn't append

[jira] [Comment Edited] (SPARK-14566) When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema

2016-04-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237690#comment-15237690 ] Cheng Lian edited comment on SPARK-14566 at 4/12/16 6:25 PM: - This bug is

[jira] [Commented] (SPARK-14566) When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema

2016-04-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237690#comment-15237690 ] Cheng Lian commented on SPARK-14566: This bug is exposed after fixing SPARK-14458. These two bugs

[jira] [Created] (SPARK-14566) When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema

2016-04-12 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-14566: -- Summary: When appending to partitioned persisted table, we should apply a projection over input query plan using existing metastore schema Key: SPARK-14566 URL:

[jira] [Resolved] (SPARK-14493) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." should always be used with a user defined path

2016-04-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14493. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12303

[jira] [Resolved] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-12 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14488. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12303

[jira] [Commented] (SPARK-14372) Dataset.randomSplit() needs a Java version

2016-04-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234779#comment-15234779 ] Cheng Lian commented on SPARK-14372: Ah, sorry for the late reply. It's already taken by others. (At

[jira] [Resolved] (SPARK-14372) Dataset.randomSplit() needs a Java version

2016-04-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14372. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12184

[jira] [Updated] (SPARK-14372) Dataset.randomSplit() needs a Java version

2016-04-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14372: --- Assignee: Rekha Joshi (was: Subhobrata Dey) > Dataset.randomSplit() needs a Java version >

[jira] [Updated] (SPARK-14372) Dataset.randomSplit() needs a Java version

2016-04-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14372: --- Assignee: Subhobrata Dey > Dataset.randomSplit() needs a Java version >

[jira] [Assigned] (SPARK-14476) Show table name or path in string of DataSourceScan

2016-04-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-14476: -- Assignee: Cheng Lian > Show table name or path in string of DataSourceScan >

[jira] [Created] (SPARK-14493) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." should always be used with a user defined path

2016-04-08 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-14493: -- Summary: "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." should always be used with a user defined path Key: SPARK-14493 URL: https://issues.apache.org/jira/browse/SPARK-14493

[jira] [Commented] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232422#comment-15232422 ] Cheng Lian commented on SPARK-14488: Yea, that's why I came to this DDL command, because this command

[jira] [Comment Edited] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232414#comment-15232414 ] Cheng Lian edited comment on SPARK-14488 at 4/8/16 4:17 PM: Discussed with

[jira] [Commented] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232414#comment-15232414 ] Cheng Lian commented on SPARK-14488: Discussed with [~yhuai] offline, and here's the summary:

[jira] [Commented] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232401#comment-15232401 ] Cheng Lian commented on SPARK-14488: Ah, sorry, the logical plan class {{CreateTableUsingAsSelect}}

[jira] [Updated] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14488: --- Description: The following Spark shell snippet reproduces this bug: {code} sqlContext range 10

[jira] [Updated] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14488: --- Description: The following Spark shell snippet reproduces this bug: {code} sqlContext range 10

[jira] [Updated] (SPARK-14488) "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14488: --- Summary: "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." creates persisted table (was: Weird

[jira] [Comment Edited] (SPARK-14488) Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..."

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232092#comment-15232092 ] Cheng Lian edited comment on SPARK-14488 at 4/8/16 12:27 PM: - Tried the same

[jira] [Updated] (SPARK-14488) Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..."

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14488: --- Description: Currently, Spark 2.0 master allows DDL statements like {{CREATE TEMPORARY TABLE ...

[jira] [Commented] (SPARK-14488) Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..."

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232118#comment-15232118 ] Cheng Lian commented on SPARK-14488: Result of {{EXPLAIN EXTENDED CREATE TEMPORARY TABLE y USING

[jira] [Updated] (SPARK-14488) Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..."

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14488: --- Description: Currently, Spark 2.0 master allows DDL statements like {{CREATE TEMPORARY TABLE ...

[jira] [Commented] (SPARK-14488) Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..."

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232111#comment-15232111 ] Cheng Lian commented on SPARK-14488: However, if {{TEMPORARY + USING + AS SELECT}} is an invalid

[jira] [Commented] (SPARK-14488) Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..."

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232092#comment-15232092 ] Cheng Lian commented on SPARK-14488: Tried the same snippet using Spark 1.6, and got the following

[jira] [Commented] (SPARK-14488) Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..."

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232078#comment-15232078 ] Cheng Lian commented on SPARK-14488: Updated title and description of this ticket. > Weird behavior

[jira] [Updated] (SPARK-14488) Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..."

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14488: --- Description: Currently, Spark 2.0 master allows DDL statements like {{CREATE TEMPORARY TABLE ...

[jira] [Updated] (SPARK-14488) Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..."

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14488: --- Description: Currently, Spark 2.0 master allows DDL statements like {{CREATE TEMPORARY TABLE ...

[jira] [Updated] (SPARK-14488) Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..."

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14488: --- Summary: Weird behavior of DDL "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." (was: Creating

[jira] [Comment Edited] (SPARK-14488) Creating temporary table using SQL DDL shouldn't write files to file system

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232030#comment-15232030 ] Cheng Lian edited comment on SPARK-14488 at 4/8/16 11:10 AM: - Oh, wait...

[jira] [Commented] (SPARK-14488) Creating temporary table using SQL DDL shouldn't write files to file system

2016-04-08 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15232030#comment-15232030 ] Cheng Lian commented on SPARK-14488: Oh, wait... Since there's a {{USING}} in the DDL statement, are

<    1   2   3   4   5   6   7   8   9   10   >