[jira] [Updated] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14959: --- Assignee: Xin Wu > ​Problem Reading partitioned ORC or Parquet fi

[jira] [Resolved] (SPARK-15733) Makes the explain output less verbose by hiding some verbose output like None, null, empty List, and etc..

[ https://issues.apache.org/jira/browse/SPARK-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15733. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13470 [https

[jira] [Updated] (SPARK-15733) Makes the explain output less verbose by hiding some verbose output like None, null, empty List, and etc..

[ https://issues.apache.org/jira/browse/SPARK-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15733: --- Assignee: Sean Zhong > Makes the explain output less verbose by hiding some verbose output l

[jira] [Resolved] (SPARK-15732) Dataset generated code "generated.java" Fails with Certain Case Classes

[ https://issues.apache.org/jira/browse/SPARK-15732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15732. Resolution: Fixed Fix Version/s: 2.0.0 Resolved by https://github.com/apache/spark/pull

[jira] [Updated] (SPARK-15732) Dataset generated code "generated.java" Fails with Certain Case Classes

[ https://issues.apache.org/jira/browse/SPARK-15732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15732: --- Assignee: Wenchen Fan > Dataset generated code "generated.java" Fails with Certain

[jira] [Resolved] (SPARK-15734) Avoids printing internal row in explain output

[ https://issues.apache.org/jira/browse/SPARK-15734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15734. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13471 [https

[jira] [Resolved] (SPARK-15719) Disable writing Parquet summary files by default

[ https://issues.apache.org/jira/browse/SPARK-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15719. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13455 [https

[jira] [Updated] (SPARK-15734) Avoids printing internal row in explain output

[ https://issues.apache.org/jira/browse/SPARK-15734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15734: --- Assignee: Sean Zhong > Avoids printing internal row in explain out

[jira] [Updated] (SPARK-13484) Filter outer joined result using a non-nullable column from the right table

[ https://issues.apache.org/jira/browse/SPARK-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13484: --- Assignee: Takeshi Yamamuro > Filter outer joined result using a non-nullable column from the ri

[jira] [Resolved] (SPARK-13484) Filter outer joined result using a non-nullable column from the right table

[ https://issues.apache.org/jira/browse/SPARK-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13484. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13290 [https

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311363#comment-15311363 ] Cheng Lian commented on SPARK-11153: Yea, right. Can we do it later on master to minimize merge

[jira] [Resolved] (SPARK-15441) dataset outer join seems to return incorrect result

[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15441. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13425 [https

[jira] [Reopened] (SPARK-9876) Upgrade parquet-mr to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reopened SPARK-9876: --- Re-opened this since we just reverted 1.8.1 upgrade for branch-2.0. https://github.com/apache/spark/pull

[jira] [Resolved] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15269. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13270 [https

[jira] [Updated] (SPARK-15712) Proper temp table support

[ https://issues.apache.org/jira/browse/SPARK-15712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15712: --- Description: For proper temp table support, I am proposing to create a temp dir for every

[jira] [Created] (SPARK-15719) Disable writing Parquet summary files by default

Cheng Lian created SPARK-15719: -- Summary: Disable writing Parquet summary files by default Key: SPARK-15719 URL: https://issues.apache.org/jira/browse/SPARK-15719 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311231#comment-15311231 ] Cheng Lian commented on SPARK-11153: Unfortunately we just decided to revert Parquet 1.8.1. See

[jira] [Commented] (SPARK-13795) ClassCast Exception while attempting to show() a DataFrame

[ https://issues.apache.org/jira/browse/SPARK-13795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310700#comment-15310700 ] Cheng Lian commented on SPARK-13795: [~ganeshkrishnan] From the stack trace, I suspect that some

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: h1. Overview Filter operations should never change query plan schema. However, Dataset

[jira] [Resolved] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14343. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13431 [https

[jira] [Assigned] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-14343: -- Assignee: Cheng Lian > Dataframe operations on a partitioned dataset (using partit

[jira] [Resolved] (SPARK-6859) Parquet File Binary column statistics error when reuse byte[] among rows

[ https://issues.apache.org/jira/browse/SPARK-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6859. --- Resolution: Fixed Assignee: Ryan Blue Fix Version/s: 2.0.0 Fixed by upgrading parquet

[jira] [Commented] (SPARK-6859) Parquet File Binary column statistics error when reuse byte[] among rows

[ https://issues.apache.org/jira/browse/SPARK-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308754#comment-15308754 ] Cheng Lian commented on SPARK-6859: --- Yea, thanks. I'm closing it. > Parquet File Binary col

Re: latest Parquet in Spark

Discussed with Kirill offline. I'm posting a tl;dr here for future reference: We've just upgraded to parquet-mr 1.8.1 for Spark 2.0. Thanks to Ryan for doing this! I believe the PR thread already answers all the questions below: https://github.com/apache/spark/pull/13280 Cheng On 5/14/16

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: h1. Overview Filter operations should never change query plan schema. However, Dataset

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308083#comment-15308083 ] Cheng Lian commented on SPARK-8118: --- Yea, unfortunately at last we found that due to a few Parquet side

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: h1. Overview Filter operations should never change query plan schema. However, Dataset

[jira] [Resolved] (SPARK-15112) Dataset filter returns garbage

[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15112. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13362 [https

[jira] [Updated] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14343: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Dataframe operations on a partitio

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: Filter operations should never change query plan schema. However, Dataset typed filter

[jira] [Updated] (SPARK-9876) Upgrade parquet-mr to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-9876: -- Assignee: Ryan Blue > Upgrade parquet-mr to 1.8.1 > --- > >

[jira] [Resolved] (SPARK-9876) Upgrade parquet-mr to 1.8.1

[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-9876. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13280 [https

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304873#comment-15304873 ] Cheng Lian commented on SPARK-15632: cc [~cloud_fan] [~marmbrus] > Dataset typed filter operat

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: Filter operations should never changes query plan schema. However, Dataset typed

[jira] [Created] (SPARK-15632) Dataset typed filter operation changes query plan schema

Cheng Lian created SPARK-15632: -- Summary: Dataset typed filter operation changes query plan schema Key: SPARK-15632 URL: https://issues.apache.org/jira/browse/SPARK-15632 Project: Spark Issue

[jira] [Updated] (SPARK-15112) Dataset filter returns garbage

[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15112: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Dataset filter returns garb

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Dataset.show() doesn't disply in

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Encoder validation is too str

[jira] [Created] (SPARK-15631) Dataset and encoder bug fixes

Cheng Lian created SPARK-15631: -- Summary: Dataset and encoder bug fixes Key: SPARK-15631 URL: https://issues.apache.org/jira/browse/SPARK-15631 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15550. Resolution: Fixed Issue resolved by pull request 13331 [https://github.com/apache/spark/pull/13331

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Description: Say we have the following nested case class: {code} case class ClassData(a: String, b

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData

[jira] [Created] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

Cheng Lian created SPARK-15550: -- Summary: Dataset.show() doesn't disply inner nested structs properly Key: SPARK-15550 URL: https://issues.apache.org/jira/browse/SPARK-15550 Project: Spark

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class ClassData

[jira] [Created] (SPARK-15547) Encoder validation is too strict for inner nested structs

Cheng Lian created SPARK-15547: -- Summary: Encoder validation is too strict for inner nested structs Key: SPARK-15547 URL: https://issues.apache.org/jira/browse/SPARK-15547 Project: Spark Issue

Re: feedback on dataset api explode

Agree, since they can be easily replaced by .flatMap (to do explosion) and .select (to rename output columns) Cheng On 5/25/16 12:30 PM, Reynold Xin wrote: Based on this discussion I'm thinking we should deprecate the two explode functions. On Wednesday, May 25, 2016, Koert Kuipers

[jira] [Resolved] (SPARK-15498) fix slow tests

[ https://issues.apache.org/jira/browse/SPARK-15498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15498. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13273 [https

[jira] [Updated] (SPARK-15431) Support LIST FILE(s)|JAR(s) command natively

[ https://issues.apache.org/jira/browse/SPARK-15431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15431: --- Assignee: Xin Wu > Support LIST FILE(s)|JAR(s) command nativ

[jira] [Resolved] (SPARK-15431) Support LIST FILE(s)|JAR(s) command natively

[ https://issues.apache.org/jira/browse/SPARK-15431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15431. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13212 [https

[jira] [Commented] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297408#comment-15297408 ] Cheng Lian commented on SPARK-15269: Two facts make this issue pretty hard to be fixed cleanly

[jira] [Commented] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297282#comment-15297282 ] Cheng Lian commented on SPARK-14343: Seems that we were reading from the wrong column, and happened

[jira] [Resolved] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14031. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13229 [https

[jira] [Updated] (SPARK-14543) SQL/Hive insertInto has unexpected results

[ https://issues.apache.org/jira/browse/SPARK-14543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14543: --- Assignee: Ryan Blue > SQL/Hive insertInto has unexpected resu

[jira] [Resolved] (SPARK-15307) Super slow to load a partitioned table from local disks

[ https://issues.apache.org/jira/browse/SPARK-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15307. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13094 [https

[jira] [Updated] (SPARK-15307) Super slow to load a partitioned table from local disks

[ https://issues.apache.org/jira/browse/SPARK-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15307: --- Assignee: Davies Liu > Super slow to load a partitioned table from local di

[jira] [Resolved] (SPARK-15334) HiveClient facade not compatible with Hive 0.12

[ https://issues.apache.org/jira/browse/SPARK-15334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15334. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13127 [https

[jira] [Updated] (SPARK-15334) HiveClient facade not compatible with Hive 0.12

[ https://issues.apache.org/jira/browse/SPARK-15334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15334: --- Assignee: Sean Zhong > HiveClient facade not compatible with Hive 0

[jira] [Updated] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15269: --- Assignee: Xin Wu > Creating external table leaves empty directory under warehouse direct

[jira] [Updated] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15269: --- Description: Adding the following test case in {{HiveDDLSuite}} may reproduce this issue: {code

[jira] [Commented] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281657#comment-15281657 ] Cheng Lian commented on SPARK-15269: [~xwu0226] Thanks a lot for the detailed investigation! Would

[jira] [Updated] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15269: --- Summary: Creating external table leaves empty directory under warehouse directory (was: Creating

[jira] [Resolved] (SPARK-15171) Deprecate registerTempTable and add dataset.createTempView

[ https://issues.apache.org/jira/browse/SPARK-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15171. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12945 [https

[jira] [Updated] (SPARK-15171) Deprecate registerTempTable and add dataset.createTempView

[ https://issues.apache.org/jira/browse/SPARK-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15171: --- Assignee: Sean Zhong > Deprecate registerTempTable and add dataset.createTempV

[jira] [Resolved] (SPARK-14933) Failed to create view out of a parquet or orc table

[ https://issues.apache.org/jira/browse/SPARK-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14933. Issue resolved by pull request 12716 [https://github.com/apache/spark/pull/12716] > Failed to cre

[jira] [Comment Edited] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279992#comment-15279992 ] Cheng Lian edited comment on SPARK-15269 at 5/11/16 11:44 AM: -- Investigated

[jira] [Commented] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15279992#comment-15279992 ] Cheng Lian commented on SPARK-15269: Investigated this issue for a while, and observed the following

[jira] [Created] (SPARK-15269) Creating external table in test code leaves empty directory under warehouse directory

Cheng Lian created SPARK-15269: -- Summary: Creating external table in test code leaves empty directory under warehouse directory Key: SPARK-15269 URL: https://issues.apache.org/jira/browse/SPARK-15269

[jira] [Updated] (SPARK-15253) For a data source table, Describe table needs to handle spark.sql.sources.schema

[ https://issues.apache.org/jira/browse/SPARK-15253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15253: --- Assignee: Sean Zhong > For a data source table, Describe table needs to han

[jira] [Updated] (SPARK-15192) RowEncoder needs to verify nullability in a more explicit way

[ https://issues.apache.org/jira/browse/SPARK-15192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15192: --- Description: When we create a Dataset from an RDD of rows with a specific schema

[jira] [Updated] (SPARK-14459) SQL partitioning must match existing tables, but is not checked.

[ https://issues.apache.org/jira/browse/SPARK-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14459: --- Assignee: Ryan Blue > SQL partitioning must match existing tables, but is not chec

[jira] [Resolved] (SPARK-14459) SQL partitioning must match existing tables, but is not checked.

[ https://issues.apache.org/jira/browse/SPARK-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14459. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12239 [https

[jira] [Updated] (SPARK-14459) SQL partitioning must match existing tables, but is not checked.

[ https://issues.apache.org/jira/browse/SPARK-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14459: --- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 > SQL partitioning must match existing tab

[jira] [Updated] (SPARK-15211) Select features column from LibSVMRelation causes failure

[ https://issues.apache.org/jira/browse/SPARK-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15211: --- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 Description: It will cause failure

[jira] [Resolved] (SPARK-15211) Select features column from LibSVMRelation causes failure

[ https://issues.apache.org/jira/browse/SPARK-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15211. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12986 [https

[jira] [Updated] (SPARK-15211) Select features column from LibSVMRelation causes failure

[ https://issues.apache.org/jira/browse/SPARK-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15211: --- Assignee: Liang-Chi Hsieh > Select features column from LibSVMRelation causes fail

[jira] [Resolved] (SPARK-14962) spark.sql.orc.filterPushdown=true breaks DataFrame where functionality

[ https://issues.apache.org/jira/browse/SPARK-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14962. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12777 [https

[jira] [Updated] (SPARK-14962) spark.sql.orc.filterPushdown=true breaks DataFrame where functionality

[ https://issues.apache.org/jira/browse/SPARK-14962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14962: --- Assignee: Hyukjin Kwon > spark.sql.orc.filterPushdown=true breaks DataFrame where functional

[jira] [Comment Edited] (SPARK-15112) Dataset filter returns garbage

[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273837#comment-15273837 ] Cheng Lian edited comment on SPARK-15112 at 5/6/16 10:22 AM: - Actually

[jira] [Commented] (SPARK-15112) Dataset filter returns garbage

[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273837#comment-15273837 ] Cheng Lian commented on SPARK-15112: Actually there's another issue that contributes to this bug

[jira] [Updated] (SPARK-14803) A bug in EliminateSerialization rule in Catalyst Optimizer

[ https://issues.apache.org/jira/browse/SPARK-14803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14803: --- Description: When I rebased my PR https://github.com/apache/spark/pull/12493 to master, I found

[jira] [Assigned] (SPARK-15112) Dataset filter returns garbage

[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-15112: -- Assignee: Cheng Lian > Dataset filter returns garb

[jira] [Updated] (SPARK-14139) Dataset loses nullability in operations with RowEncoder

[ https://issues.apache.org/jira/browse/SPARK-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14139: --- Assignee: Wenchen Fan > Dataset loses nullability in operations with RowEnco

[jira] [Resolved] (SPARK-14139) Dataset loses nullability in operations with RowEncoder

[ https://issues.apache.org/jira/browse/SPARK-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14139. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12364 [https

[jira] [Updated] (SPARK-14933) Failed to create view out of a parquet or orc table

[ https://issues.apache.org/jira/browse/SPARK-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14933: --- Assignee: Xin Wu > Failed to create view out of a parquet or orc ta

[jira] [Commented] (SPARK-14139) Dataset loses nullability in operations with RowEncoder

[ https://issues.apache.org/jira/browse/SPARK-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272074#comment-15272074 ] Cheng Lian commented on SPARK-14139: As discussed in SPARK-15112. I think we should revert to use

[jira] [Created] (SPARK-15147) Catalog should have a property to indicate case-sensitivity

Cheng Lian created SPARK-15147: -- Summary: Catalog should have a property to indicate case-sensitivity Key: SPARK-15147 URL: https://issues.apache.org/jira/browse/SPARK-15147 Project: Spark

[jira] [Comment Edited] (SPARK-15112) Dataset filter returns garbage

[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271025#comment-15271025 ] Cheng Lian edited comment on SPARK-15112 at 5/5/16 12:28 AM: - The following

[jira] [Commented] (SPARK-15112) Dataset filter returns garbage

[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271025#comment-15271025 ] Cheng Lian commented on SPARK-15112: The following Spark shell session illustrates this issue

[jira] [Resolved] (SPARK-14127) [Table related commands] Describe table

[ https://issues.apache.org/jira/browse/SPARK-14127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14127. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12844 [https

[jira] [Resolved] (SPARK-14237) De-duplicate partition value appending logic in various buildReader() implementations

[ https://issues.apache.org/jira/browse/SPARK-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14237. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12866 [https

[jira] [Created] (SPARK-14981) CatalogTable should contain sorting directions of sorting columns

Cheng Lian created SPARK-14981: -- Summary: CatalogTable should contain sorting directions of sorting columns Key: SPARK-14981 URL: https://issues.apache.org/jira/browse/SPARK-14981 Project: Spark

[jira] [Updated] (SPARK-14954) Add PARTITIONED BY and CLUSTERED BY clause for data source CTAS syntax

[ https://issues.apache.org/jira/browse/SPARK-14954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14954: --- Summary: Add PARTITIONED BY and CLUSTERED BY clause for data source CTAS syntax (was: Add PARTITION

[jira] [Updated] (SPARK-14427) Support persisting partitioned data source relations in Hive compatible format

[ https://issues.apache.org/jira/browse/SPARK-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14427: --- Assignee: Liang-Chi Hsieh > Support persisting partitioned data source relations in Hive compati

[jira] [Updated] (SPARK-14954) Add PARTITION BY and BUCKET BY clause for data source CTAS syntax

[ https://issues.apache.org/jira/browse/SPARK-14954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14954: --- Summary: Add PARTITION BY and BUCKET BY clause for data source CTAS syntax (was: Add PARTITION

[jira] [Assigned] (SPARK-14954) Add PARTITION BY and BUCKET BY clause for "CREATE TABLE ... USING ..." syntax

[ https://issues.apache.org/jira/browse/SPARK-14954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-14954: -- Assignee: Cheng Lian > Add PARTITION BY and BUCKET BY clause for "CREATE TABLE .

[jira] [Updated] (SPARK-14954) Add PARTITION BY and BUCKET BY clause for "CREATE TABLE ... USING ..." syntax

[ https://issues.apache.org/jira/browse/SPARK-14954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14954: --- Affects Version/s: 2.0.0 Target Version/s: 2.0.0 > Add PARTITION BY and BUCKET BY cla

[jira] [Created] (SPARK-14954) Add PARTITION BY and BUCKET BY clause for "CREATE TABLE ... USING ..." syntax

Cheng Lian created SPARK-14954: -- Summary: Add PARTITION BY and BUCKET BY clause for "CREATE TABLE ... USING ..." syntax Key: SPARK-14954 URL: https://issues.apache.org/jira/browse/SPARK-14954

<    1   2   3   4   5   6   7   8   9   10   >