[jira] [Commented] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-06-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341658#comment-15341658 ] Cheng Lian commented on SPARK-16032: Hey [~rdblue], [~yhuai] and [~cloud_fan] had already covered a

[jira] [Resolved] (SPARK-15894) Add doc to control #partition for input files

2016-06-21 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15894. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13797

[jira] [Updated] (SPARK-15894) Add doc to control #partition for input files

2016-06-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15894: --- Assignee: Takeshi Yamamuro > Add doc to control #partition for input files >

[jira] [Resolved] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-20 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16030. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13769

[jira] [Updated] (SPARK-16033) DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto()

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16033: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-16032 > DataFrameWriter.partitionBy() can't

[jira] [Created] (SPARK-16033) DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto()

2016-06-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16033: -- Summary: DataFrameWriter.partitionBy() can't be used together with DataFrameWriter.insertInto() Key: SPARK-16033 URL: https://issues.apache.org/jira/browse/SPARK-16033

[jira] [Updated] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16030: --- Assignee: Yin Huai > Allow specifying static partitions in an INSERT statement for data source >

[jira] [Created] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-06-17 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-16032: -- Summary: Audit semantics of various insertion operations related to partitioned tables Key: SPARK-16032 URL: https://issues.apache.org/jira/browse/SPARK-16032 Project:

[jira] [Updated] (SPARK-16030) Allow specifying static partitions in an INSERT statement for data source tables

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16030: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-16032 > Allow specifying static partitions in

[jira] [Updated] (SPARK-15916) JDBC AND/OR operator push down does not respect lower OR operator precedence

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15916: --- Description: A table from SQL server Northwind database was registered as a JDBC dataframe. A query

[jira] [Updated] (SPARK-15916) JDBC AND/OR operator push down does not respect lower OR operator precedence

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15916: --- Assignee: Hyukjin Kwon > JDBC AND/OR operator push down does not respect lower OR operator

[jira] [Resolved] (SPARK-15916) JDBC AND/OR operator push down does not respect lower OR operator precedence

2016-06-17 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15916. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13743

[jira] [Resolved] (SPARK-15862) Better Error Message When Having Database Name in CACHE TABLE AS SELECT

2016-06-16 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15862. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13572

[jira] [Resolved] (SPARK-15786) joinWith bytecode generation calling ByteBuffer.wrap with InternalRow

2016-06-16 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15786. Resolution: Duplicate Fix Version/s: 2.0.0 > joinWith bytecode generation calling

[jira] [Updated] (SPARK-15786) joinWith bytecode generation calling ByteBuffer.wrap with InternalRow

2016-06-16 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15786: --- Assignee: Sean Zhong > joinWith bytecode generation calling ByteBuffer.wrap with InternalRow >

[jira] [Updated] (SPARK-15983) Remove FileFormat.prepareRead()

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15983: --- Summary: Remove FileFormat.prepareRead() (was: Remove FileFormat.prepareRead) > Remove

[jira] [Updated] (SPARK-15983) Remove FileFormat.prepareRead

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15983: --- Description: Interface method {{FileFormat.prepareRead()}} was added in [PR

[jira] [Created] (SPARK-15983) Remove FileFormat.prepareRead

2016-06-15 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15983: -- Summary: Remove FileFormat.prepareRead Key: SPARK-15983 URL: https://issues.apache.org/jira/browse/SPARK-15983 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-15901) Test Cases for CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15901: --- Assignee: Xiao Li > Test Cases for CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET >

[jira] [Resolved] (SPARK-15901) Test Cases for CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15901. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13622

[jira] [Commented] (SPARK-14953) LocalBackend should revive offers periodically

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332560#comment-15332560 ] Cheng Lian commented on SPARK-14953: Marked this as won't fix since this isn't causing any actual

[jira] [Resolved] (SPARK-14953) LocalBackend should revive offers periodically

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14953. Resolution: Won't Fix > LocalBackend should revive offers periodically >

[jira] [Resolved] (SPARK-15929) DataFrameSuite path globbing error message tests are not fully portable

2016-06-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15929. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13649

[jira] [Commented] (SPARK-15931) SparkR tests failing on R 3.3.0

2016-06-13 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328403#comment-15328403 ] Cheng Lian commented on SPARK-15931: cc [~mengxr] > SparkR tests failing on R 3.3.0 >

[jira] [Created] (SPARK-15931) SparkR tests failing on R 3.3.0

2016-06-13 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15931: -- Summary: SparkR tests failing on R 3.3.0 Key: SPARK-15931 URL: https://issues.apache.org/jira/browse/SPARK-15931 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-15925) Replaces registerTempTable with createOrReplaceTempView in SparkR

2016-06-13 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15925: -- Summary: Replaces registerTempTable with createOrReplaceTempView in SparkR Key: SPARK-15925 URL: https://issues.apache.org/jira/browse/SPARK-15925 Project: Spark

[jira] [Reopened] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reopened SPARK-15639: We've decided to revert the merged PR, so reopening it. > Try to push down filter at RowGroups level

[jira] [Updated] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15639: --- Assignee: Liang-Chi Hsieh > Try to push down filter at RowGroups level for parquet reader >

[jira] [Updated] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15639: --- Affects Version/s: 2.0.0 > Try to push down filter at RowGroups level for parquet reader >

[jira] [Resolved] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15639. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13371

[jira] [Resolved] (SPARK-15884) Override stringArgs method in MapPartitionsInR case class in order to avoid Out Of Mermory exceptions when calling toString

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15884. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13610

[jira] [Updated] (SPARK-15884) Override stringArgs method in MapPartitionsInR case class in order to avoid Out Of Mermory exceptions when calling toString

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15884: --- Assignee: Narine Kokhlikyan > Override stringArgs method in MapPartitionsInR case class in order to

[jira] [Updated] (SPARK-15862) Better Error Message When Having Database Name in CACHE TABLE AS SELECT

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15862: --- Assignee: Xiao Li > Better Error Message When Having Database Name in CACHE TABLE AS SELECT >

[jira] [Resolved] (SPARK-15753) Move some Analyzer stuff to Analyzer from DataFrameWriter

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15753. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13496

[jira] [Updated] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15856: --- Assignee: Wenchen Fan > Revert API breaking changes made in DataFrameReader.text and

[jira] [Updated] (SPARK-15753) Move some Analyzer stuff to Analyzer from DataFrameWriter

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15753: --- Assignee: Liang-Chi Hsieh > Move some Analyzer stuff to Analyzer from DataFrameWriter >

[jira] [Created] (SPARK-15863) Update SQL programming guide for Spark 2.0

2016-06-09 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15863: -- Summary: Update SQL programming guide for Spark 2.0 Key: SPARK-15863 URL: https://issues.apache.org/jira/browse/SPARK-15863 Project: Spark Issue Type:

[jira] [Updated] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

2016-06-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15856: --- Description: In Spark 2.0, after unifying Datasets and DataFrames, we made two API breaking

[jira] [Updated] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

2016-06-09 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15856: --- Description: In Spark 2.0, after unifying Datasets and DataFrames, we made two API breaking

[jira] [Created] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

2016-06-09 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15856: -- Summary: Revert API breaking changes made in DataFrameReader.text and SQLContext.range Key: SPARK-15856 URL: https://issues.apache.org/jira/browse/SPARK-15856 Project:

[jira] [Resolved] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

2016-06-07 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15792. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13535

[jira] [Resolved] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15632. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13529

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317460#comment-15317460 ] Cheng Lian commented on SPARK-15632: The {{.map(identity)}} example is quite interesting, thanks for

[jira] [Updated] (SPARK-15654) Reading gzipped files results in duplicate rows

2016-06-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15654: --- Assignee: Takeshi Yamamuro > Reading gzipped files results in duplicate rows >

[jira] [Resolved] (SPARK-15657) RowEncoder should validate the data type of input object

2016-06-05 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15657. Resolution: Fixed Fix Version/s: 2.0.0 Resolved by

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Assignee: Sean Zhong > Dataset typed filter operation changes query plan schema >

[jira] [Commented] (SPARK-15140) encoder should make sure input object is not null

2016-06-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314853#comment-15314853 ] Cheng Lian commented on SPARK-15140: Issue resolved by pull request 13469

[jira] [Resolved] (SPARK-15140) encoder should make sure input object is not null

2016-06-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15140. Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > encoder should

[jira] [Resolved] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-06-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15547. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13474

[jira] [Resolved] (SPARK-15494) encoder code cleanup

2016-06-03 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15494. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13269

[jira] [Resolved] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

2016-06-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14959. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13463

[jira] [Updated] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

2016-06-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14959: --- Assignee: Xin Wu > ​Problem Reading partitioned ORC or Parquet files >

[jira] [Resolved] (SPARK-15733) Makes the explain output less verbose by hiding some verbose output like None, null, empty List, and etc..

2016-06-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15733. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13470

[jira] [Updated] (SPARK-15733) Makes the explain output less verbose by hiding some verbose output like None, null, empty List, and etc..

2016-06-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15733: --- Assignee: Sean Zhong > Makes the explain output less verbose by hiding some verbose output like >

[jira] [Resolved] (SPARK-15732) Dataset generated code "generated.java" Fails with Certain Case Classes

2016-06-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15732. Resolution: Fixed Fix Version/s: 2.0.0 Resolved by

[jira] [Updated] (SPARK-15732) Dataset generated code "generated.java" Fails with Certain Case Classes

2016-06-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15732: --- Assignee: Wenchen Fan > Dataset generated code "generated.java" Fails with Certain Case Classes >

[jira] [Resolved] (SPARK-15734) Avoids printing internal row in explain output

2016-06-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15734. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13471

[jira] [Resolved] (SPARK-15719) Disable writing Parquet summary files by default

2016-06-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15719. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13455

[jira] [Updated] (SPARK-15734) Avoids printing internal row in explain output

2016-06-02 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15734: --- Assignee: Sean Zhong > Avoids printing internal row in explain output >

[jira] [Updated] (SPARK-13484) Filter outer joined result using a non-nullable column from the right table

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13484: --- Assignee: Takeshi Yamamuro > Filter outer joined result using a non-nullable column from the right

[jira] [Resolved] (SPARK-13484) Filter outer joined result using a non-nullable column from the right table

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-13484. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13290

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311363#comment-15311363 ] Cheng Lian commented on SPARK-11153: Yea, right. Can we do it later on master to minimize merge

[jira] [Resolved] (SPARK-15441) dataset outer join seems to return incorrect result

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15441. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13425

[jira] [Reopened] (SPARK-9876) Upgrade parquet-mr to 1.8.1

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reopened SPARK-9876: --- Re-opened this since we just reverted 1.8.1 upgrade for branch-2.0.

[jira] [Resolved] (SPARK-15269) Creating external table leaves empty directory under warehouse directory

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15269. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13270

[jira] [Updated] (SPARK-15712) Proper temp table support

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15712: --- Description: For proper temp table support, I am proposing to create a temp dir for every

[jira] [Created] (SPARK-15719) Disable writing Parquet summary files by default

2016-06-01 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15719: -- Summary: Disable writing Parquet summary files by default Key: SPARK-15719 URL: https://issues.apache.org/jira/browse/SPARK-15719 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311231#comment-15311231 ] Cheng Lian commented on SPARK-11153: Unfortunately we just decided to revert Parquet 1.8.1. See:

[jira] [Commented] (SPARK-13795) ClassCast Exception while attempting to show() a DataFrame

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310700#comment-15310700 ] Cheng Lian commented on SPARK-13795: [~ganeshkrishnan] From the stack trace, I suspect that some

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: h1. Overview Filter operations should never change query plan schema. However, Dataset

[jira] [Resolved] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

2016-06-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14343. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13431

[jira] [Assigned] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

2016-05-31 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-14343: -- Assignee: Cheng Lian > Dataframe operations on a partitioned dataset (using partition

[jira] [Resolved] (SPARK-6859) Parquet File Binary column statistics error when reuse byte[] among rows

2016-05-31 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-6859. --- Resolution: Fixed Assignee: Ryan Blue Fix Version/s: 2.0.0 Fixed by upgrading

[jira] [Commented] (SPARK-6859) Parquet File Binary column statistics error when reuse byte[] among rows

2016-05-31 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308754#comment-15308754 ] Cheng Lian commented on SPARK-6859: --- Yea, thanks. I'm closing it. > Parquet File Binary column

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-05-31 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: h1. Overview Filter operations should never change query plan schema. However, Dataset

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

2016-05-31 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308083#comment-15308083 ] Cheng Lian commented on SPARK-8118: --- Yea, unfortunately at last we found that due to a few Parquet side

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-05-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: h1. Overview Filter operations should never change query plan schema. However, Dataset

[jira] [Resolved] (SPARK-15112) Dataset filter returns garbage

2016-05-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15112. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13362

[jira] [Updated] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

2016-05-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-14343: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Dataframe operations on a partitioned

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-05-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: Filter operations should never change query plan schema. However, Dataset typed filter

[jira] [Updated] (SPARK-9876) Upgrade parquet-mr to 1.8.1

2016-05-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-9876: -- Assignee: Ryan Blue > Upgrade parquet-mr to 1.8.1 > --- > >

[jira] [Resolved] (SPARK-9876) Upgrade parquet-mr to 1.8.1

2016-05-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-9876. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13280

[jira] [Commented] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-05-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304873#comment-15304873 ] Cheng Lian commented on SPARK-15632: cc [~cloud_fan] [~marmbrus] > Dataset typed filter operation

[jira] [Updated] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-05-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15632: --- Description: Filter operations should never changes query plan schema. However, Dataset typed

[jira] [Created] (SPARK-15632) Dataset typed filter operation changes query plan schema

2016-05-27 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15632: -- Summary: Dataset typed filter operation changes query plan schema Key: SPARK-15632 URL: https://issues.apache.org/jira/browse/SPARK-15632 Project: Spark Issue

[jira] [Updated] (SPARK-15112) Dataset filter returns garbage

2016-05-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15112: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Dataset filter returns garbage >

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

2016-05-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Dataset.show() doesn't disply inner

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-27 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15631 > Encoder validation is too strict for

[jira] [Created] (SPARK-15631) Dataset and encoder bug fixes

2016-05-27 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15631: -- Summary: Dataset and encoder bug fixes Key: SPARK-15631 URL: https://issues.apache.org/jira/browse/SPARK-15631 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

2016-05-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15550. Resolution: Fixed Issue resolved by pull request 13331

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

2016-05-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Description: Say we have the following nested case class: {code} case class ClassData(a: String, b:

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class

[jira] [Updated] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

2016-05-26 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15550: --- Description: The following Spark shell snippet reproduces this issue: {code} case class

[jira] [Created] (SPARK-15550) Dataset.show() doesn't disply inner nested structs properly

2016-05-26 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15550: -- Summary: Dataset.show() doesn't disply inner nested structs properly Key: SPARK-15550 URL: https://issues.apache.org/jira/browse/SPARK-15550 Project: Spark

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class

[jira] [Updated] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15547: --- Description: The following Spark shell snippet reproduces this issue: {code} case class

[jira] [Created] (SPARK-15547) Encoder validation is too strict for inner nested structs

2016-05-25 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15547: -- Summary: Encoder validation is too strict for inner nested structs Key: SPARK-15547 URL: https://issues.apache.org/jira/browse/SPARK-15547 Project: Spark Issue

[jira] [Resolved] (SPARK-15498) fix slow tests

2016-05-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15498. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13273

[jira] [Updated] (SPARK-15431) Support LIST FILE(s)|JAR(s) command natively

2016-05-23 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15431: --- Assignee: Xin Wu > Support LIST FILE(s)|JAR(s) command natively >

<    1   2   3   4   5   6   7   8   9   10   >