[jira] [Commented] (HIVE-14294) HiveSchemaConverter for Parquet doesn't translate TINYINT and SMALLINT into proper Parquet types

[ https://issues.apache.org/jira/browse/HIVE-14294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385452#comment-15385452 ] Cheng Lian commented on HIVE-14294: --- Hit this issue while investigating SPARK-16632. >

[jira] [Commented] (SPARK-16632) Vectorized parquet reader fails to read certain fields from Hive tables

[ https://issues.apache.org/jira/browse/SPARK-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385451#comment-15385451 ] Cheng Lian commented on SPARK-16632: Discussed with [~yhuai] after merging [PR

[jira] [Updated] (SPARK-16632) Vectorized parquet reader fails to read certain fields from Hive tables

[ https://issues.apache.org/jira/browse/SPARK-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16632: --- Assignee: Marcelo Vanzin > Vectorized parquet reader fails to read certain fields from Hive tables >

[jira] [Commented] (SPARK-16632) Vectorized parquet reader fails to read certain fields from Hive tables

[ https://issues.apache.org/jira/browse/SPARK-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385399#comment-15385399 ] Cheng Lian commented on SPARK-16632: [~vanzin] Did you post the wrong stack trace? This issue is

[jira] [Resolved] (SPARK-16632) Vectorized parquet reader fails to read certain fields from Hive tables

[ https://issues.apache.org/jira/browse/SPARK-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16632. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14272

[jira] [Created] (HIVE-14294) HiveSchemaConverter for Parquet doesn't translate TINYINT and SMALLINT into proper Parquet types

Cheng Lian created HIVE-14294: - Summary: HiveSchemaConverter for Parquet doesn't translate TINYINT and SMALLINT into proper Parquet types Key: HIVE-14294 URL: https://issues.apache.org/jira/browse/HIVE-14294

[jira] [Updated] (SPARK-16633) lag/lead does not return the default value when the offset row does not exist

[ https://issues.apache.org/jira/browse/SPARK-16633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16633: --- Attachment: window_function_bug.html JIRA went down right before [~yhuai] tried to upload the

[jira] [Commented] (SPARK-16576) Move plan SQL generation code from SQLBuilder into logical operators

[ https://issues.apache.org/jira/browse/SPARK-16576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381969#comment-15381969 ] Cheng Lian commented on SPARK-16576: [~rxin] I wrote the first version of the {{SQLBuilder}}. The

[jira] [Resolved] (SPARK-16529) SQLTestUtils.withTempDatabase should set `default` database before dropping

[ https://issues.apache.org/jira/browse/SPARK-16529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16529. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 14184

[jira] [Updated] (SPARK-16529) SQLTestUtils.withTempDatabase should set `default` database before dropping

[ https://issues.apache.org/jira/browse/SPARK-16529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16529: --- Assignee: Dongjoon Hyun > SQLTestUtils.withTempDatabase should set `default` database before

[jira] [Resolved] (SPARK-16448) RemoveAliasOnlyProject should not remove alias with metadata

[ https://issues.apache.org/jira/browse/SPARK-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16448. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14106

[jira] [Updated] (SPARK-16343) Improve the PushDownPredicate rule to pushdown predicates currectly in non-deterministic condition

[ https://issues.apache.org/jira/browse/SPARK-16343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16343: --- Assignee: Jiang Xingbo > Improve the PushDownPredicate rule to pushdown predicates currectly in >

[jira] [Updated] (SPARK-16343) Improve the PushDownPredicate rule to pushdown predicates currectly in non-deterministic condition

[ https://issues.apache.org/jira/browse/SPARK-16343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16343: --- Affects Version/s: 2.0.0 > Improve the PushDownPredicate rule to pushdown predicates currectly in >

[jira] [Resolved] (SPARK-16343) Improve the PushDownPredicate rule to pushdown predicates currectly in non-deterministic condition

[ https://issues.apache.org/jira/browse/SPARK-16343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16343. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14012

[jira] [Resolved] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16303. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 14119

[jira] [Resolved] (SPARK-16381) Update SQL examples and programming guide for R language binding

[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16381. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 14082

[jira] [Comment Edited] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369481#comment-15369481 ] Cheng Lian edited comment on SPARK-16344 at 7/10/16 8:07 AM: - Thanks to

[jira] [Commented] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369481#comment-15369481 ] Cheng Lian commented on SPARK-16344: Thanks to [~rdblue]'s comment about why there're two different

[jira] [Commented] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369032#comment-15369032 ] Cheng Lian commented on SPARK-16344: I was re-thinking about [~rdblue]'s comment above, and tried to

[jira] [Updated] (PARQUET-655) The LogicalTypes.md link in README.md points to the old Parquet GitHub repository

[ https://issues.apache.org/jira/browse/PARQUET-655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-655: --- Component/s: parquet-format > The LogicalTypes.md link in README.md points to the old Parquet GitHub

[jira] [Created] (PARQUET-655) The LogicalTypes.md link in README.md points to the old Parquet GitHub repository

Cheng Lian created PARQUET-655: -- Summary: The LogicalTypes.md link in README.md points to the old Parquet GitHub repository Key: PARQUET-655 URL: https://issues.apache.org/jira/browse/PARQUET-655

[jira] [Comment Edited] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367345#comment-15367345 ] Cheng Lian edited comment on SPARK-16303 at 7/8/16 7:28 AM: Thanks for

[jira] [Comment Edited] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367345#comment-15367345 ] Cheng Lian edited comment on SPARK-16303 at 7/8/16 7:27 AM: Thanks for

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367345#comment-15367345 ] Cheng Lian commented on SPARK-16303: Thanks for working on this! I'd suggest to send out the PR first

[jira] [Created] (PARQUET-654) Make record-level filtering optional

Cheng Lian created PARQUET-654: -- Summary: Make record-level filtering optional Key: PARQUET-654 URL: https://issues.apache.org/jira/browse/PARQUET-654 Project: Parquet Issue Type: Improvement

[jira] [Comment Edited] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366069#comment-15366069 ] Cheng Lian edited comment on SPARK-16344 at 7/8/16 12:12 AM: - Thanks for the

[jira] [Commented] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366069#comment-15366069 ] Cheng Lian commented on SPARK-16344: Thanks for the detailed response! Spark SQL also has two

[jira] [Resolved] (SPARK-16400) Remove InSet filter pushdown from Parquet

[ https://issues.apache.org/jira/browse/SPARK-16400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16400. Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 2.1.0 Resolved by

[jira] [Commented] (SPARK-16381) Update SQL examples and programming guide for R language binding

[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365777#comment-15365777 ] Cheng Lian commented on SPARK-16381: For a specific release, usually, we only make a schedule for the

[jira] [Commented] (SPARK-16380) Update SQL examples and programming guide for Python language binding

[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365591#comment-15365591 ] Cheng Lian commented on SPARK-16380: [~wm624] Considering 2.0.0 RC2 has already been cut, it's

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365590#comment-15365590 ] Cheng Lian commented on SPARK-16303: [~aokolnychyi] Considering 2.0.0 RC2 has already been cut, it's

[jira] [Commented] (SPARK-16381) Update SQL examples and programming guide for R language binding

[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365583#comment-15365583 ] Cheng Lian commented on SPARK-16381: Thanks for volunteering! I've assigned this ticket to you.

[jira] [Updated] (SPARK-16381) Update SQL examples and programming guide for R language binding

[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16381: --- Assignee: Xin Ren > Update SQL examples and programming guide for R language binding >

[jira] [Commented] (SPARK-16380) Update SQL examples and programming guide for Python language binding

[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365579#comment-15365579 ] Cheng Lian commented on SPARK-16380: I just noticed that I put "Scala" into the JIRA ticket title by

[jira] [Updated] (SPARK-16380) Update SQL examples and programming guide for Python language binding

[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16380: --- Summary: Update SQL examples and programming guide for Python language binding (was: Update SQL

[jira] [Resolved] (SPARK-16388) Remove spark.sql.nativeView and spark.sql.nativeView.canonical config

[ https://issues.apache.org/jira/browse/SPARK-16388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16388. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14061

[jira] [Updated] (SPARK-16381) Update SQL examples and programming guide for R language binding

[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16381: --- Labels: (was: Starter) > Update SQL examples and programming guide for R language binding >

[jira] [Updated] (SPARK-16381) Update SQL examples and programming guide for R language binding

[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16381: --- Labels: Starter (was: ) > Update SQL examples and programming guide for R language binding >

[jira] [Updated] (SPARK-16381) Update SQL examples and programming guide for R language binding

[ https://issues.apache.org/jira/browse/SPARK-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16381: --- Description: Please follow guidelines listed in this SPARK-16303

[jira] [Commented] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364057#comment-15364057 ] Cheng Lian commented on SPARK-16380: I've assigned this ticket to you. > Update SQL examples and

[jira] [Updated] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16380: --- Assignee: Miao Wang > Update SQL examples and programming guide for Scala Python language binding >

[jira] [Commented] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15364055#comment-15364055 ] Cheng Lian commented on SPARK-16380: Thanks for volunteering! Following guidelines listed in

[jira] [Updated] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16380: --- Description: Please follow guidelines listed in this SPARK-16303

[jira] [Updated] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

[ https://issues.apache.org/jira/browse/SPARK-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16380: --- Description: Please follow guidelines listed > Update SQL examples and programming guide for Scala

[jira] [Updated] (PARQUET-651) Parquet-avro fails to decode array of record with a single field name "element" correctly

[ https://issues.apache.org/jira/browse/PARQUET-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-651: --- Affects Version/s: 1.9.0 > Parquet-avro fails to decode array of record with a single field name >

[jira] [Resolved] (SPARK-16330) Null pointer getting count from avro file in mesos distributed

[ https://issues.apache.org/jira/browse/SPARK-16330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16330. Resolution: Invalid I'm resolving this issue as invalid since it's actually a spark-avro bug. >

[jira] [Commented] (SPARK-16330) Null pointer getting count from avro file in mesos distributed

[ https://issues.apache.org/jira/browse/SPARK-16330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362678#comment-15362678 ] Cheng Lian commented on SPARK-16330: Please find the root cause analysis in the comment area:

[jira] [Assigned] (SPARK-16330) Null pointer getting count from avro file in mesos distributed

[ https://issues.apache.org/jira/browse/SPARK-16330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-16330: -- Assignee: Cheng Lian > Null pointer getting count from avro file in mesos distributed >

[jira] [Updated] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16303: --- Assignee: Anton Okolnychyi (was: Cheng Lian) > Update SQL examples and programming guide for Scala

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362625#comment-15362625 ] Cheng Lian commented on SPARK-16303: Just assigned this ticket to you. > Update SQL examples and

[jira] [Created] (SPARK-16380) Update SQL examples and programming guide for Scala Python language binding

Cheng Lian created SPARK-16380: -- Summary: Update SQL examples and programming guide for Scala Python language binding Key: SPARK-16380 URL: https://issues.apache.org/jira/browse/SPARK-16380 Project:

[jira] [Created] (SPARK-16381) Update SQL examples and programming guide for R language binding

Cheng Lian created SPARK-16381: -- Summary: Update SQL examples and programming guide for R language binding Key: SPARK-16381 URL: https://issues.apache.org/jira/browse/SPARK-16381 Project: Spark

[jira] [Updated] (SPARK-16303) Update SQL examples and programming guide

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16303: --- Summary: Update SQL examples and programming guide (was: Update SQL examples and programming

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362621#comment-15362621 ] Cheng Lian commented on SPARK-16303: Please feel free to split the task. I'm going to narrow scope of

[jira] [Updated] (SPARK-16303) Update SQL examples and programming guide for Scala and Java language bindings

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16303: --- Summary: Update SQL examples and programming guide for Scala and Java language bindings (was:

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362579#comment-15362579 ] Cheng Lian commented on SPARK-16303: Here's the aforementioned WIP branch, which only contains

[jira] [Commented] (SPARK-16303) Update SQL examples and programming guide

[ https://issues.apache.org/jira/browse/SPARK-16303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15362575#comment-15362575 ] Cheng Lian commented on SPARK-16303: Sure, thanks for volunteering! Actually, I've started working on

[jira] [Updated] (SPARK-16360) Speed up SQL query performance by removing redundant `executePlan` call in `Dataset`

[ https://issues.apache.org/jira/browse/SPARK-16360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16360: --- Assignee: Dongjoon Hyun > Speed up SQL query performance by removing redundant `executePlan` call in

[jira] [Resolved] (SPARK-16360) Speed up SQL query performance by removing redundant `executePlan` call in `Dataset`

[ https://issues.apache.org/jira/browse/SPARK-16360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16360. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14044

[jira] [Resolved] (SPARK-15198) Support for filter push down for boolean types in ORC

[ https://issues.apache.org/jira/browse/SPARK-15198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15198. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 12972

[jira] [Updated] (SPARK-15198) Support for filter push down for boolean types in ORC

[ https://issues.apache.org/jira/browse/SPARK-15198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15198: --- Assignee: Hyukjin Kwon > Support for filter push down for boolean types in ORC >

[jira] [Updated] (PARQUET-651) Parquet-avro fails to decode array of record with a single field name "element" correctly

[ https://issues.apache.org/jira/browse/PARQUET-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-651: --- Description: Found this issue while investigating SPARK-16344. For the following Parquet schema

[jira] [Updated] (SPARK-16208) Add `PropagateEmptyRelation` optimizer

[ https://issues.apache.org/jira/browse/SPARK-16208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16208: --- Assignee: Dongjoon Hyun (was: Apache Spark) > Add `PropagateEmptyRelation` optimizer >

[jira] [Resolved] (SPARK-16208) Add `PropagateEmptyRelation` optimizer

[ https://issues.apache.org/jira/browse/SPARK-16208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16208. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13906

[jira] [Commented] (SPARK-16317) Add file filtering interface for FileFormat

[ https://issues.apache.org/jira/browse/SPARK-16317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15358975#comment-15358975 ] Cheng Lian commented on SPARK-16317: The motivation is to filter out input data files so that

[jira] [Updated] (PARQUET-651) Parquet-avro fails to decode array of record with a single field name "element" correctly

[ https://issues.apache.org/jira/browse/PARQUET-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated PARQUET-651: --- Description: Found this issue while investigating SPARK-16344. For the following Parquet schema

[jira] [Updated] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16344: --- Description: This is a weird corner case. Users may hit this issue if they have a schema that # has

[jira] [Updated] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

[ https://issues.apache.org/jira/browse/SPARK-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16344: --- Description: Array of struct with a single field name "element" can't be decoded from Parquet files

[jira] [Created] (SPARK-16344) Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+

Cheng Lian created SPARK-16344: -- Summary: Array of struct with a single field name "element" can't be decoded from Parquet files written by Spark 1.6+ Key: SPARK-16344 URL:

[jira] [Updated] (SPARK-15820) Add Catalog.refreshTable into python API

[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15820: --- Fix Version/s: (was: 2.0.0) 2.1.0 2.0.1 > Add

[jira] [Updated] (SPARK-15820) Add Catalog.refreshTable into python API

[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15820: --- Assignee: Weichen Xu > Add Catalog.refreshTable into python API >

[jira] [Resolved] (SPARK-15820) Add Catalog.refreshTable into python API

[ https://issues.apache.org/jira/browse/SPARK-15820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15820. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13558

[jira] [Created] (SPARK-16317) Add file filtering interface for FileFormat

Cheng Lian created SPARK-16317: -- Summary: Add file filtering interface for FileFormat Key: SPARK-16317 URL: https://issues.apache.org/jira/browse/SPARK-16317 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-16134) optimizer rules for typed filter

[ https://issues.apache.org/jira/browse/SPARK-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16134. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13846

[jira] [Created] (SPARK-16303) Update SQL examples and programming guide

Cheng Lian created SPARK-16303: -- Summary: Update SQL examples and programming guide Key: SPARK-16303 URL: https://issues.apache.org/jira/browse/SPARK-16303 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-16295) Extract SQL programming guide example snippets from source files instead of hard code them

[ https://issues.apache.org/jira/browse/SPARK-16295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16295: --- Description: Currently, all example snippets in the SQL programming guide are hard-coded, which can

[jira] [Updated] (SPARK-16294) Labelling support for the include_example Jekyll plugin

[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16294: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-16295 > Labelling support for the

[jira] [Created] (SPARK-16295) Extract SQL programming guide example snippets from source files instead of hard code them

Cheng Lian created SPARK-16295: -- Summary: Extract SQL programming guide example snippets from source files instead of hard code them Key: SPARK-16295 URL: https://issues.apache.org/jira/browse/SPARK-16295

[jira] [Updated] (SPARK-16294) Labelling support for the include_example Jekyll plugin

[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16294: --- Description: Part of the Spark programming guide pages are using the {{include_example}} Jekyll

[jira] [Updated] (SPARK-16294) Labelling support for the include_example Jekyll plugin

[ https://issues.apache.org/jira/browse/SPARK-16294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16294: --- Description: Part of the Spark programming guide pages are using the {{include_example}} Jekyll

[jira] [Created] (SPARK-16294) Labelling support for the include_example Jekyll plugin

Cheng Lian created SPARK-16294: -- Summary: Labelling support for the include_example Jekyll plugin Key: SPARK-16294 URL: https://issues.apache.org/jira/browse/SPARK-16294 Project: Spark Issue

[jira] [Created] (SPARK-16291) Invalid aggregate functions like MAX(COUNT(*)) are not captured by CheckAnalysis

Cheng Lian created SPARK-16291: -- Summary: Invalid aggregate functions like MAX(COUNT(*)) are not captured by CheckAnalysis Key: SPARK-16291 URL: https://issues.apache.org/jira/browse/SPARK-16291

[jira] [Resolved] (SPARK-16100) Aggregator fails with Tungsten error when complex types are used for results and partial sum

[ https://issues.apache.org/jira/browse/SPARK-16100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16100. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Resolved] (SPARK-16221) Redirect Parquet JUL logger via SLF4J for WRITE operations

[ https://issues.apache.org/jira/browse/SPARK-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16221. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Updated] (SPARK-16221) Redirect Parquet JUL logger via SLF4J for WRITE operations

[ https://issues.apache.org/jira/browse/SPARK-16221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16221: --- Assignee: Dongjoon Hyun > Redirect Parquet JUL logger via SLF4J for WRITE operations >

[jira] [Commented] (SPARK-16164) CombineFilters should keep the ordering in the logical plan

[ https://issues.apache.org/jira/browse/SPARK-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15351244#comment-15351244 ] Cheng Lian commented on SPARK-16164: I'm not saying that we should make this explicit, but our

[jira] [Updated] (SPARK-10591) False negative in QueryTest.checkAnswer

[ https://issues.apache.org/jira/browse/SPARK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10591: --- Fix Version/s: (was: 2.0.0) 2.1.0 2.0.1 > False negative

[jira] [Updated] (SPARK-10591) False negative in QueryTest.checkAnswer

[ https://issues.apache.org/jira/browse/SPARK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10591: --- Assignee: Dongjoon Hyun > False negative in QueryTest.checkAnswer >

[jira] [Resolved] (SPARK-10591) False negative in QueryTest.checkAnswer

[ https://issues.apache.org/jira/browse/SPARK-10591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-10591. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13913

[jira] [Commented] (SPARK-16164) CombineFilters should keep the ordering in the logical plan

[ https://issues.apache.org/jira/browse/SPARK-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347701#comment-15347701 ] Cheng Lian commented on SPARK-16164: I'm posting a summary of our offline and GitHub discussion about

[jira] [Resolved] (SPARK-16165) Fix the update logic for InMemoryTableScanExec.readBatches accumulator

[ https://issues.apache.org/jira/browse/SPARK-16165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16165. Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 Issue resolved by pull

[jira] [Updated] (SPARK-16165) Fix the update logic for InMemoryTableScanExec.readBatches accumulator

[ https://issues.apache.org/jira/browse/SPARK-16165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16165: --- Assignee: Dongjoon Hyun > Fix the update logic for InMemoryTableScanExec.readBatches accumulator >

[jira] [Updated] (SPARK-13709) Spark unable to decode Avro when partitioned

[ https://issues.apache.org/jira/browse/SPARK-13709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13709: --- Assignee: Cheng Lian > Spark unable to decode Avro when partitioned >

[jira] [Updated] (SPARK-13572) HiveContext reads avro Hive tables incorrectly

[ https://issues.apache.org/jira/browse/SPARK-13572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-13572: --- Description: I am using PySpark to read avro-based tables from Hive and while the avro tables can

[jira] [Updated] (SPARK-16100) Aggregator fails with Tungsten error when complex types are used for results and partial sum

[ https://issues.apache.org/jira/browse/SPARK-16100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16100: --- Description: I get a similar error when using complex types in Aggregator. Not sure if this is the

[jira] [Updated] (SPARK-16100) Aggregator fails with Tungsten error when complex types are used for results and partial sum

[ https://issues.apache.org/jira/browse/SPARK-16100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16100: --- Assignee: Wenchen Fan > Aggregator fails with Tungsten error when complex types are used for results

[jira] [Resolved] (SPARK-16097) Encoders.tuple should handle null object correctly

[ https://issues.apache.org/jira/browse/SPARK-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16097. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13807

[jira] [Updated] (SPARK-16121) ListingFileCatalog does not list in parallel anymore

[ https://issues.apache.org/jira/browse/SPARK-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16121: --- Assignee: Yin Huai > ListingFileCatalog does not list in parallel anymore >

[jira] [Resolved] (SPARK-16121) ListingFileCatalog does not list in parallel anymore

[ https://issues.apache.org/jira/browse/SPARK-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-16121. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13830

[jira] [Commented] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

[ https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341754#comment-15341754 ] Cheng Lian commented on SPARK-16032: [~rdblue], I also migrated some test cases from your PR so that

<    1   2   3   4   5   6   7   8   9   10   >