[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2021-03-24 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308000#comment-17308000 ] Xinli Shang commented on SPARK-26345: - Yes, it needs some synchronization. I have the modified

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-12-11 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248241#comment-17248241 ] Xinli Shang commented on SPARK-26345: - The Presto and Iceberg effort are not tied to each other. It

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-12-11 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248231#comment-17248231 ] Xinli Shang commented on SPARK-26345: - For the performance, there is an Eng Blog I found online

[jira] [Commented] (SPARK-27733) Upgrade to Avro 1.10.0

2020-09-22 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17200307#comment-17200307 ] Xinli Shang commented on SPARK-27733: - We talked about the Parquet 1.11.0 adoption in Spark in

[jira] [Commented] (SPARK-26345) Parquet support Column indexes

2020-07-21 Thread Xinli Shang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17162063#comment-17162063 ] Xinli Shang commented on SPARK-26345: - [~yumwang][~FelixKJose], you can assign this JIra to me. When

[jira] [Commented] (SPARK-26346) Upgrade parquet to 1.11.0

2019-03-01 Thread Xinli Shang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781953#comment-16781953 ] Xinli Shang commented on SPARK-26346: - +1, [~yumwang] , Is there any pre-testing done on RC4 or RC3? 

[jira] [Resolved] (SPARK-25858) Passing Field Metadata to Parquet

2018-10-27 Thread Xinli Shang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved SPARK-25858. - Resolution: Later It is a little early to open this issue. I will re-open it after the

[jira] [Updated] (SPARK-25858) Passing Field Metadata to Parquet

2018-10-26 Thread Xinli Shang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated SPARK-25858: Description: h1. Problem Statement The Spark WriteSupport class for Parquet is hardcoded to use

[jira] [Updated] (SPARK-25858) Passing Field Metadata to Parquet

2018-10-26 Thread Xinli Shang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated SPARK-25858: Description: h1. Problem Statement The Spark WriteSupport class for Parquet is hardcoded to use

[jira] [Updated] (SPARK-25858) Passing Field Metadata to Parquet

2018-10-26 Thread Xinli Shang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated SPARK-25858: Description: h1. Problem Statement The Spark WriteSupport class for Parquet is hardcoded to use

[jira] [Created] (SPARK-25858) Passing Field Metadata to Parquet

2018-10-26 Thread Xinli Shang (JIRA)
Xinli Shang created SPARK-25858: --- Summary: Passing Field Metadata to Parquet Key: SPARK-25858 URL: https://issues.apache.org/jira/browse/SPARK-25858 Project: Spark Issue Type: New Feature