[jira] [Commented] (SPARK-37228) Implement DataFrame.mapInArrow in Python

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439923#comment-17439923 ] Apache Spark commented on SPARK-37228: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-37228) Implement DataFrame.mapInArrow in Python

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37228: Assignee: (was: Apache Spark) > Implement DataFrame.mapInArrow in Python >

[jira] [Assigned] (SPARK-37228) Implement DataFrame.mapInArrow in Python

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37228: Assignee: Apache Spark > Implement DataFrame.mapInArrow in Python >

[jira] [Created] (SPARK-37230) Document DataFrame.mapInArrow

2021-11-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37230: Summary: Document DataFrame.mapInArrow Key: SPARK-37230 URL: https://issues.apache.org/jira/browse/SPARK-37230 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-30153) Extend data exchange options for vectorized UDF functions with vanilla Arrow serialization

2021-11-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30153. -- Resolution: Duplicate This will be tracked at SPARK-37227 > Extend data exchange options for

[jira] [Commented] (SPARK-37228) Implement DataFrame.mapInArrow in Python

2021-11-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439922#comment-17439922 ] Hyukjin Kwon commented on SPARK-37228: -- I am working on this. > Implement DataFrame.mapInArrow in

[jira] [Created] (SPARK-37229) Implement DataFrame.mapInArrow in Scala

2021-11-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37229: Summary: Implement DataFrame.mapInArrow in Scala Key: SPARK-37229 URL: https://issues.apache.org/jira/browse/SPARK-37229 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-37228) Implement DataFrame.mapInArrow in Python

2021-11-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37228: Summary: Implement DataFrame.mapInArrow in Python Key: SPARK-37228 URL: https://issues.apache.org/jira/browse/SPARK-37228 Project: Spark Issue Type:

[jira] [Updated] (SPARK-37227) DataFrame.mapInArrow

2021-11-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37227: - Description: h2. Background *Usability problem* In Scala APIs, we have added Arrow

[jira] [Created] (SPARK-37227) DataFrame.mapInArrow

2021-11-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-37227: Summary: DataFrame.mapInArrow Key: SPARK-37227 URL: https://issues.apache.org/jira/browse/SPARK-37227 Project: Spark Issue Type: Umbrella

[jira] [Commented] (SPARK-19230) View creation in Derby gets SQLDataException because definition gets very big

2021-11-06 Thread Tal Franji (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439711#comment-17439711 ] Tal Franji commented on SPARK-19230: Ran into the same issue when creating a large view and testing

[jira] [Commented] (SPARK-37226) Filter push down through window

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439710#comment-17439710 ] Apache Spark commented on SPARK-37226: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37226) Filter push down through window

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37226: Assignee: Yuming Wang (was: Apache Spark) > Filter push down through window >

[jira] [Assigned] (SPARK-37226) Filter push down through window

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37226: Assignee: Apache Spark (was: Yuming Wang) > Filter push down through window >

[jira] [Resolved] (SPARK-37220) Do not split input file for Parquet reader with aggregate push down

2021-11-06 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-37220. -- Fix Version/s: 3.3.0 Resolution: Fixed > Do not split input file for Parquet reader with

[jira] [Created] (SPARK-37226) Filter push down through window

2021-11-06 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-37226: --- Summary: Filter push down through window Key: SPARK-37226 URL: https://issues.apache.org/jira/browse/SPARK-37226 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-37225) Read/write dataframes with ANSI intervals from/to Avro files

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37225: Assignee: Max Gekk (was: Apache Spark) > Read/write dataframes with ANSI intervals

[jira] [Commented] (SPARK-37225) Read/write dataframes with ANSI intervals from/to Avro files

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439667#comment-17439667 ] Apache Spark commented on SPARK-37225: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37225) Read/write dataframes with ANSI intervals from/to Avro files

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37225: Assignee: Apache Spark (was: Max Gekk) > Read/write dataframes with ANSI intervals

[jira] [Commented] (SPARK-37225) Read/write dataframes with ANSI intervals from/to Avro files

2021-11-06 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439632#comment-17439632 ] Max Gekk commented on SPARK-37225: -- I am working on this. > Read/write dataframes with ANSI intervals

[jira] [Created] (SPARK-37225) Read/write dataframes with ANSI intervals from/to Avro files

2021-11-06 Thread Max Gekk (Jira)
Max Gekk created SPARK-37225: Summary: Read/write dataframes with ANSI intervals from/to Avro files Key: SPARK-37225 URL: https://issues.apache.org/jira/browse/SPARK-37225 Project: Spark Issue

[jira] [Updated] (SPARK-37225) Read/write dataframes with ANSI intervals from/to Avro files

2021-11-06 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37225: - Description: Implement writing and reading ANSI intervals (year-month and day-time intervals) columns

[jira] [Assigned] (SPARK-37225) Read/write dataframes with ANSI intervals from/to Avro files

2021-11-06 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-37225: Assignee: Max Gekk (was: Kousuke Saruta) > Read/write dataframes with ANSI intervals from/to

[jira] [Assigned] (SPARK-37224) Optimize write path on RocksDB state store provider

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37224: Assignee: (was: Apache Spark) > Optimize write path on RocksDB state store provider

[jira] [Assigned] (SPARK-37224) Optimize write path on RocksDB state store provider

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37224: Assignee: Apache Spark > Optimize write path on RocksDB state store provider >

[jira] [Commented] (SPARK-37224) Optimize write path on RocksDB state store provider

2021-11-06 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439615#comment-17439615 ] Apache Spark commented on SPARK-37224: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-37224) Optimize write path on RocksDB state store provider

2021-11-06 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17439611#comment-17439611 ] Jungtaek Lim commented on SPARK-37224: -- will submit a PR sooner > Optimize write path on RocksDB

[jira] [Created] (SPARK-37224) Optimize write path on RocksDB state store provider

2021-11-06 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-37224: Summary: Optimize write path on RocksDB state store provider Key: SPARK-37224 URL: https://issues.apache.org/jira/browse/SPARK-37224 Project: Spark Issue