[jira] [Assigned] (SPARK-35548) Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-16 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-35548: --- Assignee: Qi Zhu (was: zhuqi) > Handling new attempt has started error

[jira] [Updated] (SPARK-36525) DS V2 Index Support

2021-08-16 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-36525: --- Description: Many data sources support index to improvement query performance. In order to take

[jira] [Updated] (SPARK-36525) DS V2 Index Support

2021-08-16 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-36525: --- Description: Many data sources support index to improvement query performance. In order to take

[jira] [Commented] (SPARK-35028) ANSI mode: disallow group by aliases

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400144#comment-17400144 ] Apache Spark commented on SPARK-35028: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-36518) spark should support distribute directory to cluster

2021-08-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400133#comment-17400133 ] Hyukjin Kwon commented on SPARK-36518: -- I think you can use --archive for this purpose. > spark

[jira] [Resolved] (SPARK-36530) Avoid finalizing when there's no push at all in a shuffle

2021-08-16 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-36530. -- Resolution: Won't Fix > Avoid finalizing when there's no push at all in a shuffle >

[jira] [Commented] (SPARK-36525) DS V2 Index Support

2021-08-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400127#comment-17400127 ] Hyukjin Kwon commented on SPARK-36525: -- would you mind outlining the interface? > DS V2 Index

[jira] [Commented] (SPARK-36505) Improve test coverage for frame.py

2021-08-16 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400121#comment-17400121 ] Haejoon Lee commented on SPARK-36505: - I'm working on this > Improve test coverage for frame.py >

[jira] [Updated] (SPARK-33701) Adaptive shuffle merge finalization for push-based shuffle

2021-08-16 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Shen updated SPARK-33701: - Description: SPARK-32920 implements a simple approach for shuffle merge finalization, which

[jira] [Commented] (SPARK-36530) Avoid finalizing when there's no push at all in a shuffle

2021-08-16 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400122#comment-17400122 ] Min Shen commented on SPARK-36530: -- Expanded the description of SPARK-33701 to include this scenario as

[jira] [Created] (SPARK-36531) Improve test coverage for data_type_ops/*

2021-08-16 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-36531: --- Summary: Improve test coverage for data_type_ops/* Key: SPARK-36531 URL: https://issues.apache.org/jira/browse/SPARK-36531 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-36530) Avoid finalizing when there's no push at all in a shuffle

2021-08-16 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400118#comment-17400118 ] wuyi commented on SPARK-36530: -- SGTM. Could you update SPARK-33701 to include this part? Then, I'll close

[jira] [Commented] (SPARK-36368) Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400116#comment-17400116 ] Apache Spark commented on SPARK-36368: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36368) Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36368: Assignee: (was: Apache Spark) > Fix CategoricalOps.astype to follow pandas 1.3 >

[jira] [Assigned] (SPARK-36368) Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36368: Assignee: Apache Spark > Fix CategoricalOps.astype to follow pandas 1.3 >

[jira] [Commented] (SPARK-36368) Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400114#comment-17400114 ] Apache Spark commented on SPARK-36368: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-36530) Avoid finalizing when there's no push at all in a shuffle

2021-08-16 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400100#comment-17400100 ] Min Shen commented on SPARK-36530: -- You mean every shuffle partition block is larger than that

[jira] [Assigned] (SPARK-36344) Fix some typos in ShuffleBlockPusher class.

2021-08-16 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36344: --- Assignee: Qi Zhu (was: zhuqi) > Fix some typos in ShuffleBlockPusher

[jira] [Commented] (SPARK-35548) Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-16 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400091#comment-17400091 ] Qi Zhu commented on SPARK-35548: Hi [~mridulm80], [~Qi Zhu] is not me just the same name, [~zhuqi] is

[jira] [Commented] (SPARK-36530) Avoid finalizing when there's no push at all in a shuffle

2021-08-16 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400089#comment-17400089 ] wuyi commented on SPARK-36530: -- cc [~mridulm80] [~mshen] any thoughts? > Avoid finalizing when there's no

[jira] [Created] (SPARK-36530) Avoid finalizing when there's no push at all in a shuffle

2021-08-16 Thread wuyi (Jira)
wuyi created SPARK-36530: Summary: Avoid finalizing when there's no push at all in a shuffle Key: SPARK-36530 URL: https://issues.apache.org/jira/browse/SPARK-36530 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-35548) Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-16 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-35548: --- Assignee: zhuqi > Handling new attempt has started error message in

[jira] [Commented] (SPARK-36470) Implement `CategoricalIndex.map` and `DatetimeIndx.map`

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400077#comment-17400077 ] Apache Spark commented on SPARK-36470: -- User 'xinrong-databricks' has created a pull request for

[jira] [Commented] (SPARK-36470) Implement `CategoricalIndex.map` and `DatetimeIndx.map`

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400076#comment-17400076 ] Apache Spark commented on SPARK-36470: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-36470) Implement `CategoricalIndex.map` and `DatetimeIndx.map`

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36470: Assignee: Apache Spark > Implement `CategoricalIndex.map` and `DatetimeIndx.map` >

[jira] [Assigned] (SPARK-36470) Implement `CategoricalIndex.map` and `DatetimeIndx.map`

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36470: Assignee: (was: Apache Spark) > Implement `CategoricalIndex.map` and

[jira] [Updated] (SPARK-36470) Implement `CategoricalIndex.map` and `DatetimeIndx.map`

2021-08-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36470: - Summary: Implement `CategoricalIndex.map` and `DatetimeIndx.map` (was: Implement

[jira] [Issue Comment Deleted] (SPARK-36471) Implement MultiIndex.map

2021-08-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36471: - Comment: was deleted (was: Map is not implemented for DatetimeIndex in pandas yet.) >

[jira] [Reopened] (SPARK-36471) Implement MultiIndex.map

2021-08-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reopened SPARK-36471: -- > Implement MultiIndex.map > > > Key: SPARK-36471 >

[jira] [Updated] (SPARK-36471) Implement MultiIndex.map

2021-08-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36471: - Summary: Implement MultiIndex.map (was: Implement DatetimeIndex.map) > Implement

[jira] [Updated] (SPARK-36470) Implement CategoricalIndex.map and DatetimeIndx.map

2021-08-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36470: - Summary: Implement CategoricalIndex.map and DatetimeIndx.map (was: Implement

[jira] [Resolved] (SPARK-36457) Review and fix issues in API docs

2021-08-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36457. -- Resolution: Duplicate Actually let's address this as a followup each ticket. > Review and

[jira] [Commented] (SPARK-36052) Introduce pending pod limit for Spark on K8s

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400047#comment-17400047 ] Apache Spark commented on SPARK-36052: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-36052) Introduce pending pod limit for Spark on K8s

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400046#comment-17400046 ] Apache Spark commented on SPARK-36052: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-36052) Introduce pending pod limit for Spark on K8s

2021-08-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36052: -- Affects Version/s: 3.0.3 > Introduce pending pod limit for Spark on K8s >

[jira] [Updated] (SPARK-36052) Introduce pending pod limit for Spark on K8s

2021-08-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36052: -- Affects Version/s: 3.1.2 > Introduce pending pod limit for Spark on K8s >

[jira] [Updated] (SPARK-36052) Introduce pending pod limit for Spark on K8s

2021-08-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36052: -- Fix Version/s: 3.2.0 > Introduce pending pod limit for Spark on K8s >

[jira] [Updated] (SPARK-36052) Introduce pending pod limit for Spark on K8s

2021-08-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-36052: -- Affects Version/s: 3.2.0 > Introduce pending pod limit for Spark on K8s >

[jira] [Commented] (SPARK-35548) Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-16 Thread Qi Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400029#comment-17400029 ] Qi Zhu commented on SPARK-35548: cc [~mridulm80] My Apache id is: zhuqi full name: Qi Zhu Thanks. >

[jira] [Resolved] (SPARK-36521) Disallow comparison between Interval and String

2021-08-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36521. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33750

[jira] [Resolved] (SPARK-36041) Introduce the RocksDBStateStoreProvider in the programming guide

2021-08-16 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36041. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33683

[jira] [Assigned] (SPARK-36041) Introduce the RocksDBStateStoreProvider in the programming guide

2021-08-16 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-36041: --- Assignee: Yuanjian Li > Introduce the RocksDBStateStoreProvider in the programming guide >

[jira] [Resolved] (SPARK-35548) Handling new attempt has started error message in BlockPushErrorHandler in client

2021-08-16 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-35548. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Updated] (SPARK-36528) Implement lazy decoding for the vectorized Parquet reader

2021-08-16 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-36528: - Description: Currently Spark first decode (e.g., RLE/bit-packed, PLAIN) into column vector and then

[jira] [Updated] (SPARK-36527) Implement lazy materialization for the vectorized Parquet reader

2021-08-16 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-36527: - Description: At the moment the Parquet vectorized reader will eagerly decode all the columns that are

[jira] [Created] (SPARK-36529) Decouple CPU with IO work in vectorized Parquet reader

2021-08-16 Thread Chao Sun (Jira)
Chao Sun created SPARK-36529: Summary: Decouple CPU with IO work in vectorized Parquet reader Key: SPARK-36529 URL: https://issues.apache.org/jira/browse/SPARK-36529 Project: Spark Issue Type:

[jira] [Created] (SPARK-36528) Implement lazy decoding for the vectorized Parquet reader

2021-08-16 Thread Chao Sun (Jira)
Chao Sun created SPARK-36528: Summary: Implement lazy decoding for the vectorized Parquet reader Key: SPARK-36528 URL: https://issues.apache.org/jira/browse/SPARK-36528 Project: Spark Issue

[jira] [Created] (SPARK-36527) Implement lazy materialization for the vectorized Parquet reader

2021-08-16 Thread Chao Sun (Jira)
Chao Sun created SPARK-36527: Summary: Implement lazy materialization for the vectorized Parquet reader Key: SPARK-36527 URL: https://issues.apache.org/jira/browse/SPARK-36527 Project: Spark

[jira] [Assigned] (SPARK-36526) Add supportsIndex interface

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36526: Assignee: (was: Apache Spark) > Add supportsIndex interface >

[jira] [Assigned] (SPARK-36526) Add supportsIndex interface

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36526: Assignee: Apache Spark > Add supportsIndex interface > --- > >

[jira] [Commented] (SPARK-36526) Add supportsIndex interface

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399916#comment-17399916 ] Apache Spark commented on SPARK-36526: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Resolved] (SPARK-36469) Implement Index.map

2021-08-16 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36469. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng (was: Apache Spark) Resolution:

[jira] [Created] (SPARK-36526) Add supportsIndex interface

2021-08-16 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-36526: -- Summary: Add supportsIndex interface Key: SPARK-36526 URL: https://issues.apache.org/jira/browse/SPARK-36526 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36525) DS V2 Index Support

2021-08-16 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-36525: -- Summary: DS V2 Index Support Key: SPARK-36525 URL: https://issues.apache.org/jira/browse/SPARK-36525 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-33331) Limit the number of pending blocks in memory and store blocks that collide

2021-08-16 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399845#comment-17399845 ] Min Shen commented on SPARK-1: -- With the change in SPARK-36423, I think this issue is further

[jira] [Commented] (SPARK-35036) Improve push based shuffle to work with AQE by fetching partial map indexes for a reduce partition

2021-08-16 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399843#comment-17399843 ] Min Shen commented on SPARK-35036: -- Not sure if this is fixable, given the reasons you already

[jira] [Resolved] (SPARK-36484) Handle Stale block fetch failure on the client side by not retrying the requests

2021-08-16 Thread Venkata krishnan Sowrirajan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata krishnan Sowrirajan resolved SPARK-36484. - Resolution: Won't Fix Thinking more about it, it doesn't seem

[jira] [Assigned] (SPARK-36524) Add common class/trait for ANSI interval types

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36524: Assignee: Max Gekk (was: Apache Spark) > Add common class/trait for ANSI interval types

[jira] [Assigned] (SPARK-36524) Add common class/trait for ANSI interval types

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36524: Assignee: Apache Spark (was: Max Gekk) > Add common class/trait for ANSI interval types

[jira] [Commented] (SPARK-36524) Add common class/trait for ANSI interval types

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399827#comment-17399827 ] Apache Spark commented on SPARK-36524: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36401) Implement Series.cov

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36401: Assignee: Apache Spark > Implement Series.cov > > >

[jira] [Commented] (SPARK-36401) Implement Series.cov

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399822#comment-17399822 ] Apache Spark commented on SPARK-36401: -- User 'dgd-contributor' has created a pull request for this

[jira] [Assigned] (SPARK-36401) Implement Series.cov

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36401: Assignee: (was: Apache Spark) > Implement Series.cov > > >

[jira] [Resolved] (SPARK-32210) Failed to serialize large MapStatuses

2021-08-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32210. --- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 33721

[jira] [Assigned] (SPARK-32210) Failed to serialize large MapStatuses

2021-08-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32210: - Assignee: Kazuyuki Tanimura > Failed to serialize large MapStatuses >

[jira] [Created] (SPARK-36524) Add common class/trait for ANSI interval types

2021-08-16 Thread Max Gekk (Jira)
Max Gekk created SPARK-36524: Summary: Add common class/trait for ANSI interval types Key: SPARK-36524 URL: https://issues.apache.org/jira/browse/SPARK-36524 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36523) Casting invalid string to day-time interval should return null

2021-08-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-36523: --- Parent: (was: SPARK-27790) Issue Type: Bug (was: Sub-task) > Casting invalid

[jira] [Updated] (SPARK-36522) Casting invalid string to year-month interval should return null

2021-08-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-36522: --- Parent: (was: SPARK-27790) Issue Type: Bug (was: Sub-task) > Casting invalid

[jira] [Resolved] (SPARK-36523) Casting invalid string to day-time interval should return null

2021-08-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36523. Resolution: Won't Fix > Casting invalid string to day-time interval should return null >

[jira] [Resolved] (SPARK-36522) Casting invalid string to year-month interval should return null

2021-08-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-36522. Resolution: Won't Fix > Casting invalid string to year-month interval should return null

[jira] [Commented] (SPARK-36522) Casting invalid string to year-month interval should return null

2021-08-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399809#comment-17399809 ] Gengliang Wang commented on SPARK-36522: The new interval type is following ANSI SQL completely.

[jira] [Commented] (SPARK-36523) Casting invalid string to day-time interval should return null

2021-08-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399810#comment-17399810 ] Gengliang Wang commented on SPARK-36523: The new interval type is following ANSI SQL completely.

[jira] [Resolved] (SPARK-36418) Use CAST in parsing of dates/timestamps with default pattern

2021-08-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36418. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33709

[jira] [Assigned] (SPARK-36418) Use CAST in parsing of dates/timestamps with default pattern

2021-08-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36418: --- Assignee: Max Gekk > Use CAST in parsing of dates/timestamps with default pattern >

[jira] [Resolved] (SPARK-36374) Push-based shuffle documentation

2021-08-16 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-36374. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-36374) Push-based shuffle documentation

2021-08-16 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-36374: --- Assignee: Venkata krishnan Sowrirajan > Push-based shuffle documentation >

[jira] [Commented] (SPARK-33807) Data Source V2: Remove read specific distributions

2021-08-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399779#comment-17399779 ] Wenchen Fan commented on SPARK-33807: - Sounds fine to me > Data Source V2: Remove read specific

[jira] [Commented] (SPARK-33807) Data Source V2: Remove read specific distributions

2021-08-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399778#comment-17399778 ] Gengliang Wang commented on SPARK-33807: [~cloud_fan]what do you think? > Data Source V2:

[jira] [Assigned] (SPARK-36522) Casting invalid string to year-month interval should return null

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36522: Assignee: Apache Spark (was: Gengliang Wang) > Casting invalid string to year-month

[jira] [Commented] (SPARK-36522) Casting invalid string to year-month interval should return null

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399766#comment-17399766 ] Apache Spark commented on SPARK-36522: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-36522) Casting invalid string to year-month interval should return null

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36522: Assignee: Gengliang Wang (was: Apache Spark) > Casting invalid string to year-month

[jira] [Commented] (SPARK-36522) Casting invalid string to year-month interval should return null

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399767#comment-17399767 ] Apache Spark commented on SPARK-36522: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-36523) Casting invalid string to day-time interval should return null

2021-08-16 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-36523: -- Summary: Casting invalid string to day-time interval should return null Key: SPARK-36523 URL: https://issues.apache.org/jira/browse/SPARK-36523 Project: Spark

[jira] [Created] (SPARK-36522) Casting invalid string to year-month interval should return null

2021-08-16 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-36522: -- Summary: Casting invalid string to year-month interval should return null Key: SPARK-36522 URL: https://issues.apache.org/jira/browse/SPARK-36522 Project: Spark

[jira] [Assigned] (SPARK-36521) Disallow comparison between Interval and String

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36521: Assignee: Apache Spark (was: Gengliang Wang) > Disallow comparison between Interval and

[jira] [Assigned] (SPARK-36521) Disallow comparison between Interval and String

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36521: Assignee: Gengliang Wang (was: Apache Spark) > Disallow comparison between Interval and

[jira] [Commented] (SPARK-36521) Disallow comparison between Interval and String

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399655#comment-17399655 ] Apache Spark commented on SPARK-36521: -- User 'gengliangwang' has created a pull request for this

[jira] [Created] (SPARK-36521) Disallow comparison between Interval and String

2021-08-16 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-36521: -- Summary: Disallow comparison between Interval and String Key: SPARK-36521 URL: https://issues.apache.org/jira/browse/SPARK-36521 Project: Spark Issue

[jira] [Updated] (SPARK-36520) KinesisInitialPosition interface should be public

2021-08-16 Thread Noam Tamim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noam Tamim updated SPARK-36520: --- Description: In `org.apache.spark.streaming.kinesis` there's a Java interface

[jira] [Created] (SPARK-36520) KinesisInitialPosition interface should be public

2021-08-16 Thread Noam Tamim (Jira)
Noam Tamim created SPARK-36520: -- Summary: KinesisInitialPosition interface should be public Key: SPARK-36520 URL: https://issues.apache.org/jira/browse/SPARK-36520 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31460) spark-sql-kafka source in spark 2.4.4 causes reading stream failure frequently

2021-08-16 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399594#comment-17399594 ] Gabor Somogyi commented on SPARK-31460: --- When such thing happens then Kafka connector is super

[jira] [Commented] (SPARK-36519) Store the RocksDB format in the checkpoint for a streaming query

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399567#comment-17399567 ] Apache Spark commented on SPARK-36519: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36519) Store the RocksDB format in the checkpoint for a streaming query

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36519: Assignee: Shixiong Zhu (was: Apache Spark) > Store the RocksDB format in the checkpoint

[jira] [Assigned] (SPARK-36519) Store the RocksDB format in the checkpoint for a streaming query

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36519: Assignee: Apache Spark (was: Shixiong Zhu) > Store the RocksDB format in the checkpoint

[jira] [Created] (SPARK-36519) Store the RocksDB format in the checkpoint for a streaming query

2021-08-16 Thread Shixiong Zhu (Jira)
Shixiong Zhu created SPARK-36519: Summary: Store the RocksDB format in the checkpoint for a streaming query Key: SPARK-36519 URL: https://issues.apache.org/jira/browse/SPARK-36519 Project: Spark

[jira] [Resolved] (SPARK-36498) Reorder inner fields of the input query in byName V2 write

2021-08-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36498. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33728

[jira] [Assigned] (SPARK-36498) Reorder inner fields of the input query in byName V2 write

2021-08-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36498: --- Assignee: Wenchen Fan > Reorder inner fields of the input query in byName V2 write >

[jira] [Resolved] (SPARK-36491) Make from_json/to_json to handle timestamp_ntz type properly

2021-08-16 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36491. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33742

[jira] [Created] (SPARK-36518) spark should support distribute directory to cluster

2021-08-16 Thread YuanGuanhu (Jira)
YuanGuanhu created SPARK-36518: -- Summary: spark should support distribute directory to cluster Key: SPARK-36518 URL: https://issues.apache.org/jira/browse/SPARK-36518 Project: Spark Issue Type:

[jira] [Commented] (SPARK-36516) Add cache for Orc Metadata

2021-08-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399542#comment-17399542 ] Apache Spark commented on SPARK-36516: -- User 'LuciferYang' has created a pull request for this

  1   2   >