[jira] [Assigned] (SPARK-41147) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_1042

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41147: Assignee: (was: Apache Spark) > Assign a name to the legacy error class

[jira] [Assigned] (SPARK-41147) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_1042

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41147: Assignee: Apache Spark > Assign a name to the legacy error class _LEGACY_ERROR_TEMP_1042

[jira] [Commented] (SPARK-41147) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_1042

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634211#comment-17634211 ] Apache Spark commented on SPARK-41147: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-40576) Support pandas 1.5.x.

2022-11-14 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634208#comment-17634208 ] Haejoon Lee commented on SPARK-40576: - Let me keep tracking this umbrella JIRA till pandas 1.6.x is

[jira] [Assigned] (SPARK-41127) Implement DataFrame.CreateGlobalView in Python client

2022-11-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41127: Assignee: Rui Wang > Implement DataFrame.CreateGlobalView in Python client >

[jira] [Resolved] (SPARK-41127) Implement DataFrame.CreateGlobalView in Python client

2022-11-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41127. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38642

[jira] [Updated] (SPARK-41147) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_1042

2022-11-14 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-41147: Summary: Assign a name to the legacy error class _LEGACY_ERROR_TEMP_1042 (was: Rename

[jira] [Commented] (SPARK-41147) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_1042

2022-11-14 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634195#comment-17634195 ] Haejoon Lee commented on SPARK-41147: - I'm working on it > Assign a name to the legacy error class

[jira] [Created] (SPARK-41147) Rename _LEGACY_ERROR_TEMP_1042 to INVALID_FUNCTION_ARGUMENT

2022-11-14 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-41147: --- Summary: Rename _LEGACY_ERROR_TEMP_1042 to INVALID_FUNCTION_ARGUMENT Key: SPARK-41147 URL: https://issues.apache.org/jira/browse/SPARK-41147 Project: Spark

[jira] [Commented] (SPARK-41145) Assert the offset range for file stream source in Trigger.AvailableNow

2022-11-14 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634193#comment-17634193 ] Jungtaek Lim commented on SPARK-41145: -- Looks like we have proper assertion in offset range for

[jira] [Resolved] (SPARK-41145) Assert the offset range for file stream source in Trigger.AvailableNow

2022-11-14 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-41145. -- Resolution: Not A Problem > Assert the offset range for file stream source in

[jira] [Updated] (SPARK-41146) Assert the offset range for Kafka source in Trigger.AvailableNow

2022-11-14 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-41146: - Description: We encountered the issue where the data source did not properly implement the

[jira] [Created] (SPARK-41146) Assert the offset range for Kafka source in Trigger.AvailableNow

2022-11-14 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-41146: Summary: Assert the offset range for Kafka source in Trigger.AvailableNow Key: SPARK-41146 URL: https://issues.apache.org/jira/browse/SPARK-41146 Project: Spark

[jira] [Created] (SPARK-41145) Assert the offset range for file stream source in Trigger.AvailableNow

2022-11-14 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-41145: Summary: Assert the offset range for file stream source in Trigger.AvailableNow Key: SPARK-41145 URL: https://issues.apache.org/jira/browse/SPARK-41145 Project:

[jira] [Assigned] (SPARK-41137) Rename LATERAL_JOIN_OF_TYPE to INVALID_LATERAL_JOIN_TYPE

2022-11-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41137: Assignee: Haejoon Lee > Rename LATERAL_JOIN_OF_TYPE to INVALID_LATERAL_JOIN_TYPE >

[jira] [Resolved] (SPARK-41137) Rename LATERAL_JOIN_OF_TYPE to INVALID_LATERAL_JOIN_TYPE

2022-11-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41137. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38652

[jira] [Resolved] (SPARK-41025) Introduce ComparableOffset as mixed-in interface for streaming offset

2022-11-14 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-41025. -- Resolution: Abandoned > Introduce ComparableOffset as mixed-in interface for streaming offset

[jira] [Resolved] (SPARK-41072) Convert the internal error about failed stream to user-facing error

2022-11-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41072. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38629

[jira] [Assigned] (SPARK-41072) Convert the internal error about failed stream to user-facing error

2022-11-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41072: Assignee: Max Gekk > Convert the internal error about failed stream to user-facing error >

[jira] [Assigned] (SPARK-41143) Add named arguments function syntax support and trait

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41143: Assignee: (was: Apache Spark) > Add named arguments function syntax support and

[jira] [Commented] (SPARK-41143) Add named arguments function syntax support and trait

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634168#comment-17634168 ] Apache Spark commented on SPARK-41143: -- User 'Yaohua628' has created a pull request for this issue:

[jira] [Commented] (SPARK-41143) Add named arguments function syntax support and trait

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634169#comment-17634169 ] Apache Spark commented on SPARK-41143: -- User 'Yaohua628' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41143) Add named arguments function syntax support and trait

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41143: Assignee: Apache Spark > Add named arguments function syntax support and trait >

[jira] [Commented] (SPARK-41144) UnresolvedHint should not cause query failure

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634160#comment-17634160 ] Apache Spark commented on SPARK-41144: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-41144) UnresolvedHint should not cause query failure

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41144: Assignee: (was: Apache Spark) > UnresolvedHint should not cause query failure >

[jira] [Assigned] (SPARK-41144) UnresolvedHint should not cause query failure

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41144: Assignee: Apache Spark > UnresolvedHint should not cause query failure >

[jira] [Commented] (SPARK-41144) UnresolvedHint should not cause query failure

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634158#comment-17634158 ] Apache Spark commented on SPARK-41144: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-41134) improve error message of internal errors

2022-11-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41134: --- Assignee: Wenchen Fan > improve error message of internal errors >

[jira] [Resolved] (SPARK-41134) improve error message of internal errors

2022-11-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41134. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38648

[jira] [Created] (SPARK-41144) UnresolvedHint should not cause query failure

2022-11-14 Thread XiDuo You (Jira)
XiDuo You created SPARK-41144: - Summary: UnresolvedHint should not cause query failure Key: SPARK-41144 URL: https://issues.apache.org/jira/browse/SPARK-41144 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-41141) avoid introducing a new aggregate expression in the analysis phase when subquery is referencing it

2022-11-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-41141: Target Version/s: (was: 3.3.1) > avoid introducing a new aggregate expression in the analysis

[jira] [Assigned] (SPARK-41101) Add messageClassName support for pypspark-protobuf

2022-11-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41101: Assignee: Sandish Kumar HN > Add messageClassName support for pypspark-protobuf >

[jira] [Resolved] (SPARK-41101) Add messageClassName support for pypspark-protobuf

2022-11-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41101. -- Fix Version/s: 3.4.0 (was: 2.4.0) Resolution: Fixed Issue

[jira] [Resolved] (SPARK-41138) DataFrame.na.fill should have the same augment types as DataFrame.fillna

2022-11-14 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41138. --- Resolution: Not A Problem > DataFrame.na.fill should have the same augment types as

[jira] [Updated] (SPARK-41143) Add named arguments function syntax support and trait

2022-11-14 Thread Yaohua Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaohua Zhao updated SPARK-41143: Description: Parser can parse: _FUNC{_}_{_} ( key0 => value0 ) (was: Parser can parse: _FUNC_ (

[jira] [Updated] (SPARK-41143) Add named arguments function syntax support and trait

2022-11-14 Thread Yaohua Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaohua Zhao updated SPARK-41143: Description: The parser can parse: {code:java} _FUNC_ ( key0 => value0 ){code} was:Parser can

[jira] [Updated] (SPARK-41143) Add named arguments function syntax support and trait

2022-11-14 Thread Yaohua Zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yaohua Zhao updated SPARK-41143: Description: Parser can parse: _{_}FUNC_{_} ( key0 => value0 ) (was: Parser can parse:

[jira] [Closed] (SPARK-40504) Make yarn appmaster load config from client

2022-11-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-40504. - > Make yarn appmaster load config from client > --- > >

[jira] [Resolved] (SPARK-40504) Make yarn appmaster load config from client

2022-11-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40504. --- Resolution: Invalid > Make yarn appmaster load config from client >

[jira] [Reopened] (SPARK-40504) Make yarn appmaster load config from client

2022-11-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-40504: --- > Make yarn appmaster load config from client > --- > >

[jira] [Commented] (SPARK-40958) Support data masking built-in function 'null'

2022-11-14 Thread Ranga Reddy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634142#comment-17634142 ] Ranga Reddy commented on SPARK-40958: - I will work on this Jira. > Support data masking built-in

[jira] [Resolved] (SPARK-40504) Make yarn appmaster load config from client

2022-11-14 Thread zhengchenyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu resolved SPARK-40504. - Resolution: Implemented > Make yarn appmaster load config from client >

[jira] [Created] (SPARK-41143) Add named arguments function syntax support and trait

2022-11-14 Thread Yaohua Zhao (Jira)
Yaohua Zhao created SPARK-41143: --- Summary: Add named arguments function syntax support and trait Key: SPARK-41143 URL: https://issues.apache.org/jira/browse/SPARK-41143 Project: Spark Issue

[jira] [Created] (SPARK-41142) Support named arguments functions

2022-11-14 Thread Yaohua Zhao (Jira)
Yaohua Zhao created SPARK-41142: --- Summary: Support named arguments functions Key: SPARK-41142 URL: https://issues.apache.org/jira/browse/SPARK-41142 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-41123) Upgrade mysql-connector-java from 8.0.30 to 8.0.31

2022-11-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41123: Assignee: BingKun Pan > Upgrade mysql-connector-java from 8.0.30 to 8.0.31 >

[jira] [Resolved] (SPARK-41123) Upgrade mysql-connector-java from 8.0.30 to 8.0.31

2022-11-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41123. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38639

[jira] [Assigned] (SPARK-41120) Upgrade joda-time from 2.12.0 to 2.12.1

2022-11-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41120: Assignee: BingKun Pan > Upgrade joda-time from 2.12.0 to 2.12.1 >

[jira] [Resolved] (SPARK-41120) Upgrade joda-time from 2.12.0 to 2.12.1

2022-11-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41120. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38636

[jira] [Commented] (SPARK-41085) Support Bit manipulation function COUNTSET

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634124#comment-17634124 ] Apache Spark commented on SPARK-41085: -- User 'vinodkc' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41085) Support Bit manipulation function COUNTSET

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41085: Assignee: (was: Apache Spark) > Support Bit manipulation function COUNTSET >

[jira] [Assigned] (SPARK-41085) Support Bit manipulation function COUNTSET

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41085: Assignee: Apache Spark > Support Bit manipulation function COUNTSET >

[jira] [Commented] (SPARK-41073) Spark ThriftServer generate huge amounts of DelegationToken

2022-11-14 Thread zhengchenyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634119#comment-17634119 ] zhengchenyu commented on SPARK-41073: - [~xkrogen] Thank you very much for reply. This issue is

[jira] [Resolved] (SPARK-41073) Spark ThriftServer generate huge amounts of DelegationToken

2022-11-14 Thread zhengchenyu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengchenyu resolved SPARK-41073. - Resolution: Duplicate > Spark ThriftServer generate huge amounts of DelegationToken >

[jira] [Commented] (SPARK-36950) Normalize semi-structured data into a flat table.

2022-11-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634097#comment-17634097 ] Hyukjin Kwon commented on SPARK-36950: -- [~bjornjorgensen] are you interested in submitting a PR? >

[jira] [Updated] (SPARK-41141) avoid introducing a new aggregate expression in the analysis phase when subquery is referencing it

2022-11-14 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-41141: - Description: Currently the  analyzer phase rules on subquery referencing the aggregate expression in outer

[jira] [Commented] (SPARK-40199) Spark throws NPE without useful message when NULL value appears in non-null schema

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634085#comment-17634085 ] Apache Spark commented on SPARK-40199: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Created] (SPARK-41141) avoid introducing a new aggregate expression in the analysis phase when subquery is referencing it

2022-11-14 Thread Asif (Jira)
Asif created SPARK-41141: Summary: avoid introducing a new aggregate expression in the analysis phase when subquery is referencing it Key: SPARK-41141 URL: https://issues.apache.org/jira/browse/SPARK-41141

[jira] [Commented] (SPARK-40199) Spark throws NPE without useful message when NULL value appears in non-null schema

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634084#comment-17634084 ] Apache Spark commented on SPARK-40199: -- User 'xkrogen' has created a pull request for this issue:

[jira] [Commented] (SPARK-39865) Show proper error messages on the overflow errors of table insert

2022-11-14 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634062#comment-17634062 ] Gengliang Wang commented on SPARK-39865: [~catalinii] Could you provide full reproduce steps?

[jira] [Comment Edited] (SPARK-39865) Show proper error messages on the overflow errors of table insert

2022-11-14 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634062#comment-17634062 ] Gengliang Wang edited comment on SPARK-39865 at 11/14/22 10:05 PM: ---

[jira] [Assigned] (SPARK-41096) Support reading parquet FIXED_LEN_BYTE_ARRAY type

2022-11-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-41096: Assignee: Kazuyuki Tanimura > Support reading parquet FIXED_LEN_BYTE_ARRAY type >

[jira] [Resolved] (SPARK-41096) Support reading parquet FIXED_LEN_BYTE_ARRAY type

2022-11-14 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-41096. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38628

[jira] [Commented] (SPARK-40686) Support data masking and redacting built-in functions

2022-11-14 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633978#comment-17633978 ] Daniel commented on SPARK-40686: Hi [~rangareddy.av...@gmail.com] [~vinodkc] for the phi-related

[jira] [Updated] (SPARK-39601) AllocationFailure should not be treated as exitCausedByApp when driver is shutting down

2022-11-14 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-39601: -- Description: I observed some Spark Applications successfully completed all jobs but failed during

[jira] [Updated] (SPARK-39601) AllocationFailure should not be treated as exitCausedByApp when driver is shutting down

2022-11-14 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-39601: -- Description: I observed some Spark Applications successfully completed all jobs but failed during

[jira] [Updated] (SPARK-39601) AllocationFailure should not be treated as exitCausedByApp when driver is shutting down

2022-11-14 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-39601: -- Description: I observed some Spark Applications successfully completed all jobs but failed during

[jira] [Commented] (SPARK-41073) Spark ThriftServer generate huge amounts of DelegationToken

2022-11-14 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633936#comment-17633936 ] Erik Krogen commented on SPARK-41073: - I'm not very familiar with this area, but it seems you are

[jira] [Commented] (SPARK-39865) Show proper error messages on the overflow errors of table insert

2022-11-14 Thread Catalin Toda (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633915#comment-17633915 ] Catalin Toda commented on SPARK-39865: -- [~Gengliang.Wang] In our environment in Spark 3.3 we see

[jira] [Assigned] (SPARK-41114) Support local data for LocalRelation

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41114: Assignee: Apache Spark > Support local data for LocalRelation >

[jira] [Assigned] (SPARK-41114) Support local data for LocalRelation

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41114: Assignee: (was: Apache Spark) > Support local data for LocalRelation >

[jira] [Commented] (SPARK-41114) Support local data for LocalRelation

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633782#comment-17633782 ] Apache Spark commented on SPARK-41114: -- User 'dengziming' has created a pull request for this

[jira] [Commented] (SPARK-41109) Rename the error class _LEGACY_ERROR_TEMP_1216 to INVALID_LIKE_PATTERN

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633712#comment-17633712 ] Apache Spark commented on SPARK-41109: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-41109) Rename the error class _LEGACY_ERROR_TEMP_1216 to INVALID_LIKE_PATTERN

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633711#comment-17633711 ] Apache Spark commented on SPARK-41109: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-41139) Improve error message for PYTHON_UDF_IN_ON_CLAUSE

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41139: Assignee: Apache Spark > Improve error message for PYTHON_UDF_IN_ON_CLAUSE >

[jira] [Assigned] (SPARK-41139) Improve error message for PYTHON_UDF_IN_ON_CLAUSE

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41139: Assignee: (was: Apache Spark) > Improve error message for PYTHON_UDF_IN_ON_CLAUSE >

[jira] [Commented] (SPARK-41139) Improve error message for PYTHON_UDF_IN_ON_CLAUSE

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633694#comment-17633694 ] Apache Spark commented on SPARK-41139: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41140) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_2440

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41140: Assignee: Max Gekk (was: Apache Spark) > Assign a name to the legacy error class

[jira] [Assigned] (SPARK-41140) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_2440

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41140: Assignee: Apache Spark (was: Max Gekk) > Assign a name to the legacy error class

[jira] [Commented] (SPARK-41140) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_2440

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633691#comment-17633691 ] Apache Spark commented on SPARK-41140: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Updated] (SPARK-41140) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_2440

2022-11-14 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-41140: - Fix Version/s: (was: 3.4.0) > Assign a name to the legacy error class _LEGACY_ERROR_TEMP_2440 >

[jira] [Created] (SPARK-41140) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_2440

2022-11-14 Thread Max Gekk (Jira)
Max Gekk created SPARK-41140: Summary: Assign a name to the legacy error class _LEGACY_ERROR_TEMP_2440 Key: SPARK-41140 URL: https://issues.apache.org/jira/browse/SPARK-41140 Project: Spark

[jira] [Created] (SPARK-41139) Improve error message for PYTHON_UDF_IN_ON_CLAUSE

2022-11-14 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-41139: --- Summary: Improve error message for PYTHON_UDF_IN_ON_CLAUSE Key: SPARK-41139 URL: https://issues.apache.org/jira/browse/SPARK-41139 Project: Spark Issue Type:

[jira] [Commented] (SPARK-40686) Support data masking and redacting built-in functions

2022-11-14 Thread Ranga Reddy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633668#comment-17633668 ] Ranga Reddy commented on SPARK-40686: - Hi [~dtenedor]  I am not able to find more

[jira] [Assigned] (SPARK-41138) DataFrame.na.fill should have the same augment types as DataFrame.fillna

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41138: Assignee: (was: Apache Spark) > DataFrame.na.fill should have the same augment types

[jira] [Assigned] (SPARK-41138) DataFrame.na.fill should have the same augment types as DataFrame.fillna

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41138: Assignee: Apache Spark > DataFrame.na.fill should have the same augment types as

[jira] [Commented] (SPARK-41138) DataFrame.na.fill should have the same augment types as DataFrame.fillna

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633647#comment-17633647 ] Apache Spark commented on SPARK-41138: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-41138) DataFrame.na.fill should have the same augment types as DataFrame.fillna

2022-11-14 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41138: - Summary: DataFrame.na.fill should have the same augment types as DataFrame.fillna Key: SPARK-41138 URL: https://issues.apache.org/jira/browse/SPARK-41138 Project:

[jira] [Commented] (SPARK-41005) Arrow based collect

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633629#comment-17633629 ] Apache Spark commented on SPARK-41005: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-41005) Arrow based collect

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633627#comment-17633627 ] Apache Spark commented on SPARK-41005: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-41013) spark-3.1.2以cluster模式提交作业报 Could not initialize class com.github.luben.zstd.ZstdOutputStream

2022-11-14 Thread yutiantian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633613#comment-17633613 ] yutiantian commented on SPARK-41013: 最终原因:由于配置了

[jira] [Assigned] (SPARK-41128) Implement `DataFrame.fillna ` and `DataFrame.na.fill `

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41128: Assignee: Ruifeng Zheng (was: Apache Spark) > Implement `DataFrame.fillna ` and

[jira] [Commented] (SPARK-41128) Implement `DataFrame.fillna ` and `DataFrame.na.fill `

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633606#comment-17633606 ] Apache Spark commented on SPARK-41128: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41128) Implement `DataFrame.fillna ` and `DataFrame.na.fill `

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41128: Assignee: Apache Spark (was: Ruifeng Zheng) > Implement `DataFrame.fillna ` and

[jira] [Commented] (SPARK-41128) Implement `DataFrame.fillna ` and `DataFrame.na.fill `

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633605#comment-17633605 ] Apache Spark commented on SPARK-41128: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-41137) Rename LATERAL_JOIN_OF_TYPE to INVALID_LATERAL_JOIN_TYPE

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41137: Assignee: Apache Spark > Rename LATERAL_JOIN_OF_TYPE to INVALID_LATERAL_JOIN_TYPE >

[jira] [Assigned] (SPARK-41137) Rename LATERAL_JOIN_OF_TYPE to INVALID_LATERAL_JOIN_TYPE

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41137: Assignee: (was: Apache Spark) > Rename LATERAL_JOIN_OF_TYPE to

[jira] [Commented] (SPARK-41137) Rename LATERAL_JOIN_OF_TYPE to INVALID_LATERAL_JOIN_TYPE

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633598#comment-17633598 ] Apache Spark commented on SPARK-41137: -- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-41136) Shorten graceful shutdown time of ExecutorPodsSnapshotsStoreImpl to prevent blocking shutdown process

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633595#comment-17633595 ] Apache Spark commented on SPARK-41136: -- User 'pan3793' has created a pull request for this issue:

[jira] [Assigned] (SPARK-41136) Shorten graceful shutdown time of ExecutorPodsSnapshotsStoreImpl to prevent blocking shutdown process

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41136: Assignee: Apache Spark > Shorten graceful shutdown time of

[jira] [Assigned] (SPARK-41136) Shorten graceful shutdown time of ExecutorPodsSnapshotsStoreImpl to prevent blocking shutdown process

2022-11-14 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41136: Assignee: (was: Apache Spark) > Shorten graceful shutdown time of

  1   2   >