[jira] [Updated] (SPARK-43941) Add any_value, approx_percentile,count_if,first_value,histogram_numeric,last_value to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43941: -- Summary: Add any_value, approx_percentile,count_if,first_value,histogram_numeric,last_value

[jira] [Updated] (SPARK-43941) Add any_value, approx_percentile,count_if,first_value,histogram_numeric to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43941: -- Description: Add following functions: * any_value * approx_percentile * count_if *

[jira] [Updated] (SPARK-43941) Add any_value, approx_percentile,count_if,first_value,histogram_numeric to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43941: -- Summary: Add any_value, approx_percentile,count_if,first_value,histogram_numeric to Scala and

[jira] [Updated] (SPARK-43941) Add any_value, approx_percentile,count_if to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43941: -- Description: Add following functions: * any_value * approx_percentile * count_if *

[jira] [Updated] (SPARK-43941) Add any_value, approx_percentile,count_if to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43941: -- Description: Add following functions: * any_value * approx_percentile * count_if *

[jira] [Created] (SPARK-43943) Add math functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43943: - Summary: Add math functions to Scala and Python Key: SPARK-43943 URL: https://issues.apache.org/jira/browse/SPARK-43943 Project: Spark Issue Type:

[jira] [Created] (SPARK-43942) Add string functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43942: - Summary: Add string functions to Scala and Python Key: SPARK-43942 URL: https://issues.apache.org/jira/browse/SPARK-43942 Project: Spark Issue Type:

[jira] [Created] (SPARK-43941) Add any_value, approx_percentile,count_if to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43941: - Summary: Add any_value, approx_percentile,count_if to Scala and Python Key: SPARK-43941 URL: https://issues.apache.org/jira/browse/SPARK-43941 Project: Spark

[jira] [Updated] (SPARK-43929) Add date time functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43929: -- Description: Add following functions: * date_diff * date_from_unix_date * date_part *

[jira] [Updated] (SPARK-43929) Add date time functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43929: -- Description: Add following functions: * date_diff * date_from_unix_date * date_part *

[jira] [Resolved] (SPARK-43936) Fix bug for toSQLId

2023-06-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43936. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41430

[jira] [Assigned] (SPARK-43936) Fix bug for toSQLId

2023-06-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43936: Assignee: BingKun Pan > Fix bug for toSQLId > --- > > Key:

[jira] [Created] (SPARK-43939) Add try_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43939: - Summary: Add try_* functions to Scala and Python Key: SPARK-43939 URL: https://issues.apache.org/jira/browse/SPARK-43939 Project: Spark Issue Type:

[jira] [Created] (SPARK-43940) Write test for CANNOT_FIND_BATCH (Prev _LEGACY_ERROR_TEMP_2132)

2023-06-01 Thread Amanda Liu (Jira)
Amanda Liu created SPARK-43940: -- Summary: Write test for CANNOT_FIND_BATCH (Prev _LEGACY_ERROR_TEMP_2132) Key: SPARK-43940 URL: https://issues.apache.org/jira/browse/SPARK-43940 Project: Spark

[jira] [Updated] (SPARK-43935) Add xpath_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43935: -- Description: Add following functions: * xpath * xpath_boolean * xpath_double * xpath_float *

[jira] [Updated] (SPARK-43935) Add xpath_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43935: -- Environment: (was: Add following functions: * xpath * xpath_boolean * xpath_double *

[jira] [Updated] (SPARK-43926) Add array_agg, array_size, cardinality, count_min_sketch to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43926: -- Summary: Add array_agg, array_size, cardinality, count_min_sketch to Scala and Python (was:

[jira] [Updated] (SPARK-43926) Add array_agg, array_size, cardinality, count_min_sketch to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43926: -- Description: Add array_agg, array_size, cardinality, count_min_sketch to: * Scala API *

[jira] [Updated] (SPARK-43925) Add any, some, bool_or,bool_and,every to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43925: -- Description: Add following functions: * any * some * bool_or * bool_and * every to: *

[jira] [Commented] (SPARK-43936) Fix bug for toSQLId

2023-06-01 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728557#comment-17728557 ] Snoot.io commented on SPARK-43936: -- User 'panbingkun' has created a pull request for this issue:

[jira] [Updated] (SPARK-43937) Add if,ifnull,isnotnull,equal_null,nullif,nvl,nvl2 to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43937: -- Description: Add following functions: * if * ifnull * isnotnull * nullif * nvl * nvl2 to:

[jira] [Updated] (SPARK-43937) Add if,ifnull,isnotnull,equal_null,nullif,nvl,nvl2 to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43937: -- Description: Add following functions: * if * ifnull * isnotnull * equal_null * nullif * nvl

[jira] [Updated] (SPARK-43937) Add if,ifnull,isnotnull,equal_null,nullif,nvl,nvl2 to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43937: -- Summary: Add if,ifnull,isnotnull,equal_null,nullif,nvl,nvl2 to Scala and Python (was: Add

[jira] [Updated] (SPARK-43924) Add misc functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43924: -- Description: Add following functions: * uuid * aes_encrypt * aes_decrypt * sha *

[jira] [Updated] (SPARK-43924) Add misc functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43924: -- Summary: Add misc functions to Scala and Python (was: Add aes_decrypt, aes_encrypt to Scala

[jira] [Created] (SPARK-43938) Add to_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43938: - Summary: Add to_* functions to Scala and Python Key: SPARK-43938 URL: https://issues.apache.org/jira/browse/SPARK-43938 Project: Spark Issue Type:

[jira] [Created] (SPARK-43937) Add if,ifnull,nullif,nvl,nvl2 to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43937: - Summary: Add if,ifnull,nullif,nvl,nvl2 to Scala and Python Key: SPARK-43937 URL: https://issues.apache.org/jira/browse/SPARK-43937 Project: Spark Issue

[jira] [Commented] (SPARK-43297) Make improvement to LocalKMeans

2023-06-01 Thread Snoot.io (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728553#comment-17728553 ] Snoot.io commented on SPARK-43297: -- User 'wenwj0' has created a pull request for this issue:

[jira] [Commented] (SPARK-43936) Fix bug for toSQLId

2023-06-01 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728540#comment-17728540 ] BingKun Pan commented on SPARK-43936: - h4. Hi,[~maxgekk]  Do you have time to help review this PR?

[jira] [Updated] (SPARK-43936) Fix bug for toSQLId

2023-06-01 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43936: Description: After SPARK-43910, {{__auto_generated_subquery_name}} from ids in errors should

[jira] [Updated] (SPARK-43925) Add any, some, bool_or,bool_and,every to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43925: -- Description: Add following functions: * any * some * bool_or * bool_and * bit_xor * every

[jira] [Created] (SPARK-43936) Fix bug for toSQLId

2023-06-01 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-43936: --- Summary: Fix bug for toSQLId Key: SPARK-43936 URL: https://issues.apache.org/jira/browse/SPARK-43936 Project: Spark Issue Type: Bug Components: SQL

[jira] [Updated] (SPARK-43925) Add any, some, bool_or,bool_and,every to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43925: -- Summary: Add any, some, bool_or,bool_and,every to Scala and Python (was: Add any, some,

[jira] [Created] (SPARK-43935) Add xpath_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43935: - Summary: Add xpath_* functions to Scala and Python Key: SPARK-43935 URL: https://issues.apache.org/jira/browse/SPARK-43935 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-43368) Address DOI comments about /etc/passwd

2023-06-01 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang reassigned SPARK-43368: --- Assignee: Yikun Jiang > Address DOI comments about /etc/passwd >

[jira] [Resolved] (SPARK-43368) Address DOI comments about /etc/passwd

2023-06-01 Thread Yikun Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-43368. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 45

[jira] [Created] (SPARK-43934) Add regexp_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43934: - Summary: Add regexp_* functions to Scala and Python Key: SPARK-43934 URL: https://issues.apache.org/jira/browse/SPARK-43934 Project: Spark Issue Type:

[jira] [Created] (SPARK-43933) Add regr_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43933: - Summary: Add regr_* functions to Scala and Python Key: SPARK-43933 URL: https://issues.apache.org/jira/browse/SPARK-43933 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-43267) Support creating data frame from a Postgres table that contains user-defined array column

2023-06-01 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-43267. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40953

[jira] [Assigned] (SPARK-43267) Support creating data frame from a Postgres table that contains user-defined array column

2023-06-01 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-43267: Assignee: Jia Fan > Support creating data frame from a Postgres table that contains user-defined

[jira] [Updated] (SPARK-43932) Add current_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43932: -- Description: Add following functions: * curdate * current_catalog * current_database *

[jira] [Created] (SPARK-43932) Add current_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43932: - Summary: Add current_* functions to Scala and Python Key: SPARK-43932 URL: https://issues.apache.org/jira/browse/SPARK-43932 Project: Spark Issue Type:

[jira] [Created] (SPARK-43931) Add make_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43931: - Summary: Add make_* functions to Scala and Python Key: SPARK-43931 URL: https://issues.apache.org/jira/browse/SPARK-43931 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43926) Add array_agg, array_size, cardinality to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43926: -- Summary: Add array_agg, array_size, cardinality to Scala and Python (was: Add array_agg,

[jira] [Updated] (SPARK-43926) Add array_agg, array_size, cardinality to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43926: -- Description: Add array_agg, array_size, cardinality to: * Scala API * Python API * Spark

[jira] [Updated] (SPARK-43926) Add array_agg, array_size, cardinality to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43926: -- Description: Add array_agg, array_size, to: * Scala API * Python API * Spark Connect Scala

[jira] [Created] (SPARK-43930) Add unix_* functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43930: - Summary: Add unix_* functions to Scala and Python Key: SPARK-43930 URL: https://issues.apache.org/jira/browse/SPARK-43930 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43928) Add bit operations to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43928: -- Description: Add following functions: * bit_and * bit_count * bit_get * bit_or * bit_xor *

[jira] [Created] (SPARK-43929) Add date time functions to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43929: - Summary: Add date time functions to Scala and Python Key: SPARK-43929 URL: https://issues.apache.org/jira/browse/SPARK-43929 Project: Spark Issue Type:

[jira] [Created] (SPARK-43928) Add bit operations to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43928: - Summary: Add bit operations to Scala and Python Key: SPARK-43928 URL: https://issues.apache.org/jira/browse/SPARK-43928 Project: Spark Issue Type:

[jira] [Created] (SPARK-43927) Add cast alias to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43927: - Summary: Add cast alias to Scala and Python Key: SPARK-43927 URL: https://issues.apache.org/jira/browse/SPARK-43927 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-43926) Add array_agg, array_size to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43926: - Summary: Add array_agg, array_size to Scala and Python Key: SPARK-43926 URL: https://issues.apache.org/jira/browse/SPARK-43926 Project: Spark Issue Type:

[jira] [Created] (SPARK-43925) Add any, some, bool_or to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43925: - Summary: Add any, some, bool_or to Scala and Python Key: SPARK-43925 URL: https://issues.apache.org/jira/browse/SPARK-43925 Project: Spark Issue Type:

[jira] [Created] (SPARK-43924) Add aes_decrypt, aes_encrypt to Scala and Python

2023-06-01 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-43924: - Summary: Add aes_decrypt, aes_encrypt to Scala and Python Key: SPARK-43924 URL: https://issues.apache.org/jira/browse/SPARK-43924 Project: Spark Issue

[jira] [Updated] (SPARK-43923) [CONNECT] Post listenerBus events during ExecutePlanRequest

2023-06-01 Thread Jean-Francois Desjeans Gauthier (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Francois Desjeans Gauthier updated SPARK-43923: Summary: [CONNECT] Post listenerBus events during

[jira] [Commented] (SPARK-43907) Add SQL functions into Scala, Python and R API

2023-06-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728519#comment-17728519 ] Hyukjin Kwon commented on SPARK-43907: -- Yes, please go ahead. But I think we should implement them

[jira] [Resolved] (SPARK-43892) Add support for df[|] suggest column

2023-06-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43892. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41396

[jira] [Assigned] (SPARK-43892) Add support for df[|] suggest column

2023-06-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43892: Assignee: Beishao Cao > Add support for df[|] suggest column >

[jira] [Updated] (SPARK-43923) [Connect] Post listenerBus events during ExecutePlanRequest

2023-06-01 Thread Jean-Francois Desjeans Gauthier (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Francois Desjeans Gauthier updated SPARK-43923: Description: Post events

[jira] [Created] (SPARK-43923) [Connect] Post listenerBus events during ExecutePlanRequest

2023-06-01 Thread Jean-Francois Desjeans Gauthier (Jira)
Jean-Francois Desjeans Gauthier created SPARK-43923: --- Summary: [Connect] Post listenerBus events during ExecutePlanRequest Key: SPARK-43923 URL: https://issues.apache.org/jira/browse/SPARK-43923

[jira] [Updated] (SPARK-43922) Add named parameter support in parser for function call

2023-06-01 Thread Richard Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Yu updated SPARK-43922: --- Description: Today, we are implementing named parameter support for user defined functions, some

[jira] [Updated] (SPARK-43922) Add named parameter support in parser for function call

2023-06-01 Thread Richard Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Yu updated SPARK-43922: --- Description: Today, we are implementing named parameter support for user defined functions, some

[jira] [Created] (SPARK-43922) Add named parameter support in parser for function call

2023-06-01 Thread Richard Yu (Jira)
Richard Yu created SPARK-43922: -- Summary: Add named parameter support in parser for function call Key: SPARK-43922 URL: https://issues.apache.org/jira/browse/SPARK-43922 Project: Spark Issue

[jira] [Created] (SPARK-43921) Generate Protobuf descriptor files at build time

2023-06-01 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-43921: Summary: Generate Protobuf descriptor files at build time Key: SPARK-43921 URL: https://issues.apache.org/jira/browse/SPARK-43921 Project: Spark Issue Type:

[jira] [Updated] (SPARK-43920) Create sql/api module

2023-06-01 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-43920: - Description: We need a sql/api module to host public API like DataType, Row, etc. This module can be

[jira] [Created] (SPARK-43920) Create sql/api module

2023-06-01 Thread Rui Wang (Jira)
Rui Wang created SPARK-43920: Summary: Create sql/api module Key: SPARK-43920 URL: https://issues.apache.org/jira/browse/SPARK-43920 Project: Spark Issue Type: Task Components:

[jira] [Commented] (SPARK-41958) Disallow arbitrary custom classpath with proxy user in cluster mode

2023-06-01 Thread Degant Puri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728449#comment-17728449 ] Degant Puri commented on SPARK-41958: - [~gurwls223] [~Ngone51] is it possible to backport this fix

[jira] [Updated] (SPARK-43919) Extract JSON functionality out of Row

2023-06-01 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang updated SPARK-43919: - Summary: Extract JSON functionality out of Row (was: Extract JSON functionality to Row) > Extract

[jira] [Created] (SPARK-43919) Extract JSON functionality to Row

2023-06-01 Thread Rui Wang (Jira)
Rui Wang created SPARK-43919: Summary: Extract JSON functionality to Row Key: SPARK-43919 URL: https://issues.apache.org/jira/browse/SPARK-43919 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-43902) Use keyMayExist to avoid gets if key is absent for tracking rocksdb state store updates

2023-06-01 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar resolved SPARK-43902. Resolution: Won't Do > Use keyMayExist to avoid gets if key is absent for

[jira] [Created] (SPARK-43918) Cannot CREATE VIEW despite columns explicitly aliased

2023-06-01 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-43918: Summary: Cannot CREATE VIEW despite columns explicitly aliased Key: SPARK-43918 URL: https://issues.apache.org/jira/browse/SPARK-43918 Project: Spark Issue

[jira] [Resolved] (SPARK-43899) Upgrade protobuf-java to 3.23.2

2023-06-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43899. --- Fix Version/s: 3.5.0 Assignee: Yang Jie Resolution: Fixed This is resolved

[jira] [Created] (SPARK-43917) Upgrade pandas to 2.0.2

2023-06-01 Thread Jira
Bjørn Jørgensen created SPARK-43917: --- Summary: Upgrade pandas to 2.0.2 Key: SPARK-43917 URL: https://issues.apache.org/jira/browse/SPARK-43917 Project: Spark Issue Type: Dependency upgrade

[jira] [Commented] (SPARK-43910) Strip "__auto_generated_subquery_name" from ids in errors

2023-06-01 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728405#comment-17728405 ] Hudson commented on SPARK-43910: User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-43916) Add percentile to Scala, Python and R API

2023-06-01 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728345#comment-17728345 ] jiaan.geng commented on SPARK-43916: Because I involved in the development of percentile. So I want

[jira] [Commented] (SPARK-43907) Add SQL functions into Scala, Python and R API

2023-06-01 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728344#comment-17728344 ] jiaan.geng commented on SPARK-43907: [~gurwls223]Do we really start this job ? > Add SQL functions

[jira] [Updated] (SPARK-43916) Add percentile to Scala, Python and R API

2023-06-01 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43916: --- Summary: Add percentile to Scala, Python and R API (was: Add percentile to Scala, Python, R API)

[jira] [Created] (SPARK-43916) Add percentile to Scala, Python, R API

2023-06-01 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-43916: -- Summary: Add percentile to Scala, Python, R API Key: SPARK-43916 URL: https://issues.apache.org/jira/browse/SPARK-43916 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-43913) Assign names to the error class _LEGACY_ERROR_TEMP_[2426-2432]

2023-06-01 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43913: --- Summary: Assign names to the error class _LEGACY_ERROR_TEMP_[2426-2432] (was: Assign a name to the

[jira] [Updated] (SPARK-43913) Assign a name to the error class _LEGACY_ERROR_TEMP_[2426-2432]

2023-06-01 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43913: --- Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_[2426-2432] (was: Assign a name to

[jira] [Commented] (SPARK-43906) Implement the file support in SparkSession.addArtifacts

2023-06-01 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728335#comment-17728335 ] Hudson commented on SPARK-43906: User 'HyukjinKwon' has created a pull request for this issue:

[jira] [Commented] (SPARK-43646) Maven test failed: `from_protobuf_messageClassName` and `from_protobuf_messageClassName_options`

2023-06-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728328#comment-17728328 ] Yang Jie commented on SPARK-43646: -- Can maven also use the original jar for testing?     > Maven

[jira] [Comment Edited] (SPARK-43646) Maven test failed: `from_protobuf_messageClassName` and `from_protobuf_messageClassName_options`

2023-06-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728328#comment-17728328 ] Yang Jie edited comment on SPARK-43646 at 6/1/23 12:22 PM: --- Can maven also use

[jira] [Commented] (SPARK-43523) Memory leak in Spark UI

2023-06-01 Thread Amine Bagdouri (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728327#comment-17728327 ] Amine Bagdouri commented on SPARK-43523: Hello [~srowen], Here's the change I'm suggesting :

[jira] [Commented] (SPARK-37935) Migrate onto error classes

2023-06-01 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728312#comment-17728312 ] jiaan.geng commented on SPARK-37935: [~maxgekk]I agree 3+ error classes per PR. I will create a

[jira] [Created] (SPARK-43914) Assign a name to the error class _LEGACY_ERROR_TEMP_2427

2023-06-01 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-43914: -- Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_2427 Key: SPARK-43914 URL: https://issues.apache.org/jira/browse/SPARK-43914 Project: Spark Issue

[jira] [Created] (SPARK-43915) Assign a name to the error class _LEGACY_ERROR_TEMP_2428

2023-06-01 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-43915: -- Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_2428 Key: SPARK-43915 URL: https://issues.apache.org/jira/browse/SPARK-43915 Project: Spark Issue

[jira] [Created] (SPARK-43913) Assign a name to the error class _LEGACY_ERROR_TEMP_2426

2023-06-01 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-43913: -- Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_2426 Key: SPARK-43913 URL: https://issues.apache.org/jira/browse/SPARK-43913 Project: Spark Issue

[jira] [Commented] (SPARK-43516) Basic estimator / transformer / model / evaluator interfaces and basic transformer / evaluator implementation

2023-06-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728273#comment-17728273 ] ASF GitHub Bot commented on SPARK-43516: User 'WeichenXu123' has created a pull request for this

[jira] [Commented] (SPARK-43772) Move version configuration in `connect` module to parent

2023-06-01 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728272#comment-17728272 ] ASF GitHub Bot commented on SPARK-43772: User 'panbingkun' has created a pull request for this

[jira] [Created] (SPARK-43912) Incorrect SparkException for Stage-Level Scheduling in local mode

2023-06-01 Thread Jacek Laskowski (Jira)
Jacek Laskowski created SPARK-43912: --- Summary: Incorrect SparkException for Stage-Level Scheduling in local mode Key: SPARK-43912 URL: https://issues.apache.org/jira/browse/SPARK-43912 Project:

[jira] [Updated] (SPARK-43905) Consolidate BlockId parsing and creation

2023-06-01 Thread Henry Mai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Mai updated SPARK-43905: -- External issue URL: (was: https://github.com/apache/spark/pull/41413) > Consolidate BlockId parsing

[jira] [Updated] (SPARK-43905) Consolidate BlockId parsing and creation

2023-06-01 Thread Henry Mai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henry Mai updated SPARK-43905: -- External issue URL: https://github.com/apache/spark/pull/41413 > Consolidate BlockId parsing and

[jira] [Closed] (SPARK-43836) Make Scala 2.13 as default Scala version in Spark 3.5

2023-06-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-43836. - Assignee: (was: Dongjoon Hyun) > Make Scala 2.13 as default Scala version in Spark 3.5 >

[jira] [Created] (SPARK-43911) Directly use Set to consume iterator data to deduplicate, thereby reducing memory usage

2023-06-01 Thread mcdull_zhang (Jira)
mcdull_zhang created SPARK-43911: Summary: Directly use Set to consume iterator data to deduplicate, thereby reducing memory usage Key: SPARK-43911 URL: https://issues.apache.org/jira/browse/SPARK-43911

[jira] [Comment Edited] (SPARK-43646) Maven test failed: `from_protobuf_messageClassName` and `from_protobuf_messageClassName_options`

2023-06-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728257#comment-17728257 ] Yang Jie edited comment on SPARK-43646 at 6/1/23 8:14 AM: -- GA use sbt test and

[jira] [Commented] (SPARK-43646) Maven test failed: `from_protobuf_messageClassName` and `from_protobuf_messageClassName_options`

2023-06-01 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728257#comment-17728257 ] Yang Jie commented on SPARK-43646: -- GA use sbt test and sbt test always use non-shaded jar now  >

[jira] [Resolved] (SPARK-43865) spark cluster deploy mode cannot initialize metastore java.sql.SQLException: No suitable driver found for jdbc:mysql

2023-06-01 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43865. - Resolution: Not A Problem > spark cluster deploy mode cannot initialize metastore

[jira] [Resolved] (SPARK-43910) Strip "__auto_generated_subquery_name" from ids in errors

2023-06-01 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-43910. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41411

[jira] [Updated] (SPARK-43760) Incorrect attribute nullability after RewriteCorrelatedScalarSubquery leads to incorrect query results

2023-06-01 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-43760: Fix Version/s: 3.4.1 > Incorrect attribute nullability after RewriteCorrelatedScalarSubquery

  1   2   >