[jira] [Commented] (SPARK-42125) Pandas UDF in Spark Connect

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680913#comment-17680913 ] Apache Spark commented on SPARK-42125: -- User 'xinrong-meng' has created a pull request for this

[jira] [Assigned] (SPARK-42125) Pandas UDF in Spark Connect

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42125: Assignee: Apache Spark > Pandas UDF in Spark Connect > --- > >

[jira] [Assigned] (SPARK-42125) Pandas UDF in Spark Connect

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42125: Assignee: (was: Apache Spark) > Pandas UDF in Spark Connect >

[jira] [Commented] (SPARK-42168) CoGroup with window function returns incorrect result when partition keys differ in order

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680909#comment-17680909 ] Apache Spark commented on SPARK-42168: -- User 'EnricoMi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42197) Reuses JVM initialization, and separate configuration groups to set in remote local mode

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42197: Assignee: (was: Apache Spark) > Reuses JVM initialization, and separate

[jira] [Assigned] (SPARK-42197) Reuses JVM initialization, and separate configuration groups to set in remote local mode

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42197: Assignee: Apache Spark > Reuses JVM initialization, and separate configuration groups to

[jira] [Commented] (SPARK-42197) Reuses JVM initialization, and separate configuration groups to set in remote local mode

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680900#comment-17680900 ] Apache Spark commented on SPARK-42197: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-42197) Reuses JVM initialization, and separate configuration groups to set in remote local mode

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42197: - Summary: Reuses JVM initialization, and separate configuration groups to set in remote local

[jira] [Updated] (SPARK-42197) Refactor remote local mode code path

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42197: - Epic Link: SPARK-39375 > Refactor remote local mode code path >

[jira] [Updated] (SPARK-42197) Refactor remote local mode code path

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42197: - Epic Link: (was: SPARK-39375) > Refactor remote local mode code path >

[jira] [Updated] (SPARK-42197) Refactor remote local mode code path

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42197: - Parent: (was: SPARK-41285) Issue Type: Bug (was: Sub-task) > Refactor remote local

[jira] [Updated] (SPARK-42197) Refactor remote local mode code path

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42197: - Parent: SPARK-41932 Issue Type: Sub-task (was: Bug) > Refactor remote local mode code

[jira] [Created] (SPARK-42197) Refactor remote local mode code path

2023-01-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42197: Summary: Refactor remote local mode code path Key: SPARK-42197 URL: https://issues.apache.org/jira/browse/SPARK-42197 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-42196) Typo in StreamingQuery.scala

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680898#comment-17680898 ] Apache Spark commented on SPARK-42196: -- User 'ganeshchand' has created a pull request for this

[jira] [Assigned] (SPARK-42196) Typo in StreamingQuery.scala

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42196: Assignee: Apache Spark > Typo in StreamingQuery.scala > > >

[jira] [Commented] (SPARK-42196) Typo in StreamingQuery.scala

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680897#comment-17680897 ] Apache Spark commented on SPARK-42196: -- User 'ganeshchand' has created a pull request for this

[jira] [Assigned] (SPARK-42196) Typo in StreamingQuery.scala

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42196: Assignee: (was: Apache Spark) > Typo in StreamingQuery.scala >

[jira] [Commented] (SPARK-42196) Typo in StreamingQuery.scala

2023-01-25 Thread Ganesh Chand (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680896#comment-17680896 ] Ganesh Chand commented on SPARK-42196: -- I have created a PR with a fix -

[jira] [Updated] (SPARK-42196) Typo in StreamingQuery.scala

2023-01-25 Thread Ganesh Chand (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ganesh Chand updated SPARK-42196: - Description: {color:#172b4d}/**{color} {color:#172b4d} * Returns the unique id of this run of

[jira] [Created] (SPARK-42196) Typo in StreamingQuery.scala

2023-01-25 Thread Ganesh Chand (Jira)
Ganesh Chand created SPARK-42196: Summary: Typo in StreamingQuery.scala Key: SPARK-42196 URL: https://issues.apache.org/jira/browse/SPARK-42196 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-42195) Add Github action test job for branch-3.4

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42195: Assignee: Apache Spark > Add Github action test job for branch-3.4 >

[jira] [Commented] (SPARK-42195) Add Github action test job for branch-3.4

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680893#comment-17680893 ] Apache Spark commented on SPARK-42195: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-42195) Add Github action test job for branch-3.4

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42195: Assignee: (was: Apache Spark) > Add Github action test job for branch-3.4 >

[jira] [Created] (SPARK-42195) Add Github action test job for branch-3.4

2023-01-25 Thread Yang Jie (Jira)
Yang Jie created SPARK-42195: Summary: Add Github action test job for branch-3.4 Key: SPARK-42195 URL: https://issues.apache.org/jira/browse/SPARK-42195 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-42187) Avoid using RemoteSparkSession.builder.getOrCreate in tests

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42187. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39743

[jira] [Assigned] (SPARK-42187) Avoid using RemoteSparkSession.builder.getOrCreate in tests

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42187: Assignee: Hyukjin Kwon > Avoid using RemoteSparkSession.builder.getOrCreate in tests >

[jira] [Updated] (SPARK-42190) Support `local[*]` in `spark-submit` in K8s environment

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42190: -- Parent: SPARK-41550 Issue Type: Sub-task (was: New Feature) > Support `local[*]` in

[jira] [Updated] (SPARK-42190) Support `local[*]` in `spark-submit` in K8s environment

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42190: -- Affects Version/s: 3.4.0 (was: 3.5.0) > Support `local[*]` in

[jira] [Assigned] (SPARK-42190) Support `local[*]` in `spark-submit` in K8s environment

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42190: - Assignee: Dongjoon Hyun > Support `local[*]` in `spark-submit` in K8s environment >

[jira] [Resolved] (SPARK-42190) Support `local[*]` in `spark-submit` in K8s environment

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42190. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39748

[jira] [Resolved] (SPARK-42182) Make `ReusedConnectTestCase` to take Spark configurations

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42182. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39738

[jira] [Assigned] (SPARK-42182) Make `ReusedConnectTestCase` to take Spark configurations

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42182: Assignee: Hyukjin Kwon > Make `ReusedConnectTestCase` to take Spark configurations >

[jira] [Resolved] (SPARK-41550) Dynamic Allocation on K8S GA

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41550. --- Fix Version/s: 3.4.0 Assignee: Dongjoon Hyun Resolution: Fixed > Dynamic

[jira] [Updated] (SPARK-42194) Allow `columns` parameter when creating DataFrame with Series.

2023-01-25 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42194: Description: pandas API on Spark doesn't allow creating DataFrame with Series by specifying the

[jira] [Commented] (SPARK-42192) Migrate the errors from `pyspark/sql/dataframe.py` into error class.

2023-01-25 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680861#comment-17680861 ] Haejoon Lee commented on SPARK-42192: - I'm working on it > Migrate the errors from

[jira] [Commented] (SPARK-42194) Allow `columns` parameter when creating DataFrame with Series.

2023-01-25 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680859#comment-17680859 ] Haejoon Lee commented on SPARK-42194: - I'm working on it > Allow `columns` parameter when creating

[jira] [Created] (SPARK-42194) Allow `columns` parameter when creating DataFrame with Series.

2023-01-25 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42194: --- Summary: Allow `columns` parameter when creating DataFrame with Series. Key: SPARK-42194 URL: https://issues.apache.org/jira/browse/SPARK-42194 Project: Spark

[jira] [Resolved] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2 and 3.3

2023-01-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-42188. Assignee: Steve Vaughan Resolution: Fixed > Force SBT protobuf version to match Maven on

[jira] [Updated] (SPARK-42193) dataframe API filter criteria throwing ParseException when reading a JDBC column name with special characters

2023-01-25 Thread Shanmugavel Kuttiyandi Chandrakasu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shanmugavel Kuttiyandi Chandrakasu updated SPARK-42193: --- Description: *On Spark 3.3.0,* when reading from a

[jira] [Updated] (SPARK-42193) dataframe API filter criteria throwing ParseException when reading a JDBC column name with special characters

2023-01-25 Thread Shanmugavel Kuttiyandi Chandrakasu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shanmugavel Kuttiyandi Chandrakasu updated SPARK-42193: --- Priority: Minor (was: Major) > dataframe API

[jira] [Created] (SPARK-42193) dataframe API filter criteria throwing ParseException when reading a JDBC column name with special characters

2023-01-25 Thread Shanmugavel Kuttiyandi Chandrakasu (Jira)
Shanmugavel Kuttiyandi Chandrakasu created SPARK-42193: -- Summary: dataframe API filter criteria throwing ParseException when reading a JDBC column name with special characters Key: SPARK-42193

[jira] [Updated] (SPARK-42192) Migrate the errors from `pyspark/sql/dataframe.py` into error class.

2023-01-25 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42192: Summary: Migrate the errors from `pyspark/sql/dataframe.py` into error class. (was: Migrate the

[jira] [Updated] (SPARK-41597) Improve PySpark errors

2023-01-25 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-41597: Affects Version/s: 3.5.0 > Improve PySpark errors > -- > >

[jira] [Resolved] (SPARK-42168) CoGroup with window function returns incorrect result when partition keys differ in order

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42168. -- Fix Version/s: 3.2.4 Resolution: Fixed Issue resolved by pull request 39717

[jira] [Assigned] (SPARK-42168) CoGroup with window function returns incorrect result when partition keys differ in order

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42168: Assignee: Enrico Minack > CoGroup with window function returns incorrect result when

[jira] [Updated] (SPARK-42192) Migrate the errors from `pyspark/sql/functions.py` into error class.

2023-01-25 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42192: Description: Migrate the existing errors into new PySpark error framework. > Migrate the errors

[jira] [Created] (SPARK-42192) Migrate the errors from `pyspark/sql/functions.py` into error class.

2023-01-25 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-42192: --- Summary: Migrate the errors from `pyspark/sql/functions.py` into error class. Key: SPARK-42192 URL: https://issues.apache.org/jira/browse/SPARK-42192 Project: Spark

[jira] [Assigned] (SPARK-42123) Include column default values in DESCRIBE output

2023-01-25 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-42123: -- Assignee: Daniel > Include column default values in DESCRIBE output >

[jira] [Resolved] (SPARK-42123) Include column default values in DESCRIBE output

2023-01-25 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-42123. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39726

[jira] [Assigned] (SPARK-42190) Support `local[*]` in `spark-submit` in K8s environment

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42190: Assignee: Apache Spark > Support `local[*]` in `spark-submit` in K8s environment >

[jira] [Commented] (SPARK-42190) Support `local[*]` in `spark-submit` in K8s environment

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680782#comment-17680782 ] Apache Spark commented on SPARK-42190: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-42190) Support `local[*]` in `spark-submit` in K8s environment

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42190: Assignee: (was: Apache Spark) > Support `local[*]` in `spark-submit` in K8s

[jira] [Updated] (SPARK-42191) Support udf luhn_check

2023-01-25 Thread Vinod KC (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod KC updated SPARK-42191: - Description: Support UDF to check if a given number string is a valid Luhn number. It shall return

[jira] [Commented] (SPARK-42191) Support udf luhn_check

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680777#comment-17680777 ] Apache Spark commented on SPARK-42191: -- User 'vinodkc' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42191) Support udf luhn_check

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42191: Assignee: Apache Spark > Support udf luhn_check > --- > >

[jira] [Commented] (SPARK-42191) Support udf luhn_check

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680776#comment-17680776 ] Apache Spark commented on SPARK-42191: -- User 'vinodkc' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42191) Support udf luhn_check

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42191: Assignee: (was: Apache Spark) > Support udf luhn_check > --- >

[jira] [Assigned] (SPARK-40686) Support data masking and redacting built-in functions

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40686: Assignee: (was: Apache Spark) > Support data masking and redacting built-in

[jira] [Assigned] (SPARK-40686) Support data masking and redacting built-in functions

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40686: Assignee: Apache Spark > Support data masking and redacting built-in functions >

[jira] [Commented] (SPARK-40686) Support data masking and redacting built-in functions

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680770#comment-17680770 ] Apache Spark commented on SPARK-40686: -- User 'vinodkc' has created a pull request for this issue:

[jira] [Created] (SPARK-42191) Support udf luhn_check

2023-01-25 Thread Vinod KC (Jira)
Vinod KC created SPARK-42191: Summary: Support udf luhn_check Key: SPARK-42191 URL: https://issues.apache.org/jira/browse/SPARK-42191 Project: Spark Issue Type: Sub-task Components:

[jira] [Resolved] (SPARK-42178) Handle remaining null string values in ui protobuf serializer and add tests

2023-01-25 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-42178. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39732

[jira] [Updated] (SPARK-42190) Support `local[*]` in `spark-submit` in K8s environment

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42190: -- Summary: Support `local[*]` in `spark-submit` in K8s environment (was: Support `local[*]` in

[jira] [Created] (SPARK-42190) Support `local[*]` in `spark-submit`

2023-01-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-42190: - Summary: Support `local[*]` in `spark-submit` Key: SPARK-42190 URL: https://issues.apache.org/jira/browse/SPARK-42190 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2 and 3.3

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680721#comment-17680721 ] Apache Spark commented on SPARK-42188: -- User 'snmvaughan' has created a pull request for this

[jira] [Commented] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2 and 3.3

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680720#comment-17680720 ] Apache Spark commented on SPARK-42188: -- User 'snmvaughan' has created a pull request for this

[jira] [Updated] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2 and 3.3

2023-01-25 Thread Steve Vaughan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Vaughan updated SPARK-42188: -- Affects Version/s: 3.3.1 > Force SBT protobuf version to match Maven on branch 3.2 and 3.3 >

[jira] [Updated] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2 and 3.3

2023-01-25 Thread Steve Vaughan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Vaughan updated SPARK-42188: -- Summary: Force SBT protobuf version to match Maven on branch 3.2 and 3.3 (was: Force SBT

[jira] [Updated] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2 and 3.3

2023-01-25 Thread Steve Vaughan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Vaughan updated SPARK-42188: -- Fix Version/s: 3.3.2 > Force SBT protobuf version to match Maven on branch 3.2 and 3.3 >

[jira] [Commented] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680713#comment-17680713 ] Apache Spark commented on SPARK-42188: -- User 'snmvaughan' has created a pull request for this

[jira] [Assigned] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42188: Assignee: (was: Apache Spark) > Force SBT protobuf version to match Maven on branch

[jira] [Commented] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680712#comment-17680712 ] Apache Spark commented on SPARK-42188: -- User 'snmvaughan' has created a pull request for this

[jira] [Assigned] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42188: Assignee: Apache Spark > Force SBT protobuf version to match Maven on branch 3.2 >

[jira] [Assigned] (SPARK-42186) Make SparkR able to stop properly when the connection is timed-out

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42186: - Assignee: Hyukjin Kwon > Make SparkR able to stop properly when the connection is

[jira] [Resolved] (SPARK-42186) Make SparkR able to stop properly when the connection is timed-out

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42186. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39742

[jira] [Resolved] (SPARK-42184) Setting version to 3.5.0-SNAPSHOT

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42184. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39733

[jira] [Assigned] (SPARK-42184) Setting version to 3.5.0-SNAPSHOT

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42184: - Assignee: Xinrong Meng > Setting version to 3.5.0-SNAPSHOT >

[jira] [Created] (SPARK-42189) Guides for implementing custom data sources & sinks for structured streaming

2023-01-25 Thread Kai-Michael Roesner (Jira)
Kai-Michael Roesner created SPARK-42189: --- Summary: Guides for implementing custom data sources & sinks for structured streaming Key: SPARK-42189 URL: https://issues.apache.org/jira/browse/SPARK-42189

[jira] [Updated] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2

2023-01-25 Thread Steve Vaughan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Vaughan updated SPARK-42188: -- Fix Version/s: 3.2.4 (was: 3.2.3) > Force SBT protobuf version to

[jira] [Created] (SPARK-42188) Force SBT protobuf version to match Maven on branch 3.2

2023-01-25 Thread Steve Vaughan (Jira)
Steve Vaughan created SPARK-42188: - Summary: Force SBT protobuf version to match Maven on branch 3.2 Key: SPARK-42188 URL: https://issues.apache.org/jira/browse/SPARK-42188 Project: Spark

[jira] [Commented] (SPARK-38591) Add sortWithinGroups to KeyValueGroupedDataset

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680629#comment-17680629 ] Apache Spark commented on SPARK-38591: -- User 'EnricoMi' has created a pull request for this issue:

[jira] [Resolved] (SPARK-42183) Exclude pyspark.ml.torch.tests in MyPy tests

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42183. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39740

[jira] [Assigned] (SPARK-42187) Avoid using RemoteSparkSession.builder.getOrCreate in tests

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42187: Assignee: Apache Spark > Avoid using RemoteSparkSession.builder.getOrCreate in tests >

[jira] [Assigned] (SPARK-42187) Avoid using RemoteSparkSession.builder.getOrCreate in tests

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42187: Assignee: (was: Apache Spark) > Avoid using RemoteSparkSession.builder.getOrCreate

[jira] [Assigned] (SPARK-42183) Exclude pyspark.ml.torch.tests in MyPy tests

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42183: Assignee: Hyukjin Kwon > Exclude pyspark.ml.torch.tests in MyPy tests >

[jira] [Commented] (SPARK-42187) Avoid using RemoteSparkSession.builder.getOrCreate in tests

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680621#comment-17680621 ] Apache Spark commented on SPARK-42187: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-42187) Avoid using RemoteSparkSession.builder.getOrCreate in tests

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42187: - Parent: SPARK-41285 Issue Type: Sub-task (was: Bug) > Avoid using

[jira] [Updated] (SPARK-42182) Make `ReusedConnectTestCase` to take Spark configurations

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42182: - Parent: SPARK-41285 Issue Type: Sub-task (was: Test) > Make `ReusedConnectTestCase` to

[jira] [Created] (SPARK-42187) Avoid using RemoteSparkSession.builder.getOrCreate in tests

2023-01-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42187: Summary: Avoid using RemoteSparkSession.builder.getOrCreate in tests Key: SPARK-42187 URL: https://issues.apache.org/jira/browse/SPARK-42187 Project: Spark

[jira] [Assigned] (SPARK-42186) Make SparkR able to stop properly when the connection is timed-out

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42186: Assignee: (was: Apache Spark) > Make SparkR able to stop properly when the

[jira] [Assigned] (SPARK-42186) Make SparkR able to stop properly when the connection is timed-out

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42186: Assignee: Apache Spark > Make SparkR able to stop properly when the connection is

[jira] [Commented] (SPARK-42186) Make SparkR able to stop properly when the connection is timed-out

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680600#comment-17680600 ] Apache Spark commented on SPARK-42186: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-42186) Make SparkR able to stop properly when the connection is timed-out

2023-01-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-42186: Summary: Make SparkR able to stop properly when the connection is timed-out Key: SPARK-42186 URL: https://issues.apache.org/jira/browse/SPARK-42186 Project: Spark

[jira] [Assigned] (SPARK-42185) Add `branch-3.4` to publish_snapshot GitHub Action job

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42185: Assignee: Dongjoon Hyun > Add `branch-3.4` to publish_snapshot GitHub Action job >

[jira] [Resolved] (SPARK-42185) Add `branch-3.4` to publish_snapshot GitHub Action job

2023-01-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42185. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 39741

[jira] [Assigned] (SPARK-42185) Add `branch-3.4` to publish_snapshot GitHub Action job

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42185: Assignee: (was: Apache Spark) > Add `branch-3.4` to publish_snapshot GitHub Action

[jira] [Commented] (SPARK-42185) Add `branch-3.4` to publish_snapshot GitHub Action job

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680582#comment-17680582 ] Apache Spark commented on SPARK-42185: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-42185) Add `branch-3.4` to publish_snapshot GitHub Action job

2023-01-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42185: Assignee: Apache Spark > Add `branch-3.4` to publish_snapshot GitHub Action job >

[jira] [Created] (SPARK-42185) Add `branch-3.4` to publish_snapshot GitHub Action job

2023-01-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-42185: - Summary: Add `branch-3.4` to publish_snapshot GitHub Action job Key: SPARK-42185 URL: https://issues.apache.org/jira/browse/SPARK-42185 Project: Spark

[jira] [Assigned] (SPARK-42179) Upgrade ORC to 1.7.8

2023-01-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42179: - Assignee: Dongjoon Hyun > Upgrade ORC to 1.7.8 > > >

  1   2   >