[jira] [Commented] (SPARK-41786) Deduplicate helper functions

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653045#comment-17653045 ] Apache Spark commented on SPARK-41786: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41786) Deduplicate helper functions

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41786: Assignee: Apache Spark > Deduplicate helper functions > > >

[jira] [Assigned] (SPARK-41786) Deduplicate helper functions

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41786: Assignee: (was: Apache Spark) > Deduplicate helper functions > --

[jira] [Commented] (SPARK-41786) Deduplicate helper functions

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653046#comment-17653046 ] Apache Spark commented on SPARK-41786: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-41786) Deduplicate helper functions

2022-12-29 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41786: - Summary: Deduplicate helper functions Key: SPARK-41786 URL: https://issues.apache.org/jira/browse/SPARK-41786 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-41781) Add the ability to create pvc before creating driver/executor pod

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41781: Assignee: Apache Spark > Add the ability to create pvc before creating driver/executor po

[jira] [Assigned] (SPARK-41781) Add the ability to create pvc before creating driver/executor pod

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41781: Assignee: (was: Apache Spark) > Add the ability to create pvc before creating driver/

[jira] [Commented] (SPARK-41781) Add the ability to create pvc before creating driver/executor pod

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653044#comment-17653044 ] Apache Spark commented on SPARK-41781: -- User 'dcoliversun' has created a pull reque

[jira] [Commented] (SPARK-41745) SparkSession.createDataFrame does not respect the column names in the row

2022-12-29 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653043#comment-17653043 ] Ruifeng Zheng commented on SPARK-41745: --- I will take a look > SparkSession.create

[jira] [Assigned] (SPARK-41580) Assign name to _LEGACY_ERROR_TEMP_2137

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41580: Assignee: (was: Apache Spark) > Assign name to _LEGACY_ERROR_TEMP_2137 >

[jira] [Commented] (SPARK-41580) Assign name to _LEGACY_ERROR_TEMP_2137

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653042#comment-17653042 ] Apache Spark commented on SPARK-41580: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-41580) Assign name to _LEGACY_ERROR_TEMP_2137

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41580: Assignee: Apache Spark > Assign name to _LEGACY_ERROR_TEMP_2137 > ---

[jira] [Resolved] (SPARK-41578) Assign name to _LEGACY_ERROR_TEMP_2141

2022-12-29 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41578. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39279 [https://github.com

[jira] [Assigned] (SPARK-41578) Assign name to _LEGACY_ERROR_TEMP_2141

2022-12-29 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41578: Assignee: Haejoon Lee > Assign name to _LEGACY_ERROR_TEMP_2141 >

[jira] [Updated] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2022-12-29 Thread Remzi Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remzi Yang updated SPARK-41780: --- Description: {code:scala} scala> spark.sql("select regexp_replace('', '[ad]{0,2}', 'x')").show +

[jira] [Updated] (SPARK-41781) Add the ability to create pvc before creating driver/executor pod

2022-12-29 Thread Qian Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qian Sun updated SPARK-41781: - Description: Creating pvc after driver/executor pod has Warning event from default-scheduler, such as {

[jira] [Assigned] (SPARK-41782) Regenerate benchmark results

2022-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41782: - Assignee: Dongjoon Hyun > Regenerate benchmark results > >

[jira] [Resolved] (SPARK-41782) Regenerate benchmark results

2022-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41782. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39301 [https://

[jira] [Updated] (SPARK-41781) Add the ability to create pvc before creating driver/executor pod

2022-12-29 Thread Qian Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qian Sun updated SPARK-41781: - Description: Creating pvc after driver/executor pod has Warning event from default-scheduler, such as {

[jira] [Updated] (SPARK-41781) Add the ability to create pvc before creating driver/executor pod

2022-12-29 Thread Qian Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qian Sun updated SPARK-41781: - Summary: Add the ability to create pvc before creating driver/executor pod (was: Add the ability to cre

[jira] [Assigned] (SPARK-41785) Implement `GroupedData.mean`

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41785: Assignee: Apache Spark > Implement `GroupedData.mean` > > >

[jira] [Commented] (SPARK-41785) Implement `GroupedData.mean`

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653028#comment-17653028 ] Apache Spark commented on SPARK-41785: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-41785) Implement `GroupedData.mean`

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653029#comment-17653029 ] Apache Spark commented on SPARK-41785: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41785) Implement `GroupedData.mean`

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41785: Assignee: (was: Apache Spark) > Implement `GroupedData.mean` > --

[jira] [Created] (SPARK-41785) Implement `GroupedData.mean`

2022-12-29 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41785: - Summary: Implement `GroupedData.mean` Key: SPARK-41785 URL: https://issues.apache.org/jira/browse/SPARK-41785 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-41068) Implement `DataFrame.stat.corr`

2022-12-29 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41068. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39236 [https://

[jira] [Assigned] (SPARK-41784) Add missing `__rmod__`

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41784: Assignee: (was: Apache Spark) > Add missing `__rmod__` > -- > >

[jira] [Commented] (SPARK-41784) Add missing `__rmod__`

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653020#comment-17653020 ] Apache Spark commented on SPARK-41784: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41784) Add missing `__rmod__`

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41784: Assignee: Apache Spark > Add missing `__rmod__` > -- > >

[jira] [Created] (SPARK-41784) Add missing `__rmod__`

2022-12-29 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41784: - Summary: Add missing `__rmod__` Key: SPARK-41784 URL: https://issues.apache.org/jira/browse/SPARK-41784 Project: Spark Issue Type: Sub-task Compo

[jira] [Assigned] (SPARK-41770) eqNullSafe does not support None as its argument

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41770: Assignee: Apache Spark > eqNullSafe does not support None as its argument > -

[jira] [Commented] (SPARK-41770) eqNullSafe does not support None as its argument

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653015#comment-17653015 ] Apache Spark commented on SPARK-41770: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-41770) eqNullSafe does not support None as its argument

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653014#comment-17653014 ] Apache Spark commented on SPARK-41770: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41770) eqNullSafe does not support None as its argument

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41770: Assignee: (was: Apache Spark) > eqNullSafe does not support None as its argument > --

[jira] [Commented] (SPARK-41770) eqNullSafe does not support None as its argument

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653013#comment-17653013 ] Apache Spark commented on SPARK-41770: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41783) Make column op support None

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41783: Assignee: (was: Apache Spark) > Make column op support None > ---

[jira] [Assigned] (SPARK-41783) Make column op support None

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41783: Assignee: Apache Spark > Make column op support None > --- > >

[jira] [Commented] (SPARK-41783) Make column op support None

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653012#comment-17653012 ] Apache Spark commented on SPARK-41783: -- User 'zhengruifeng' has created a pull requ

[jira] [Resolved] (SPARK-41760) Enforce scalafmt for Spark Connect Client module

2022-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41760. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39274 [https://gith

[jira] [Assigned] (SPARK-41760) Enforce scalafmt for Spark Connect Client module

2022-12-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41760: --- Assignee: Deng Ziming > Enforce scalafmt for Spark Connect Client module >

[jira] [Created] (SPARK-41783) Make column op support None

2022-12-29 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41783: - Summary: Make column op support None Key: SPARK-41783 URL: https://issues.apache.org/jira/browse/SPARK-41783 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41775) Implement training functions as input

2022-12-29 Thread Rithwik Ediga Lakhamsani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rithwik Ediga Lakhamsani updated SPARK-41775: - Description: Currently, `Distributor().run(...)` takes only files as inp

[jira] [Updated] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2022-12-29 Thread Remzi Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remzi Yang updated SPARK-41780: --- Description: {code:scala} scala> spark.sql("select regexp_replace('', '[ad]\{0,2}', 'x')").show

[jira] [Commented] (SPARK-41782) Regenerate benchmark results

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653004#comment-17653004 ] Apache Spark commented on SPARK-41782: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-41782) Regenerate benchmark results

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653003#comment-17653003 ] Apache Spark commented on SPARK-41782: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-41782) Regenerate benchmark results

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41782: Assignee: (was: Apache Spark) > Regenerate benchmark results > --

[jira] [Assigned] (SPARK-41782) Regenerate benchmark results

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41782: Assignee: Apache Spark > Regenerate benchmark results > > >

[jira] [Created] (SPARK-41782) Regenerate benchmark results

2022-12-29 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-41782: - Summary: Regenerate benchmark results Key: SPARK-41782 URL: https://issues.apache.org/jira/browse/SPARK-41782 Project: Spark Issue Type: Test Com

[jira] [Commented] (SPARK-41771) __getitem__ does not work with Column.isin

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653002#comment-17653002 ] Apache Spark commented on SPARK-41771: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41771) __getitem__ does not work with Column.isin

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41771: Assignee: Apache Spark > __getitem__ does not work with Column.isin > ---

[jira] [Commented] (SPARK-41771) __getitem__ does not work with Column.isin

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653001#comment-17653001 ] Apache Spark commented on SPARK-41771: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41771) __getitem__ does not work with Column.isin

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41771: Assignee: (was: Apache Spark) > __getitem__ does not work with Column.isin >

[jira] [Commented] (SPARK-41771) __getitem__ does not work with Column.isin

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17653000#comment-17653000 ] Apache Spark commented on SPARK-41771: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41779) Make getitem support filter and select

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41779: Assignee: Apache Spark > Make getitem support filter and select > ---

[jira] [Assigned] (SPARK-41779) Make getitem support filter and select

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41779: Assignee: (was: Apache Spark) > Make getitem support filter and select >

[jira] [Created] (SPARK-41781) Add the ability to create resources before creating executor pod

2022-12-29 Thread Qian Sun (Jira)
Qian Sun created SPARK-41781: Summary: Add the ability to create resources before creating executor pod Key: SPARK-41781 URL: https://issues.apache.org/jira/browse/SPARK-41781 Project: Spark Iss

[jira] [Commented] (SPARK-41779) Make getitem support filter and select

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652999#comment-17652999 ] Apache Spark commented on SPARK-41779: -- User 'zhengruifeng' has created a pull requ

[jira] [Updated] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2022-12-29 Thread Remzi Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remzi Yang updated SPARK-41780: --- Description: scala> spark.sql("select regexp_replace('', '[ad]\{0,2}', 'x')").show +

[jira] [Created] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

2022-12-29 Thread Remzi Yang (Jira)
Remzi Yang created SPARK-41780: -- Summary: `regexp_replace('', '[ad]{0, 2}', 'x')` causes an internal error Key: SPARK-41780 URL: https://issues.apache.org/jira/browse/SPARK-41780 Project: Spark

[jira] [Created] (SPARK-41779) Make getitem support filter and select

2022-12-29 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41779: - Summary: Make getitem support filter and select Key: SPARK-41779 URL: https://issues.apache.org/jira/browse/SPARK-41779 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-41771) __getitem__ does not work with Column.isin

2022-12-29 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652996#comment-17652996 ] Ruifeng Zheng commented on SPARK-41771: --- I am working on this one > __getitem__

[jira] [Updated] (SPARK-41774) Remove def test_vectorized_udf_unsupported_types

2022-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41774: -- Issue Type: Test (was: Improvement) > Remove def test_vectorized_udf_unsupported_types >

[jira] [Resolved] (SPARK-41774) Remove def test_vectorized_udf_unsupported_types

2022-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41774. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39293 [https://

[jira] [Assigned] (SPARK-41774) Remove def test_vectorized_udf_unsupported_types

2022-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41774: - Assignee: Bjørn Jørgensen > Remove def test_vectorized_udf_unsupported_types >

[jira] [Updated] (SPARK-41708) Pull v1write information to WriteFiles

2022-12-29 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-41708: -- Summary: Pull v1write information to WriteFiles (was: Pull v1write information to write file node) >

[jira] [Assigned] (SPARK-41743) groupBy(...).agg(...).sort does not actually sort the output

2022-12-29 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41743: - Assignee: Martin Grund > groupBy(...).agg(...).sort does not actually sort the output >

[jira] [Resolved] (SPARK-41743) groupBy(...).agg(...).sort does not actually sort the output

2022-12-29 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41743. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39295 [https://

[jira] [Resolved] (SPARK-41742) Support star in groupBy.agg()

2022-12-29 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41742. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39298 [https://

[jira] [Assigned] (SPARK-41778) Add an alias "reduce" to ArrayAggregate

2022-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41778: - Assignee: Takuya Ueshin > Add an alias "reduce" to ArrayAggregate > ---

[jira] [Resolved] (SPARK-41778) Add an alias "reduce" to ArrayAggregate

2022-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41778. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39297 [https://

[jira] [Assigned] (SPARK-41726) Remove OptimizedCreateHiveTableAsSelectCommand

2022-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41726: - Assignee: XiDuo You > Remove OptimizedCreateHiveTableAsSelectCommand >

[jira] [Resolved] (SPARK-41726) Remove OptimizedCreateHiveTableAsSelectCommand

2022-12-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41726. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39263 [https://

[jira] [Resolved] (SPARK-41429) Protobuf serializer for RDDOperationGraphWrapper

2022-12-29 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41429. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39110 [https:

[jira] [Assigned] (SPARK-41429) Protobuf serializer for RDDOperationGraphWrapper

2022-12-29 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-41429: -- Assignee: Sandeep Singh > Protobuf serializer for RDDOperationGraphWrapper >

[jira] [Commented] (SPARK-41593) Implement logging from the executor nodes

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652970#comment-17652970 ] Apache Spark commented on SPARK-41593: -- User 'rithwik-db' has created a pull reques

[jira] [Assigned] (SPARK-41593) Implement logging from the executor nodes

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41593: Assignee: Apache Spark > Implement logging from the executor nodes >

[jira] [Assigned] (SPARK-41593) Implement logging from the executor nodes

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41593: Assignee: (was: Apache Spark) > Implement logging from the executor nodes > -

[jira] [Assigned] (SPARK-41742) Support star in groupBy.agg()

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41742: Assignee: (was: Apache Spark) > Support star in groupBy.agg() > -

[jira] [Commented] (SPARK-41742) Support star in groupBy.agg()

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652953#comment-17652953 ] Apache Spark commented on SPARK-41742: -- User 'grundprinzip' has created a pull requ

[jira] [Assigned] (SPARK-41742) Support star in groupBy.agg()

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41742: Assignee: Apache Spark > Support star in groupBy.agg() > - >

[jira] [Assigned] (SPARK-41778) Add an alias "reduce" to ArrayAggregate

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41778: Assignee: (was: Apache Spark) > Add an alias "reduce" to ArrayAggregate > ---

[jira] [Commented] (SPARK-41778) Add an alias "reduce" to ArrayAggregate

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652948#comment-17652948 ] Apache Spark commented on SPARK-41778: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-41778) Add an alias "reduce" to ArrayAggregate

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41778: Assignee: Apache Spark > Add an alias "reduce" to ArrayAggregate > --

[jira] [Created] (SPARK-41778) Add an alias "reduce" to ArrayAggregate

2022-12-29 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-41778: - Summary: Add an alias "reduce" to ArrayAggregate Key: SPARK-41778 URL: https://issues.apache.org/jira/browse/SPARK-41778 Project: Spark Issue Type: Improve

[jira] [Assigned] (SPARK-41757) Compatibility of string representation in Column

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41757: Assignee: (was: Apache Spark) > Compatibility of string representation in Column > --

[jira] [Commented] (SPARK-41757) Compatibility of string representation in Column

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652943#comment-17652943 ] Apache Spark commented on SPARK-41757: -- User 'grundprinzip' has created a pull requ

[jira] [Commented] (SPARK-41757) Compatibility of string representation in Column

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652942#comment-17652942 ] Apache Spark commented on SPARK-41757: -- User 'grundprinzip' has created a pull requ

[jira] [Assigned] (SPARK-41757) Compatibility of string representation in Column

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41757: Assignee: Apache Spark > Compatibility of string representation in Column > -

[jira] [Commented] (SPARK-41743) groupBy(...).agg(...).sort does not actually sort the output

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652938#comment-17652938 ] Apache Spark commented on SPARK-41743: -- User 'grundprinzip' has created a pull requ

[jira] [Commented] (SPARK-41743) groupBy(...).agg(...).sort does not actually sort the output

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652937#comment-17652937 ] Apache Spark commented on SPARK-41743: -- User 'grundprinzip' has created a pull requ

[jira] [Assigned] (SPARK-41743) groupBy(...).agg(...).sort does not actually sort the output

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41743: Assignee: Apache Spark > groupBy(...).agg(...).sort does not actually sort the output > -

[jira] [Assigned] (SPARK-41743) groupBy(...).agg(...).sort does not actually sort the output

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41743: Assignee: (was: Apache Spark) > groupBy(...).agg(...).sort does not actually sort the

[jira] [Commented] (SPARK-41743) groupBy(...).agg(...).sort does not actually sort the output

2022-12-29 Thread Martin Grund (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652936#comment-17652936 ] Martin Grund commented on SPARK-41743: -- Running the doc tests actually passes for m

[jira] [Comment Edited] (SPARK-41743) groupBy(...).agg(...).sort does not actually sort the output

2022-12-29 Thread Martin Grund (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652934#comment-17652934 ] Martin Grund edited comment on SPARK-41743 at 12/29/22 8:24 PM: --

[jira] [Commented] (SPARK-41743) groupBy(...).agg(...).sort does not actually sort the output

2022-12-29 Thread Martin Grund (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652934#comment-17652934 ] Martin Grund commented on SPARK-41743: -- In the following example, I cannot reproduc

[jira] [Assigned] (SPARK-41537) Protobuf backwards compatibility testing

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41537: Assignee: Apache Spark (was: Rui Wang) > Protobuf backwards compatibility testing >

[jira] [Commented] (SPARK-41537) Protobuf backwards compatibility testing

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652924#comment-17652924 ] Apache Spark commented on SPARK-41537: -- User 'grundprinzip' has created a pull requ

[jira] [Assigned] (SPARK-41537) Protobuf backwards compatibility testing

2022-12-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41537: Assignee: Rui Wang (was: Apache Spark) > Protobuf backwards compatibility testing >

[jira] [Updated] (SPARK-41775) Implement training functions as input

2022-12-29 Thread Rithwik Ediga Lakhamsani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rithwik Ediga Lakhamsani updated SPARK-41775: - Description: Currently, `Distributor().run(...)` takes only files as inp

[jira] [Updated] (SPARK-41775) Implement training functions as input

2022-12-29 Thread Rithwik Ediga Lakhamsani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rithwik Ediga Lakhamsani updated SPARK-41775: - Description: Currently, `Distributor().run(...)` takes only files as inp

  1   2   >