[jira] [Assigned] (SPARK-41767) Implement `Column.{withField, dropFields}`

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41767: Assignee: Apache Spark > Implement `Column.{withField, dropFields}` > ---

[jira] [Commented] (SPARK-41767) Implement `Column.{withField, dropFields}`

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652746#comment-17652746 ] Apache Spark commented on SPARK-41767: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41767) Implement `Column.{withField, dropFields}`

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41767: Assignee: (was: Apache Spark) > Implement `Column.{withField, dropFields}` >

[jira] [Created] (SPARK-41767) Implement `Column.{withField, dropFields}`

2022-12-28 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41767: - Summary: Implement `Column.{withField, dropFields}` Key: SPARK-41767 URL: https://issues.apache.org/jira/browse/SPARK-41767 Project: Spark Issue Type: Sub-

[jira] [Commented] (SPARK-41581) Assign name to _LEGACY_ERROR_TEMP_1230

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652740#comment-17652740 ] Apache Spark commented on SPARK-41581: -- User 'itholic' has created a pull request f

[jira] [Commented] (SPARK-41581) Assign name to _LEGACY_ERROR_TEMP_1230

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652738#comment-17652738 ] Apache Spark commented on SPARK-41581: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-41581) Assign name to _LEGACY_ERROR_TEMP_1230

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41581: Assignee: (was: Apache Spark) > Assign name to _LEGACY_ERROR_TEMP_1230 >

[jira] [Assigned] (SPARK-41581) Assign name to _LEGACY_ERROR_TEMP_1230

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41581: Assignee: Apache Spark > Assign name to _LEGACY_ERROR_TEMP_1230 > ---

[jira] [Assigned] (SPARK-41576) Assign name to _LEGACY_ERROR_TEMP_2051

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41576: Assignee: Apache Spark > Assign name to _LEGACY_ERROR_TEMP_2051 > ---

[jira] [Assigned] (SPARK-41576) Assign name to _LEGACY_ERROR_TEMP_2051

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41576: Assignee: (was: Apache Spark) > Assign name to _LEGACY_ERROR_TEMP_2051 >

[jira] [Commented] (SPARK-41576) Assign name to _LEGACY_ERROR_TEMP_2051

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652716#comment-17652716 ] Apache Spark commented on SPARK-41576: -- User 'itholic' has created a pull request f

[jira] [Resolved] (SPARK-41685) Support optional using Protobuf serializer for KVStore in History server

2022-12-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41685. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39202 [https:

[jira] [Assigned] (SPARK-41685) Support optional using Protobuf serializer for KVStore in History server

2022-12-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-41685: -- Assignee: Gengliang Wang > Support optional using Protobuf serializer for KVStore in

[jira] [Assigned] (SPARK-41766) Handle decommission request sent before executor registration

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41766: Assignee: (was: Apache Spark) > Handle decommission request sent before executor regi

[jira] [Assigned] (SPARK-41766) Handle decommission request sent before executor registration

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41766: Assignee: Apache Spark > Handle decommission request sent before executor registration >

[jira] [Commented] (SPARK-41766) Handle decommission request sent before executor registration

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652711#comment-17652711 ] Apache Spark commented on SPARK-41766: -- User 'warrenzhu25' has created a pull reque

[jira] [Updated] (SPARK-41766) Handle decommission request sent before executor registration

2022-12-28 Thread Zhongwei Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongwei Zhu updated SPARK-41766: - Summary: Handle decommission request sent before executor registration (was: Handle decommissio

[jira] [Created] (SPARK-41766) Handle decommission request for unregistered executor

2022-12-28 Thread Zhongwei Zhu (Jira)
Zhongwei Zhu created SPARK-41766: Summary: Handle decommission request for unregistered executor Key: SPARK-41766 URL: https://issues.apache.org/jira/browse/SPARK-41766 Project: Spark Issue T

[jira] [Assigned] (SPARK-41578) Assign name to _LEGACY_ERROR_TEMP_2141

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41578: Assignee: (was: Apache Spark) > Assign name to _LEGACY_ERROR_TEMP_2141 >

[jira] [Commented] (SPARK-41578) Assign name to _LEGACY_ERROR_TEMP_2141

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652704#comment-17652704 ] Apache Spark commented on SPARK-41578: -- User 'itholic' has created a pull request f

[jira] [Commented] (SPARK-41578) Assign name to _LEGACY_ERROR_TEMP_2141

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652705#comment-17652705 ] Apache Spark commented on SPARK-41578: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-41578) Assign name to _LEGACY_ERROR_TEMP_2141

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41578: Assignee: Apache Spark > Assign name to _LEGACY_ERROR_TEMP_2141 > ---

[jira] [Assigned] (SPARK-41751) Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull,isNull,isin

2022-12-28 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41751: - Assignee: Ruifeng Zheng > Support > Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,

[jira] [Resolved] (SPARK-41751) Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull,isNull,isin

2022-12-28 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41751. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39272 [https://

[jira] [Commented] (SPARK-41763) Improve v1 writes

2022-12-28 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652692#comment-17652692 ] XiDuo You commented on SPARK-41763: --- cc [~allisonwang] [~cloud_fan] FYI > Improve v1

[jira] [Created] (SPARK-41765) Move v1 write metrics to WriteFiles

2022-12-28 Thread XiDuo You (Jira)
XiDuo You created SPARK-41765: - Summary: Move v1 write metrics to WriteFiles Key: SPARK-41765 URL: https://issues.apache.org/jira/browse/SPARK-41765 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-41764) Make the internal string op name consistent with FunctionRegistry

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41764: Assignee: Apache Spark > Make the internal string op name consistent with FunctionRegistr

[jira] [Commented] (SPARK-41764) Make the internal string op name consistent with FunctionRegistry

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652689#comment-17652689 ] Apache Spark commented on SPARK-41764: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41764) Make the internal string op name consistent with FunctionRegistry

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41764: Assignee: (was: Apache Spark) > Make the internal string op name consistent with Func

[jira] [Commented] (SPARK-41764) Make the internal string op name consistent with FunctionRegistry

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652690#comment-17652690 ] Apache Spark commented on SPARK-41764: -- User 'zhengruifeng' has created a pull requ

[jira] [Updated] (SPARK-41708) Pull v1write information to write file node

2022-12-28 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-41708: -- Parent: SPARK-41763 Issue Type: Sub-task (was: Improvement) > Pull v1write information to wri

[jira] [Updated] (SPARK-41726) Remove OptimizedCreateHiveTableAsSelectCommand

2022-12-28 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-41726: -- Parent: SPARK-41763 Issue Type: Sub-task (was: Improvement) > Remove OptimizedCreateHiveTable

[jira] [Updated] (SPARK-41713) Make CTAS hold a nested execution for data writing

2022-12-28 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-41713: -- Parent: SPARK-41763 Issue Type: Sub-task (was: Improvement) > Make CTAS hold a nested executi

[jira] [Updated] (SPARK-41407) Pull out v1 write to WriteFiles

2022-12-28 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-41407: -- Parent: SPARK-41763 Issue Type: Sub-task (was: Improvement) > Pull out v1 write to WriteFiles

[jira] [Updated] (SPARK-40201) Improve v1 write test coverage

2022-12-28 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-40201: -- Parent: SPARK-41763 Issue Type: Sub-task (was: Improvement) > Improve v1 write test coverage

[jira] [Updated] (SPARK-41764) Make the internal string op name consistent with FunctionRegistry

2022-12-28 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-41764: -- Summary: Make the internal string op name consistent with FunctionRegistry (was: Make the int

[jira] [Updated] (SPARK-40107) Pull out empty2null conversion from FileFormatWriter

2022-12-28 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-40107: -- Parent: SPARK-41763 Issue Type: Sub-task (was: Improvement) > Pull out empty2null conversion

[jira] [Created] (SPARK-41764) Make the internal op name consistent with FunctionRegistry

2022-12-28 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41764: - Summary: Make the internal op name consistent with FunctionRegistry Key: SPARK-41764 URL: https://issues.apache.org/jira/browse/SPARK-41764 Project: Spark

[jira] [Updated] (SPARK-37194) Avoid unnecessary sort in FileFormatWriter if it's not dynamic partition

2022-12-28 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37194: -- Parent: SPARK-41763 Issue Type: Sub-task (was: Improvement) > Avoid unnecessary sort in FileF

[jira] [Updated] (SPARK-37287) Pull out dynamic partition and bucket sort from FileFormatWriter

2022-12-28 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37287: -- Parent: SPARK-41763 Issue Type: Sub-task (was: Improvement) > Pull out dynamic partition and

[jira] [Created] (SPARK-41763) Improve v1 writes

2022-12-28 Thread XiDuo You (Jira)
XiDuo You created SPARK-41763: - Summary: Improve v1 writes Key: SPARK-41763 URL: https://issues.apache.org/jira/browse/SPARK-41763 Project: Spark Issue Type: Umbrella Components: SQL

[jira] [Assigned] (SPARK-41708) Pull v1write information to write file node

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41708: Assignee: (was: Apache Spark) > Pull v1write information to write file node > ---

[jira] [Commented] (SPARK-41708) Pull v1write information to write file node

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652687#comment-17652687 ] Apache Spark commented on SPARK-41708: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-41708) Pull v1write information to write file node

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41708: Assignee: Apache Spark > Pull v1write information to write file node > --

[jira] [Assigned] (SPARK-41761) Fix arithmetic ops: negate, pow

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41761: Assignee: (was: Apache Spark) > Fix arithmetic ops: negate, pow > ---

[jira] [Assigned] (SPARK-41761) Fix arithmetic ops: negate, pow

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41761: Assignee: Apache Spark > Fix arithmetic ops: negate, pow > --

[jira] [Commented] (SPARK-41761) Fix arithmetic ops: negate, pow

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652686#comment-17652686 ] Apache Spark commented on SPARK-41761: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-41759) Use `weakIntern` on string values in create new objects during deserialization

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652685#comment-17652685 ] Apache Spark commented on SPARK-41759: -- User 'panbingkun' has created a pull reques

[jira] [Assigned] (SPARK-41759) Use `weakIntern` on string values in create new objects during deserialization

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41759: Assignee: (was: Apache Spark) > Use `weakIntern` on string values in create new objec

[jira] [Commented] (SPARK-41759) Use `weakIntern` on string values in create new objects during deserialization

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652683#comment-17652683 ] Apache Spark commented on SPARK-41759: -- User 'panbingkun' has created a pull reques

[jira] [Assigned] (SPARK-41759) Use `weakIntern` on string values in create new objects during deserialization

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41759: Assignee: Apache Spark > Use `weakIntern` on string values in create new objects during d

[jira] [Created] (SPARK-41762) Make `Column.__neg__` return the same column name as PySpark

2022-12-28 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41762: - Summary: Make `Column.__neg__` return the same column name as PySpark Key: SPARK-41762 URL: https://issues.apache.org/jira/browse/SPARK-41762 Project: Spark

[jira] [Created] (SPARK-41761) Fix arithmetic ops: negate, pow

2022-12-28 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41761: - Summary: Fix arithmetic ops: negate, pow Key: SPARK-41761 URL: https://issues.apache.org/jira/browse/SPARK-41761 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-41760) Enforce scalafmt for Spark Connect Client module

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652682#comment-17652682 ] Apache Spark commented on SPARK-41760: -- User 'dengziming' has created a pull reques

[jira] [Assigned] (SPARK-41760) Enforce scalafmt for Spark Connect Client module

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41760: Assignee: Apache Spark > Enforce scalafmt for Spark Connect Client module > -

[jira] [Assigned] (SPARK-41760) Enforce scalafmt for Spark Connect Client module

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41760: Assignee: (was: Apache Spark) > Enforce scalafmt for Spark Connect Client module > --

[jira] [Commented] (SPARK-41760) Enforce scalafmt for Spark Connect Client module

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652681#comment-17652681 ] Apache Spark commented on SPARK-41760: -- User 'dengziming' has created a pull reques

[jira] [Commented] (SPARK-41751) Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull,isNull,isin

2022-12-28 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652677#comment-17652677 ] Ruifeng Zheng commented on SPARK-41751: --- {isin} was added in https://issues.apache

[jira] [Created] (SPARK-41760) Enforce scalafmt for Spark Connect Client module

2022-12-28 Thread Deng Ziming (Jira)
Deng Ziming created SPARK-41760: --- Summary: Enforce scalafmt for Spark Connect Client module Key: SPARK-41760 URL: https://issues.apache.org/jira/browse/SPARK-41760 Project: Spark Issue Type: Im

[jira] [Comment Edited] (SPARK-41758) Support Window.rowsBetween

2022-12-28 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652666#comment-17652666 ] Ruifeng Zheng edited comment on SPARK-41758 at 12/29/22 3:08 AM: -

[jira] [Commented] (SPARK-41756) Support Column.getField, Column.getItem

2022-12-28 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652676#comment-17652676 ] Ruifeng Zheng commented on SPARK-41756: --- they were added in https://issues.apache.

[jira] [Commented] (SPARK-41751) Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull,isNull,isin

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652674#comment-17652674 ] Apache Spark commented on SPARK-41751: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-41759) Use `weakIntern` on string values in create new objects during deserialization

2022-12-28 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652673#comment-17652673 ] BingKun Pan commented on SPARK-41759: - I Work on it. > Use `weakIntern` on string v

[jira] [Created] (SPARK-41759) Use `weakIntern` on string values in create new objects during deserialization

2022-12-28 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-41759: --- Summary: Use `weakIntern` on string values in create new objects during deserialization Key: SPARK-41759 URL: https://issues.apache.org/jira/browse/SPARK-41759 Project:

[jira] [Commented] (SPARK-41751) Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull,isNull,isin

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652669#comment-17652669 ] Apache Spark commented on SPARK-41751: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41751) Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull,isNull,isin

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41751: Assignee: (was: Apache Spark) > Support > Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqN

[jira] [Commented] (SPARK-41751) Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull,isNull,isin

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652668#comment-17652668 ] Apache Spark commented on SPARK-41751: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41751) Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull,isNull,isin

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41751: Assignee: Apache Spark > Support > Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isN

[jira] [Assigned] (SPARK-41753) Add tests for ArrayZip to check the result size and nullability.

2022-12-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41753: --- Assignee: Takuya Ueshin > Add tests for ArrayZip to check the result size and nullability.

[jira] [Resolved] (SPARK-41753) Add tests for ArrayZip to check the result size and nullability.

2022-12-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41753. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39266 [https://gith

[jira] [Commented] (SPARK-41758) Support Window.rowsBetween

2022-12-28 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652666#comment-17652666 ] Ruifeng Zheng commented on SPARK-41758: --- it is a bit weird that `Window.rowsBetwee

[jira] [Updated] (SPARK-41758) Support Window.rowsBetween

2022-12-28 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41758: -- Description: Doctest in pyspark.sql.connect.window.Window.rowsBetween fails with the error be

[jira] [Updated] (SPARK-41758) Support Window.rowsBetween

2022-12-28 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41758: -- Description: Doctest in pyspark.sql.connect.window.Window.bitwiseAnd fails with the error bel

[jira] [Created] (SPARK-41758) Support Window.rowsBetween

2022-12-28 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41758: - Summary: Support Window.rowsBetween Key: SPARK-41758 URL: https://issues.apache.org/jira/browse/SPARK-41758 Project: Spark Issue Type: Sub-task C

[jira] [Updated] (SPARK-41751) Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull,isNull,isin

2022-12-28 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41751: -- Summary: Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull,isNull,isin (was

[jira] [Updated] (SPARK-41751) Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull

2022-12-28 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41751: -- Summary: Support Column.bitwiseAND,bitwiseOR,bitwiseXOR,eqNullSafe,isNotNull (was: Support C

[jira] [Updated] (SPARK-41757) Compatibility of string representation in Column

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41757: - Parent: SPARK-41282 Issue Type: Sub-task (was: Bug) > Compatibility of string represent

[jira] [Updated] (SPARK-41757) Compatibility of string representation in Column

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41757: - Epic Link: SPARK-39375 > Compatibility of string representation in Column >

[jira] [Updated] (SPARK-41757) Compatibility of string representation in Column

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41757: - Epic Link: (was: SPARK-39375) > Compatibility of string representation in Column > -

[jira] [Updated] (SPARK-41757) Compatibility of string representation in Column

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41757: - Parent: (was: SPARK-41279) Issue Type: Bug (was: Sub-task) > Compatibility of strin

[jira] [Updated] (SPARK-41751) Support Column.bitwiseAND

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41751: - Epic Link: (was: SPARK-39375) > Support Column.bitwiseAND > - > >

[jira] [Updated] (SPARK-41756) Support Column.getField, Column.getItem

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41756: - Parent: (was: SPARK-41279) Issue Type: Bug (was: Sub-task) > Support Column.getFiel

[jira] [Updated] (SPARK-41751) Support Column.bitwiseAND

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41751: - Parent: SPARK-41282 Issue Type: Sub-task (was: Bug) > Support Column.bitwiseAND > -

[jira] [Updated] (SPARK-41756) Support Column.getField, Column.getItem

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41756: - Epic Link: (was: SPARK-39375) > Support Column.getField, Column.getItem > --

[jira] [Updated] (SPARK-41756) Support Column.getField, Column.getItem

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41756: - Parent: SPARK-41282 Issue Type: Sub-task (was: Bug) > Support Column.getField, Column.g

[jira] [Updated] (SPARK-41756) Support Column.getField, Column.getItem

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41756: - Epic Link: SPARK-39375 > Support Column.getField, Column.getItem > -

[jira] [Updated] (SPARK-41751) Support Column.bitwiseAND

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41751: - Parent: (was: SPARK-41279) Issue Type: Bug (was: Sub-task) > Support Column.bitwise

[jira] [Updated] (SPARK-41751) Support Column.bitwiseAND

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41751: - Epic Link: SPARK-39375 > Support Column.bitwiseAND > - > >

[jira] [Resolved] (SPARK-41748) Support multiple arguments in groupBy.min(...)

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41748. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39271 [https://gi

[jira] [Assigned] (SPARK-41747) Support multiple arguments in groupBy.avg(...)

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41747: Assignee: Hyukjin Kwon > Support multiple arguments in groupBy.avg(...) > ---

[jira] [Resolved] (SPARK-41749) Support multiple arguments in groupBy.sum(...)

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41749. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39271 [https://gi

[jira] [Resolved] (SPARK-41744) Support multiple arguments in groupBy.max(...)

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41744. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39271 [https://gi

[jira] [Assigned] (SPARK-41748) Support multiple arguments in groupBy.min(...)

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41748: Assignee: Hyukjin Kwon > Support multiple arguments in groupBy.min(...) > ---

[jira] [Assigned] (SPARK-41744) Support multiple arguments in groupBy.max(...)

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41744: Assignee: Hyukjin Kwon > Support multiple arguments in groupBy.max(...) > ---

[jira] [Resolved] (SPARK-41747) Support multiple arguments in groupBy.avg(...)

2022-12-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41747. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39271 [https://gi

[jira] [Created] (SPARK-41757) Compatibility of string representation in Column

2022-12-28 Thread Sandeep Singh (Jira)
Sandeep Singh created SPARK-41757: - Summary: Compatibility of string representation in Column Key: SPARK-41757 URL: https://issues.apache.org/jira/browse/SPARK-41757 Project: Spark Issue Type

[jira] [Updated] (SPARK-41757) Compatibility of string representation in Column

2022-12-28 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41757: -- Description: Doctest in pyspark.sql.connect.column.Columnfails with the error below: {code:jav

[jira] [Updated] (SPARK-41756) Support Column.getField, Column.getItem

2022-12-28 Thread Sandeep Singh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Singh updated SPARK-41756: -- Description: Doctest in pyspark.sql.connect.column.Column.getField fails with the error below

[jira] [Commented] (SPARK-41749) Support multiple arguments in groupBy.sum(...)

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652654#comment-17652654 ] Apache Spark commented on SPARK-41749: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-41749) Support multiple arguments in groupBy.sum(...)

2022-12-28 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652655#comment-17652655 ] Apache Spark commented on SPARK-41749: -- User 'HyukjinKwon' has created a pull reque

  1   2   3   >