[jira] [Created] (SPARK-41961) Support table-valued functions with LATERAL

Allison Wang created SPARK-41961: Summary: Support table-valued functions with LATERAL Key: SPARK-41961 URL: https://issues.apache.org/jira/browse/SPARK-41961 Project: Spark Issue Type: Sub-t

[jira] [Comment Edited] (SPARK-41953) Shuffle output location refetch during shuffle migration in decommission

[ https://issues.apache.org/jira/browse/SPARK-41953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656480#comment-17656480 ] Mridul Muralidharan edited comment on SPARK-41953 at 1/10/23 7:48 AM:

[jira] [Commented] (SPARK-41953) Shuffle output location refetch during shuffle migration in decommission

[ https://issues.apache.org/jira/browse/SPARK-41953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656480#comment-17656480 ] Mridul Muralidharan commented on SPARK-41953: - A few things: * Looking at

[jira] [Commented] (SPARK-41934) Add the unsupported function list for `session`

[ https://issues.apache.org/jira/browse/SPARK-41934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656465#comment-17656465 ] Apache Spark commented on SPARK-41934: -- User 'zhengruifeng' has created a pull requ

[jira] [Comment Edited] (SPARK-40885) Spark will filter out data field sorting when dynamic partitions and data fields are sorted at the same time

[ https://issues.apache.org/jira/browse/SPARK-40885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656459#comment-17656459 ] Enrico Minack edited comment on SPARK-40885 at 1/10/23 7:02 AM: --

[jira] [Comment Edited] (SPARK-40885) Spark will filter out data field sorting when dynamic partitions and data fields are sorted at the same time

[ https://issues.apache.org/jira/browse/SPARK-40885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656459#comment-17656459 ] Enrico Minack edited comment on SPARK-40885 at 1/10/23 7:00 AM: --

[jira] [Commented] (SPARK-40885) Spark will filter out data field sorting when dynamic partitions and data fields are sorted at the same time

[ https://issues.apache.org/jira/browse/SPARK-40885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656459#comment-17656459 ] Enrico Minack commented on SPARK-40885: --- This should be fixed by SPARK-40885. > S

[jira] [Created] (SPARK-41960) Assign name to _LEGACY_ERROR_TEMP_1056

Haejoon Lee created SPARK-41960: --- Summary: Assign name to _LEGACY_ERROR_TEMP_1056 Key: SPARK-41960 URL: https://issues.apache.org/jira/browse/SPARK-41960 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-41595) Support generator function explode/explode_outer in the FROM clause

[ https://issues.apache.org/jira/browse/SPARK-41595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41595. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39133 [https://gith

[jira] [Assigned] (SPARK-41595) Support generator function explode/explode_outer in the FROM clause

[ https://issues.apache.org/jira/browse/SPARK-41595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41595: --- Assignee: Allison Wang > Support generator function explode/explode_outer in the FROM claus

[jira] [Commented] (SPARK-41872) Fix DataFrame createDataframe handling of None

[ https://issues.apache.org/jira/browse/SPARK-41872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656437#comment-17656437 ] Apache Spark commented on SPARK-41872: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41872) Fix DataFrame createDataframe handling of None

[ https://issues.apache.org/jira/browse/SPARK-41872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41872: Assignee: Apache Spark > Fix DataFrame createDataframe handling of None > ---

[jira] [Assigned] (SPARK-41872) Fix DataFrame createDataframe handling of None

[ https://issues.apache.org/jira/browse/SPARK-41872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41872: Assignee: (was: Apache Spark) > Fix DataFrame createDataframe handling of None >

[jira] [Assigned] (SPARK-41957) Enable the doctest for `DataFrame.hint`

[ https://issues.apache.org/jira/browse/SPARK-41957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41957: - Assignee: Ruifeng Zheng > Enable the doctest for `DataFrame.hint` > ---

[jira] [Resolved] (SPARK-41957) Enable the doctest for `DataFrame.hint`

[ https://issues.apache.org/jira/browse/SPARK-41957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41957. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39472 [https://

[jira] [Commented] (SPARK-41907) Function `sampleby` return parity

[ https://issues.apache.org/jira/browse/SPARK-41907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656431#comment-17656431 ] Apache Spark commented on SPARK-41907: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-41907) Function `sampleby` return parity

[ https://issues.apache.org/jira/browse/SPARK-41907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41907: Assignee: (was: Apache Spark) > Function `sampleby` return parity > -

[jira] [Commented] (SPARK-41907) Function `sampleby` return parity

[ https://issues.apache.org/jira/browse/SPARK-41907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656430#comment-17656430 ] Apache Spark commented on SPARK-41907: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-41907) Function `sampleby` return parity

[ https://issues.apache.org/jira/browse/SPARK-41907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41907: Assignee: Apache Spark > Function `sampleby` return parity >

[jira] (SPARK-41907) Function `sampleby` return parity

[ https://issues.apache.org/jira/browse/SPARK-41907 ] jiaan.geng deleted comment on SPARK-41907: was (Author: beliefer): I want take a look. > Function `sampleby` return parity > - > > Key: SPARK-

[jira] [Commented] (SPARK-41907) Function `sampleby` return parity

[ https://issues.apache.org/jira/browse/SPARK-41907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656429#comment-17656429 ] jiaan.geng commented on SPARK-41907: The root cause is the plan is different from py

[jira] [Resolved] (SPARK-41914) Sorting issue with partitioned-writing and planned write optimization disabled

[ https://issues.apache.org/jira/browse/SPARK-41914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41914. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39431 [https://gith

[jira] [Assigned] (SPARK-41914) Sorting issue with partitioned-writing and planned write optimization disabled

[ https://issues.apache.org/jira/browse/SPARK-41914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-41914: --- Assignee: Enrico Minack > Sorting issue with partitioned-writing and planned write optimiza

[jira] [Commented] (SPARK-41959) Improve v1 writes with empty2null

[ https://issues.apache.org/jira/browse/SPARK-41959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656416#comment-17656416 ] Apache Spark commented on SPARK-41959: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-41959) Improve v1 writes with empty2null

[ https://issues.apache.org/jira/browse/SPARK-41959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41959: Assignee: (was: Apache Spark) > Improve v1 writes with empty2null > -

[jira] [Assigned] (SPARK-41959) Improve v1 writes with empty2null

[ https://issues.apache.org/jira/browse/SPARK-41959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41959: Assignee: Apache Spark > Improve v1 writes with empty2null >

[jira] [Commented] (SPARK-41959) Improve v1 writes with empty2null

[ https://issues.apache.org/jira/browse/SPARK-41959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656415#comment-17656415 ] Apache Spark commented on SPARK-41959: -- User 'ulysses-you' has created a pull reque

[jira] [Created] (SPARK-41959) Improve v1 writes with empty2null

XiDuo You created SPARK-41959: - Summary: Improve v1 writes with empty2null Key: SPARK-41959 URL: https://issues.apache.org/jira/browse/SPARK-41959 Project: Spark Issue Type: Sub-task Co

[jira] [Commented] (SPARK-41887) Support DataFrame hint parameter to be list

[ https://issues.apache.org/jira/browse/SPARK-41887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656407#comment-17656407 ] Ruifeng Zheng commented on SPARK-41887: --- I am going to work on this one > Support

[jira] [Commented] (SPARK-41958) Disallow arbitrary custom classpath with proxy user in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-41958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656397#comment-17656397 ] Apache Spark commented on SPARK-41958: -- User 'Ngone51' has created a pull request f

[jira] [Commented] (SPARK-41958) Disallow arbitrary custom classpath with proxy user in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-41958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656396#comment-17656396 ] Apache Spark commented on SPARK-41958: -- User 'Ngone51' has created a pull request f

[jira] [Assigned] (SPARK-41958) Disallow arbitrary custom classpath with proxy user in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-41958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41958: Assignee: Apache Spark > Disallow arbitrary custom classpath with proxy user in cluster m

[jira] [Assigned] (SPARK-41958) Disallow arbitrary custom classpath with proxy user in cluster mode

[ https://issues.apache.org/jira/browse/SPARK-41958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41958: Assignee: (was: Apache Spark) > Disallow arbitrary custom classpath with proxy user i

[jira] [Created] (SPARK-41958) Disallow arbitrary custom classpath with proxy user in cluster mode

wuyi created SPARK-41958: Summary: Disallow arbitrary custom classpath with proxy user in cluster mode Key: SPARK-41958 URL: https://issues.apache.org/jira/browse/SPARK-41958 Project: Spark Issue Ty

[jira] [Commented] (SPARK-41919) Unify the schema or datatype in protos

[ https://issues.apache.org/jira/browse/SPARK-41919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656387#comment-17656387 ] Ruifeng Zheng commented on SPARK-41919: --- actually, JSON is the standard format of

[jira] [Commented] (SPARK-38173) Quoted column cannot be recognized correctly when quotedRegexColumnNames is true

[ https://issues.apache.org/jira/browse/SPARK-38173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656380#comment-17656380 ] Apache Spark commented on SPARK-38173: -- User 'huaxingao' has created a pull request

[jira] [Resolved] (SPARK-41943) Use java api to create files and grant permissions is DiskBlockManager

[ https://issues.apache.org/jira/browse/SPARK-41943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-41943. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39448 [https://gith

[jira] [Commented] (SPARK-38173) Quoted column cannot be recognized correctly when quotedRegexColumnNames is true

[ https://issues.apache.org/jira/browse/SPARK-38173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656381#comment-17656381 ] Apache Spark commented on SPARK-38173: -- User 'huaxingao' has created a pull request

[jira] [Commented] (SPARK-41957) Enable the doctest for `DataFrame.hint`

[ https://issues.apache.org/jira/browse/SPARK-41957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656378#comment-17656378 ] Apache Spark commented on SPARK-41957: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41957) Enable the doctest for `DataFrame.hint`

[ https://issues.apache.org/jira/browse/SPARK-41957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41957: Assignee: (was: Apache Spark) > Enable the doctest for `DataFrame.hint` > ---

[jira] [Assigned] (SPARK-41957) Enable the doctest for `DataFrame.hint`

[ https://issues.apache.org/jira/browse/SPARK-41957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41957: Assignee: Apache Spark > Enable the doctest for `DataFrame.hint` > --

[jira] [Commented] (SPARK-41907) Function `sampleby` return parity

[ https://issues.apache.org/jira/browse/SPARK-41907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656376#comment-17656376 ] jiaan.geng commented on SPARK-41907: I want take a look. > Function `sampleby` retu

[jira] [Created] (SPARK-41957) Enable the doctest for `DataFrame.hint`

Ruifeng Zheng created SPARK-41957: - Summary: Enable the doctest for `DataFrame.hint` Key: SPARK-41957 URL: https://issues.apache.org/jira/browse/SPARK-41957 Project: Spark Issue Type: Sub-tas

[jira] [Resolved] (SPARK-41904) Fix Function `nth_value` functions output

[ https://issues.apache.org/jira/browse/SPARK-41904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng resolved SPARK-41904. Resolution: Won't Fix > Fix Function `nth_value` functions output > --

[jira] [Commented] (SPARK-41904) Fix Function `nth_value` functions output

[ https://issues.apache.org/jira/browse/SPARK-41904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656374#comment-17656374 ] jiaan.geng commented on SPARK-41904: The root reason caused by https://issues.apache

[jira] [Created] (SPARK-41956) Shuffle output location refetch in ShuffleBlockFetcherIterator

Zhongwei Zhu created SPARK-41956: Summary: Shuffle output location refetch in ShuffleBlockFetcherIterator Key: SPARK-41956 URL: https://issues.apache.org/jira/browse/SPARK-41956 Project: Spark

[jira] [Created] (SPARK-41955) Support fetch latest map output from worker

Zhongwei Zhu created SPARK-41955: Summary: Support fetch latest map output from worker Key: SPARK-41955 URL: https://issues.apache.org/jira/browse/SPARK-41955 Project: Spark Issue Type: Sub-

[jira] [Created] (SPARK-41954) Add isDecommissioned in ExecutorDeadException

Zhongwei Zhu created SPARK-41954: Summary: Add isDecommissioned in ExecutorDeadException Key: SPARK-41954 URL: https://issues.apache.org/jira/browse/SPARK-41954 Project: Spark Issue Type: Sub

[jira] [Commented] (SPARK-41953) Shuffle output location refetch during shuffle migration in decommission

[ https://issues.apache.org/jira/browse/SPARK-41953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656365#comment-17656365 ] Zhongwei Zhu commented on SPARK-41953: -- [~dongjoon] [~mridulm80] [~Ngone51] Any com

[jira] [Updated] (SPARK-41953) Shuffle output location refetch during shuffle migration in decommission

[ https://issues.apache.org/jira/browse/SPARK-41953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongwei Zhu updated SPARK-41953: - Description: When shuffle migration enabled during spark decommissionm, shuffle data will be mi

[jira] [Assigned] (SPARK-41232) High-order function: array_append

[ https://issues.apache.org/jira/browse/SPARK-41232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41232: Assignee: Ankit Prakash Gupta (was: Senthil Kumar) > High-order function: array_append >

[jira] [Created] (SPARK-41953) Shuffle output location refetch during shuffle migration in decommission

Zhongwei Zhu created SPARK-41953: Summary: Shuffle output location refetch during shuffle migration in decommission Key: SPARK-41953 URL: https://issues.apache.org/jira/browse/SPARK-41953 Project: Spa

[jira] [Resolved] (SPARK-41232) High-order function: array_append

[ https://issues.apache.org/jira/browse/SPARK-41232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41232. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38865 [https://gi

[jira] [Assigned] (SPARK-41232) High-order function: array_append

[ https://issues.apache.org/jira/browse/SPARK-41232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41232: Assignee: Senthil Kumar > High-order function: array_append > ---

[jira] [Commented] (SPARK-41432) Protobuf serializer for SparkPlanGraphWrapper

[ https://issues.apache.org/jira/browse/SPARK-41432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656340#comment-17656340 ] Apache Spark commented on SPARK-41432: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-41432) Protobuf serializer for SparkPlanGraphWrapper

[ https://issues.apache.org/jira/browse/SPARK-41432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656339#comment-17656339 ] Apache Spark commented on SPARK-41432: -- User 'gengliangwang' has created a pull req

[jira] [Updated] (SPARK-41952) Upgrade Parquet to fix off-heap memory leaks in Zstd codec

[ https://issues.apache.org/jira/browse/SPARK-41952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Kudinkin updated SPARK-41952: Shepherd: Dongjoon Hyun > Upgrade Parquet to fix off-heap memory leaks in Zstd codec > ---

[jira] [Created] (SPARK-41952) Upgrade Parquet to fix off-heap memory leaks in Zstd codec

Alexey Kudinkin created SPARK-41952: --- Summary: Upgrade Parquet to fix off-heap memory leaks in Zstd codec Key: SPARK-41952 URL: https://issues.apache.org/jira/browse/SPARK-41952 Project: Spark

[jira] [Resolved] (SPARK-41951) Update SQL migration guide and documentations

[ https://issues.apache.org/jira/browse/SPARK-41951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41951. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39470 [https://

[jira] [Assigned] (SPARK-41951) Update SQL migration guide and documentations

[ https://issues.apache.org/jira/browse/SPARK-41951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41951: - Assignee: Dongjoon Hyun > Update SQL migration guide and documentations > -

[jira] [Assigned] (SPARK-41951) Update SQL migration guide and documentations

[ https://issues.apache.org/jira/browse/SPARK-41951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41951: Assignee: Apache Spark > Update SQL migration guide and documentations >

[jira] [Commented] (SPARK-41951) Update SQL migration guide and documentations

[ https://issues.apache.org/jira/browse/SPARK-41951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656321#comment-17656321 ] Apache Spark commented on SPARK-41951: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-41951) Update SQL migration guide and documentations

[ https://issues.apache.org/jira/browse/SPARK-41951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41951: Assignee: (was: Apache Spark) > Update SQL migration guide and documentations > -

[jira] [Created] (SPARK-41951) Update SQL migration guide and documentations

Dongjoon Hyun created SPARK-41951: - Summary: Update SQL migration guide and documentations Key: SPARK-41951 URL: https://issues.apache.org/jira/browse/SPARK-41951 Project: Spark Issue Type: D

[jira] [Assigned] (SPARK-41947) Update the contents of error class guidelines

[ https://issues.apache.org/jira/browse/SPARK-41947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41947: Assignee: Haejoon Lee > Update the contents of error class guidelines > -

[jira] [Resolved] (SPARK-41947) Update the contents of error class guidelines

[ https://issues.apache.org/jira/browse/SPARK-41947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41947. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39464 [https://github.com

[jira] [Resolved] (SPARK-41848) Tasks are over-scheduled with TaskResourceProfile

[ https://issues.apache.org/jira/browse/SPARK-41848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41848. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39410 [https://

[jira] [Assigned] (SPARK-41848) Tasks are over-scheduled with TaskResourceProfile

[ https://issues.apache.org/jira/browse/SPARK-41848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41848: - Assignee: huangtengfei > Tasks are over-scheduled with TaskResourceProfile > --

[jira] [Resolved] (SPARK-41860) Make AvroScanBuilder and JsonScanBuilder case classes

[ https://issues.apache.org/jira/browse/SPARK-41860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41860. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39366 [https://

[jira] [Assigned] (SPARK-41860) Make AvroScanBuilder and JsonScanBuilder case classes

[ https://issues.apache.org/jira/browse/SPARK-41860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41860: - Assignee: Lorenzo Martini > Make AvroScanBuilder and JsonScanBuilder case classes > ---

[jira] [Resolved] (SPARK-41147) Assign a name to the legacy error class _LEGACY_ERROR_TEMP_1042

[ https://issues.apache.org/jira/browse/SPARK-41147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-41147. - Resolution: Duplicate > Assign a name to the legacy error class _LEGACY_ERROR_TEMP_1042 > --

[jira] [Commented] (SPARK-41708) Pull v1write information to WriteFiles

[ https://issues.apache.org/jira/browse/SPARK-41708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656105#comment-17656105 ] Apache Spark commented on SPARK-41708: -- User 'ulysses-you' has created a pull reque

[jira] [Commented] (SPARK-41708) Pull v1write information to WriteFiles

[ https://issues.apache.org/jira/browse/SPARK-41708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656104#comment-17656104 ] Apache Spark commented on SPARK-41708: -- User 'ulysses-you' has created a pull reque

[jira] [Assigned] (SPARK-41950) mlflow doctest fails for pandas API on SPark

[ https://issues.apache.org/jira/browse/SPARK-41950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41950: Assignee: Apache Spark > mlflow doctest fails for pandas API on SPark > -

[jira] [Commented] (SPARK-41950) mlflow doctest fails for pandas API on SPark

[ https://issues.apache.org/jira/browse/SPARK-41950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656097#comment-17656097 ] Apache Spark commented on SPARK-41950: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-41950) mlflow doctest fails for pandas API on SPark

[ https://issues.apache.org/jira/browse/SPARK-41950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41950: Assignee: (was: Apache Spark) > mlflow doctest fails for pandas API on SPark > --

[jira] [Updated] (SPARK-41950) mlflow doctest fails for pandas API on SPark

[ https://issues.apache.org/jira/browse/SPARK-41950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41950: - Issue Type: Test (was: Bug) > mlflow doctest fails for pandas API on SPark > --

[jira] [Created] (SPARK-41950) mlflow doctest fails for pandas API on SPark

Hyukjin Kwon created SPARK-41950: Summary: mlflow doctest fails for pandas API on SPark Key: SPARK-41950 URL: https://issues.apache.org/jira/browse/SPARK-41950 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-41944) Pass configurations when local remote mode is on

[ https://issues.apache.org/jira/browse/SPARK-41944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41944: Assignee: Hyukjin Kwon > Pass configurations when local remote mode is on > -

[jira] [Resolved] (SPARK-41944) Pass configurations when local remote mode is on

[ https://issues.apache.org/jira/browse/SPARK-41944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41944. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39463 [https://gi

[jira] [Assigned] (SPARK-41945) Python: connect client lost column data with pyarrow.Table.to_pylist

[ https://issues.apache.org/jira/browse/SPARK-41945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41945: - Assignee: jiaan.geng > Python: connect client lost column data with pyarrow.Table.to_py

[jira] [Resolved] (SPARK-41945) Python: connect client lost column data with pyarrow.Table.to_pylist

[ https://issues.apache.org/jira/browse/SPARK-41945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41945. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39461 [https://

[jira] [Updated] (SPARK-41842) Support data type Timestamp(NANOSECOND, null)

[ https://issues.apache.org/jira/browse/SPARK-41842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41842: - Fix Version/s: (was: 3.4.0) > Support data type Timestamp(NANOSECOND, null) > --

[jira] [Reopened] (SPARK-41842) Support data type Timestamp(NANOSECOND, null)

[ https://issues.apache.org/jira/browse/SPARK-41842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-41842: -- Let me reopen this since https://github.com/apache/spark/blob/56c7cf33929d7d42b7d299c0bb7e8959632

[jira] [Commented] (SPARK-41741) [SQL] ParquetFilters StringStartsWith push down matching string do not use UTF-8

[ https://issues.apache.org/jira/browse/SPARK-41741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656023#comment-17656023 ] Jiale He commented on SPARK-41741: -- [~yumwang]  Do you mean this? !image-2023-01-09-1

[jira] [Updated] (SPARK-41741) [SQL] ParquetFilters StringStartsWith push down matching string do not use UTF-8

[ https://issues.apache.org/jira/browse/SPARK-41741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale He updated SPARK-41741: - Attachment: image-2023-01-09-18-27-53-479.png > [SQL] ParquetFilters StringStartsWith push down matching

[jira] [Assigned] (SPARK-41949) Make stage scheduling support local-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-41949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41949: Assignee: (was: Apache Spark) > Make stage scheduling support local-cluster mode > --

[jira] [Commented] (SPARK-41949) Make stage scheduling support local-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-41949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655989#comment-17655989 ] Apache Spark commented on SPARK-41949: -- User 'WeichenXu123' has created a pull requ

[jira] [Assigned] (SPARK-41949) Make stage scheduling support local-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-41949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41949: Assignee: Apache Spark > Make stage scheduling support local-cluster mode > -

[jira] [Created] (SPARK-41949) Make stage scheduling support local-cluster mode

Weichen Xu created SPARK-41949: -- Summary: Make stage scheduling support local-cluster mode Key: SPARK-41949 URL: https://issues.apache.org/jira/browse/SPARK-41949 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-41948) Fix NPE for error classes: CANNOT_PARSE_JSON_FIELD

[ https://issues.apache.org/jira/browse/SPARK-41948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655982#comment-17655982 ] Apache Spark commented on SPARK-41948: -- User 'panbingkun' has created a pull reques

[jira] [Commented] (SPARK-41948) Fix NPE for error classes: CANNOT_PARSE_JSON_FIELD

[ https://issues.apache.org/jira/browse/SPARK-41948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655981#comment-17655981 ] Apache Spark commented on SPARK-41948: -- User 'panbingkun' has created a pull reques

[jira] [Assigned] (SPARK-41948) Fix NPE for error classes: CANNOT_PARSE_JSON_FIELD

[ https://issues.apache.org/jira/browse/SPARK-41948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41948: Assignee: (was: Apache Spark) > Fix NPE for error classes: CANNOT_PARSE_JSON_FIELD >

[jira] [Assigned] (SPARK-41948) Fix NPE for error classes: CANNOT_PARSE_JSON_FIELD

[ https://issues.apache.org/jira/browse/SPARK-41948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41948: Assignee: Apache Spark > Fix NPE for error classes: CANNOT_PARSE_JSON_FIELD > ---

[jira] [Assigned] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41780: Assignee: BingKun Pan > `regexp_replace('', '[ad]{0, 2}', 'x')` causes an internal error > --

[jira] [Resolved] (SPARK-41780) `regexp_replace('', '[a\\\\d]{0, 2}', 'x')` causes an internal error

[ https://issues.apache.org/jira/browse/SPARK-41780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41780. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39383 [https://github.com

[jira] [Created] (SPARK-41948) Fix NPE for error classes: CANNOT_PARSE_JSON_FIELD

BingKun Pan created SPARK-41948: --- Summary: Fix NPE for error classes: CANNOT_PARSE_JSON_FIELD Key: SPARK-41948 URL: https://issues.apache.org/jira/browse/SPARK-41948 Project: Spark Issue Type:

[jira] (SPARK-41945) Python: connect client lost column data with pyarrow.Table.to_pylist

[ https://issues.apache.org/jira/browse/SPARK-41945 ] jiaan.geng deleted comment on SPARK-41945: was (Author: beliefer): I'm working on. > Python: connect client lost column data with pyarrow.Table.to_pylist > --

[jira] [Updated] (SPARK-41945) Python: connect client lost column data with pyarrow.Table.to_pylist

[ https://issues.apache.org/jira/browse/SPARK-41945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-41945: --- Description: Python: connect client should not use pyarrow.Table.to_pylist to transform fetched dat