[jira] [Assigned] (SPARK-41417) Assign a name to the error class _LEGACY_ERROR_TEMP_0019

2022-12-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-41417: Assignee: Yang Jie > Assign a name to the error class _LEGACY_ERROR_TEMP_0019 > -

[jira] [Resolved] (SPARK-41417) Assign a name to the error class _LEGACY_ERROR_TEMP_0019

2022-12-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41417. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38954 [https://github.com

[jira] [Assigned] (SPARK-41467) Upgrade httpclient from 4.5.13 to 4.5.14

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41467: - Assignee: BingKun Pan > Upgrade httpclient from 4.5.13 to 4.5.14 >

[jira] [Resolved] (SPARK-41467) Upgrade httpclient from 4.5.13 to 4.5.14

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41467. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39005 [https://

[jira] [Commented] (SPARK-41477) Correctly infer the datatype of literal integers

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645557#comment-17645557 ] Apache Spark commented on SPARK-41477: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-41477) Correctly infer the datatype of literal integers

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41477: Assignee: Apache Spark > Correctly infer the datatype of literal integers > -

[jira] [Assigned] (SPARK-41477) Correctly infer the datatype of literal integers

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41477: Assignee: (was: Apache Spark) > Correctly infer the datatype of literal integers > --

[jira] [Commented] (SPARK-41477) Correctly infer the datatype of literal integers

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645556#comment-17645556 ] Apache Spark commented on SPARK-41477: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-41477) Correctly infer the datatype of literal integers

2022-12-09 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-41477: - Summary: Correctly infer the datatype of literal integers Key: SPARK-41477 URL: https://issues.apache.org/jira/browse/SPARK-41477 Project: Spark Issue Type

[jira] [Assigned] (SPARK-41457) Refactor pandas, pyarrow and grpc check in tests

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41457: - Assignee: Hyukjin Kwon > Refactor pandas, pyarrow and grpc check in tests > ---

[jira] [Resolved] (SPARK-41457) Refactor pandas, pyarrow and grpc check in tests

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41457. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38991 [https://

[jira] [Resolved] (SPARK-41475) Fix lint-scala command error

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41475. --- Resolution: Fixed Issue resolved by pull request 39012 [https://github.com/apache/spark/pull

[jira] [Assigned] (SPARK-41475) Fix lint-scala command error

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41475: - Assignee: Deng Ziming > Fix lint-scala command error > > >

[jira] [Assigned] (SPARK-41474) Exclude proto files from spark-protobuf

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41474: - Assignee: Dongjoon Hyun > Exclude proto files from spark-protobuf > ---

[jira] [Resolved] (SPARK-41474) Exclude proto files from spark-protobuf

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41474. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39014 [https://

[jira] [Commented] (SPARK-41476) Prevent `README.md` from triggering CIs

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645537#comment-17645537 ] Apache Spark commented on SPARK-41476: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-41476) Prevent `README.md` from triggering CIs

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645538#comment-17645538 ] Apache Spark commented on SPARK-41476: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-41476) Prevent `README.md` from triggering CIs

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41476: Assignee: (was: Apache Spark) > Prevent `README.md` from triggering CIs > ---

[jira] [Assigned] (SPARK-41476) Prevent `README.md` from triggering CIs

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41476: Assignee: Apache Spark > Prevent `README.md` from triggering CIs > --

[jira] [Created] (SPARK-41476) Prevent `README.md` from triggering CIs

2022-12-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-41476: - Summary: Prevent `README.md` from triggering CIs Key: SPARK-41476 URL: https://issues.apache.org/jira/browse/SPARK-41476 Project: Spark Issue Type: Improve

[jira] [Resolved] (SPARK-41414) Implement date/timestamp functions

2022-12-09 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41414. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38946 [https://

[jira] [Assigned] (SPARK-41414) Implement date/timestamp functions

2022-12-09 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41414: - Assignee: Xinrong Meng > Implement date/timestamp functions > -

[jira] [Assigned] (SPARK-41475) Fix lint-scala command error

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41475: Assignee: (was: Apache Spark) > Fix lint-scala command error > --

[jira] [Commented] (SPARK-41475) Fix lint-scala command error

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645530#comment-17645530 ] Apache Spark commented on SPARK-41475: -- User 'dengziming' has created a pull reques

[jira] [Assigned] (SPARK-41475) Fix lint-scala command error

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41475: Assignee: Apache Spark > Fix lint-scala command error > > >

[jira] [Created] (SPARK-41475) Fix lint-scala command error

2022-12-09 Thread Deng Ziming (Jira)
Deng Ziming created SPARK-41475: --- Summary: Fix lint-scala command error Key: SPARK-41475 URL: https://issues.apache.org/jira/browse/SPARK-41475 Project: Spark Issue Type: Bug Componen

[jira] [Assigned] (SPARK-41474) Exclude proto files from spark-protobuf

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41474: Assignee: Apache Spark > Exclude proto files from spark-protobuf > --

[jira] [Commented] (SPARK-41474) Exclude proto files from spark-protobuf

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645529#comment-17645529 ] Apache Spark commented on SPARK-41474: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-41474) Exclude proto files from spark-protobuf

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41474: Assignee: (was: Apache Spark) > Exclude proto files from spark-protobuf > ---

[jira] [Created] (SPARK-41474) Exclude proto files from spark-protobuf

2022-12-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-41474: - Summary: Exclude proto files from spark-protobuf Key: SPARK-41474 URL: https://issues.apache.org/jira/browse/SPARK-41474 Project: Spark Issue Type: Improve

[jira] [Resolved] (SPARK-41329) Solve circular import between Column and _typing/functions

2022-12-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41329. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38994 [https://gi

[jira] [Assigned] (SPARK-41329) Solve circular import between Column and _typing/functions

2022-12-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41329: Assignee: Rui Wang > Solve circular import between Column and _typing/functions > ---

[jira] [Assigned] (SPARK-41329) Solve circular import between Column and _typing/functions

2022-12-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41329: Assignee: Hyukjin Kwon (was: Rui Wang) > Solve circular import between Column and _typin

[jira] [Closed] (SPARK-41451) Avoid using empty abbrevMarker in StringUtils.abbreviate

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-41451. - > Avoid using empty abbrevMarker in StringUtils.abbreviate > ---

[jira] [Resolved] (SPARK-41451) Avoid using empty abbrevMarker in StringUtils.abbreviate

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41451. --- Resolution: Not A Problem It turns out that Apache Spark is not affected by LANG-1463 becaus

[jira] [Resolved] (SPARK-41456) Improve the performance of try_cast

2022-12-09 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-41456. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38988 [https:

[jira] [Created] (SPARK-41473) Implement `functions.format_number`

2022-12-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41473: Summary: Implement `functions.format_number` Key: SPARK-41473 URL: https://issues.apache.org/jira/browse/SPARK-41473 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-41472) Implement the rest of string/binary functions

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645507#comment-17645507 ] Apache Spark commented on SPARK-41472: -- User 'xinrong-meng' has created a pull requ

[jira] [Commented] (SPARK-41472) Implement the rest of string/binary functions

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645505#comment-17645505 ] Apache Spark commented on SPARK-41472: -- User 'xinrong-meng' has created a pull requ

[jira] [Assigned] (SPARK-41472) Implement the rest of string/binary functions

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41472: Assignee: Apache Spark > Implement the rest of string/binary functions >

[jira] [Assigned] (SPARK-41472) Implement the rest of string/binary functions

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41472: Assignee: (was: Apache Spark) > Implement the rest of string/binary functions > -

[jira] [Assigned] (SPARK-41283) Feature parity: Functions API in Spark Connect

2022-12-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-41283: Assignee: Ruifeng Zheng (was: Xinrong Meng) > Feature parity: Functions API in Spark Con

[jira] [Created] (SPARK-41472) Implement the rest of string/binary functions

2022-12-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41472: Summary: Implement the rest of string/binary functions Key: SPARK-41472 URL: https://issues.apache.org/jira/browse/SPARK-41472 Project: Spark Issue Type: Sub

[jira] [Updated] (SPARK-41376) Executor netty direct memory check should respect spark.shuffle.io.preferDirectBufs

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-41376: -- Fix Version/s: 3.2.4 3.3.2 > Executor netty direct memory check should resp

[jira] [Assigned] (SPARK-41466) Change Scala Style configuration to catch AnyFunSuite instead of FunSuite

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41466: - Assignee: Hyukjin Kwon > Change Scala Style configuration to catch AnyFunSuite instead

[jira] [Resolved] (SPARK-41466) Change Scala Style configuration to catch AnyFunSuite instead of FunSuite

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41466. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39004 [https://

[jira] [Resolved] (SPARK-41462) Date and timestamp type can up cast to TimestampNTZ

2022-12-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-41462. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38997 [https://github.com

[jira] [Updated] (SPARK-41470) SPJ: Spark shouldn't assume InternalRow implements equals and hashCode

2022-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-41470: - Summary: SPJ: Spark shouldn't assume InternalRow implements equals and hashCode (was: SPJ shouldn't ass

[jira] [Updated] (SPARK-41471) SPJ: Reduce Spark shuffle when only one side of a join is KeyGroupedPartitioning

2022-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-41471: - Summary: SPJ: Reduce Spark shuffle when only one side of a join is KeyGroupedPartitioning (was: SPJ: re

[jira] [Updated] (SPARK-40946) SPJ: Introduce a new DataSource V2 interface SupportsPushDownClusterKeys

2022-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-40946: - Summary: SPJ: Introduce a new DataSource V2 interface SupportsPushDownClusterKeys (was: Introduce a new

[jira] [Updated] (SPARK-41398) SPJ: Relax constraints on Storage-Partitioned Join when partition keys after runtime filtering do not match

2022-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-41398: - Summary: SPJ: Relax constraints on Storage-Partitioned Join when partition keys after runtime filtering

[jira] [Updated] (SPARK-37375) Umbrella: Storage Partitioned Join (SPJ)

2022-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-37375: - Summary: Umbrella: Storage Partitioned Join (SPJ) (was: Umbrella: Storage Partitioned Join) > Umbrella

[jira] [Updated] (SPARK-41413) SPJ: Spark should avoid shuffle when partition keys mismatch, but join expressions are compatible

2022-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-41413: - Summary: SPJ: Spark should avoid shuffle when partition keys mismatch, but join expressions are compatib

[jira] [Updated] (SPARK-41413) SPJ: Avoid shuffle when partition keys mismatch, but join expressions are compatible

2022-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-41413: - Summary: SPJ: Avoid shuffle when partition keys mismatch, but join expressions are compatible (was: SPJ

[jira] [Updated] (SPARK-37377) SPJ: Initial implementation of Storage-Partitioned Join

2022-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-37377: - Summary: SPJ: Initial implementation of Storage-Partitioned Join (was: Initial implementation of Storag

[jira] [Created] (SPARK-41471) SPJ: reduce Spark shuffle when only one side of a join is KeyGroupedPartitioning

2022-12-09 Thread Chao Sun (Jira)
Chao Sun created SPARK-41471: Summary: SPJ: reduce Spark shuffle when only one side of a join is KeyGroupedPartitioning Key: SPARK-41471 URL: https://issues.apache.org/jira/browse/SPARK-41471 Project: Spa

[jira] [Updated] (SPARK-37378) SPJ: Convert V2 Transform expressions into catalyst expressions and load their associated functions from V2 FunctionCatalog

2022-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-37378: - Summary: SPJ: Convert V2 Transform expressions into catalyst expressions and load their associated funct

[jira] [Updated] (SPARK-37376) SPJ: Introduce a new DataSource V2 interface HasPartitionKey

2022-12-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-37376: - Summary: SPJ: Introduce a new DataSource V2 interface HasPartitionKey (was: Introduce a new DataSource

[jira] [Created] (SPARK-41470) SPJ shouldn't assume InternalRow implements equals and hashCode

2022-12-09 Thread Chao Sun (Jira)
Chao Sun created SPARK-41470: Summary: SPJ shouldn't assume InternalRow implements equals and hashCode Key: SPARK-41470 URL: https://issues.apache.org/jira/browse/SPARK-41470 Project: Spark Issu

[jira] [Resolved] (SPARK-41450) PySpark built from master code raise error "java.lang.ClassNotFoundException: org.eclipse.jetty.server.Handler"

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41450. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38999 [https://

[jira] [Assigned] (SPARK-41450) PySpark built from master code raise error "java.lang.ClassNotFoundException: org.eclipse.jetty.server.Handler"

2022-12-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41450: - Assignee: Weichen Xu > PySpark built from master code raise error "java.lang.ClassNotFo

[jira] [Assigned] (SPARK-41469) Task rerun on decommissioned executor can be avoided if shuffle data has migrated

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41469: Assignee: Apache Spark > Task rerun on decommissioned executor can be avoided if shuffle

[jira] [Commented] (SPARK-41469) Task rerun on decommissioned executor can be avoided if shuffle data has migrated

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645344#comment-17645344 ] Apache Spark commented on SPARK-41469: -- User 'Ngone51' has created a pull request f

[jira] [Assigned] (SPARK-41469) Task rerun on decommissioned executor can be avoided if shuffle data has migrated

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41469: Assignee: (was: Apache Spark) > Task rerun on decommissioned executor can be avoided

[jira] [Created] (SPARK-41469) Task rerun on decommissioned executor can be avoided if shuffle data has migrated

2022-12-09 Thread wuyi (Jira)
wuyi created SPARK-41469: Summary: Task rerun on decommissioned executor can be avoided if shuffle data has migrated Key: SPARK-41469 URL: https://issues.apache.org/jira/browse/SPARK-41469 Project: Spark

[jira] [Assigned] (SPARK-41402) Override nodeName of StringDecode

2022-12-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41402: Assignee: Xinrong Meng > Override nodeName of StringDecode >

[jira] [Resolved] (SPARK-41402) Override nodeName of StringDecode

2022-12-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41402. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39001 [https://gi

[jira] [Updated] (SPARK-41458) Correctly transform the SPI services for Yarn Shuffle Service

2022-12-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-41458: - Priority: Minor (was: Major) > Correctly transform the SPI services for Yarn Shuffle Service >

[jira] [Resolved] (SPARK-41458) Correctly transform the SPI services for Yarn Shuffle Service

2022-12-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-41458. -- Fix Version/s: 3.3.2 3.4.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-41458) Correctly transform the SPI services for Yarn Shuffle Service

2022-12-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-41458: Assignee: Cheng Pan > Correctly transform the SPI services for Yarn Shuffle Service > ---

[jira] [Assigned] (SPARK-41446) Make `createDataFrame` support schema and more input dataset types

2022-12-09 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41446: - Assignee: Ruifeng Zheng > Make `createDataFrame` support schema and more input dataset

[jira] [Resolved] (SPARK-41446) Make `createDataFrame` support schema and more input dataset types

2022-12-09 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41446. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38979 [https://

[jira] [Commented] (SPARK-41468) Fix PlanExpression handling in EquivalentExpressions

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645290#comment-17645290 ] Apache Spark commented on SPARK-41468: -- User 'peter-toth' has created a pull reques

[jira] [Assigned] (SPARK-41468) Fix PlanExpression handling in EquivalentExpressions

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41468: Assignee: (was: Apache Spark) > Fix PlanExpression handling in EquivalentExpressions

[jira] [Commented] (SPARK-41468) Fix PlanExpression handling in EquivalentExpressions

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645289#comment-17645289 ] Apache Spark commented on SPARK-41468: -- User 'peter-toth' has created a pull reques

[jira] [Assigned] (SPARK-41468) Fix PlanExpression handling in EquivalentExpressions

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41468: Assignee: Apache Spark > Fix PlanExpression handling in EquivalentExpressions > -

[jira] [Updated] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41395: - Fix Version/s: 3.2.3 3.3.2 > InterpretedMutableProjection can corrupt unsafe

[jira] [Assigned] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-41395: Assignee: Bruce Robbins > InterpretedMutableProjection can corrupt unsafe buffer when use

[jira] [Resolved] (SPARK-41395) InterpretedMutableProjection can corrupt unsafe buffer when used with decimal data

2022-12-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41395. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38923 [https://gi

[jira] [Commented] (SPARK-41225) Disable unsupported functions

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645281#comment-17645281 ] Apache Spark commented on SPARK-41225: -- User 'grundprinzip' has created a pull requ

[jira] [Created] (SPARK-41468) Fix PlanExpression handling in EquivalentExpressions

2022-12-09 Thread Peter Toth (Jira)
Peter Toth created SPARK-41468: -- Summary: Fix PlanExpression handling in EquivalentExpressions Key: SPARK-41468 URL: https://issues.apache.org/jira/browse/SPARK-41468 Project: Spark Issue Type:

[jira] [Commented] (SPARK-40270) Make compute.max_rows as None working in DataFrame.style

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645263#comment-17645263 ] Apache Spark commented on SPARK-40270: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-41392) spark builds against hadoop trunk/3.4.0-SNAPSHOT fail in scala-maven plugin

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645261#comment-17645261 ] Apache Spark commented on SPARK-41392: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-41392) spark builds against hadoop trunk/3.4.0-SNAPSHOT fail in scala-maven plugin

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41392: Assignee: (was: Apache Spark) > spark builds against hadoop trunk/3.4.0-SNAPSHOT fail

[jira] [Assigned] (SPARK-41392) spark builds against hadoop trunk/3.4.0-SNAPSHOT fail in scala-maven plugin

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41392: Assignee: Apache Spark > spark builds against hadoop trunk/3.4.0-SNAPSHOT fail in scala-m

[jira] [Commented] (SPARK-41392) spark builds against hadoop trunk/3.4.0-SNAPSHOT fail in scala-maven plugin

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645260#comment-17645260 ] Apache Spark commented on SPARK-41392: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-40270) Make compute.max_rows as None working in DataFrame.style

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645258#comment-17645258 ] Apache Spark commented on SPARK-40270: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-40270) Make compute.max_rows as None working in DataFrame.style

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645257#comment-17645257 ] Apache Spark commented on SPARK-40270: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-41450) PySpark built from master code raise error "java.lang.ClassNotFoundException: org.eclipse.jetty.server.Handler"

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645256#comment-17645256 ] Apache Spark commented on SPARK-41450: -- User 'pan3793' has created a pull request f

[jira] [Assigned] (SPARK-41450) PySpark built from master code raise error "java.lang.ClassNotFoundException: org.eclipse.jetty.server.Handler"

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41450: Assignee: Apache Spark > PySpark built from master code raise error "java.lang.ClassNotFo

[jira] [Assigned] (SPARK-41450) PySpark built from master code raise error "java.lang.ClassNotFoundException: org.eclipse.jetty.server.Handler"

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41450: Assignee: (was: Apache Spark) > PySpark built from master code raise error "java.lang

[jira] [Commented] (SPARK-41233) High-order function: array_prepend

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645255#comment-17645255 ] Apache Spark commented on SPARK-41233: -- User 'infoankitp' has created a pull reques

[jira] [Commented] (SPARK-41233) High-order function: array_prepend

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645254#comment-17645254 ] Apache Spark commented on SPARK-41233: -- User 'infoankitp' has created a pull reques

[jira] [Assigned] (SPARK-41467) Upgrade httpclient from 4.5.13 to 4.5.14

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41467: Assignee: (was: Apache Spark) > Upgrade httpclient from 4.5.13 to 4.5.14 > --

[jira] [Assigned] (SPARK-41467) Upgrade httpclient from 4.5.13 to 4.5.14

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41467: Assignee: Apache Spark > Upgrade httpclient from 4.5.13 to 4.5.14 > -

[jira] [Commented] (SPARK-41467) Upgrade httpclient from 4.5.13 to 4.5.14

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645248#comment-17645248 ] Apache Spark commented on SPARK-41467: -- User 'panbingkun' has created a pull reques

[jira] [Created] (SPARK-41467) Upgrade httpclient from 4.5.13 to 4.5.14

2022-12-09 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-41467: --- Summary: Upgrade httpclient from 4.5.13 to 4.5.14 Key: SPARK-41467 URL: https://issues.apache.org/jira/browse/SPARK-41467 Project: Spark Issue Type: Improvemen

[jira] [Commented] (SPARK-41466) Change Scala Style configuration to catch AnyFunSuite instead of FunSuite

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17645229#comment-17645229 ] Apache Spark commented on SPARK-41466: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-41466) Change Scala Style configuration to catch AnyFunSuite instead of FunSuite

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41466: Assignee: Apache Spark > Change Scala Style configuration to catch AnyFunSuite instead of

[jira] [Assigned] (SPARK-41466) Change Scala Style configuration to catch AnyFunSuite instead of FunSuite

2022-12-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-41466: Assignee: (was: Apache Spark) > Change Scala Style configuration to catch AnyFunSuite

  1   2   >